We closed the arrays lesson with two debts: ${amount%.*}, which trimmed the cents, and ${SHIPMENTS[$c]:-0}, which supplied a default value. You already know the second from 03-06; the first belongs to a much bigger family. Bash can cut, trim, substitute and transform strings without calling any external program, using the same old ${...} syntax. This matters for two reasons: the code comes out shorter, and above all it is ten to a hundred times faster, because every sed or cut inside a loop is a brand-new operating system process. In this lesson you complete the whole family and learn to format the output of daily-report.sh with printf.
Contents
- Length and substrings
- Removing prefixes and suffixes
- Text substitution
- Uppercase and lowercase
- Concatenation
- Comparison and pattern matching
- Splitting a string into an array
printf: widths, alignment and decimals- Expansions versus
sed,cut,basename - Integrating example
- Length and substrings
s="2026-08-03 10:15:22 [ERROR] timeout contacting the payment gateway"
echo "${#s}" # 66 → number of characters
echo "${s:0:10}" # 2026-08-03 → from position 0, 10 characters
echo "${s:11:8}" # 10:15:22 → from 11, 8 characters
echo "${s:20}" # [ERROR] timeout ... → from 20 to the end
echo "${s: -7}" # gateway → the last 7 (careful: space before the -7)
echo "${s:0:-7}" # everything except the last 7The syntax is ${string:offset:length}, with the offset starting at 0. Three practical notes: the space before a negative number is mandatory, because without it ${s:-7} means "default value 7", a completely different expansion (${s:(-7)} also works); both numbers accept arithmetic, like ${s:i:n} or ${s:0:${#s}-4}; and it counts characters, not bytes, as long as the locale is UTF-8, so an ñ counts as 1. It works on arguments too: ${1:0:4} is the first four characters of $1.
- Removing prefixes and suffixes
Four operators that trim whatever matches a glob pattern (the ones from 02-05: *, ?, [...]), not a regular expression:
| Expansion | Removes from | Match |
|---|---|---|
${s#pattern} |
The beginning | The shortest |
${s##pattern} |
The beginning | The longest |
${s%pattern} |
The end | The shortest |
${s%%pattern} |
The end | The longest |
Mnemonic, looking at a keyboard: # is to the left of $ and trims from the left; % is to the right and trims from the right. And doubling the symbol means "eat as much as you can".
path="/var/log/veloz/app.log.3.gz"
echo "${path##*/}" # app.log.3.gz → file name (removes up to the last /)
echo "${path#*/}" # var/log/... → removes only up to the FIRST /
echo "${path%/*}" # /var/log/veloz → directory (removes from the last /)
echo "${path%%.*}" # /var/log/veloz/app → removes from the first dot
echo "${path%.*}" # /var/log/veloz/app.log.3 → removes only the last extensionApplied to our data:
amount="24.50"
echo "${amount%.*}" # 24 → integer part (the debt from 04-03)
echo "${amount#*.}" # 50 → cents
f="app.log.3.gz"
echo "${f%%.*}" # app → base name
echo "${f##*.}" # gz → extension
line="2026-08-03 10:15:22 [ERROR] timeout contacting"
rest="${line#* }" # removes the date
hour="${rest%% *}" # 10:15:22
level="${line#*\[}"; level="${level%%\]*}" # ERROR
message="${line#*] }" # timeout contactingThat final block extracts date, time, level and message from an app.log line without launching a single process. The \\[ and \\] are escaped because in a glob pattern brackets define a character class.
- Text substitution
s="Valencia,Sevilla,Bilbao,Madrid"
echo "${s/,/ }" # Valencia Sevilla,Bilbao,Madrid → only the FIRST
echo "${s//,/ }" # Valencia Sevilla Bilbao Madrid → ALL of them
echo "${s//,}" # ValenciaSevillaBilbaoMadrid → deletes (empty replacement)
echo "${s/#Valencia/VLC}" # VLC,Sevilla,... → only if it is at the BEGINNING
echo "${s/%Madrid/MAD}" # ...,Sevilla,Bilbao,MAD → only if it is at the ENDSummary: one slash substitutes the first match, two slashes all of them, /# anchors at the beginning and /% at the end. The search pattern is once again a glob, which allows things like:
echo "${path//\//-}" # the / is escaped as \/
name=" alopez "
name="${name#"${name%%[![:space:]]*}"}" # removes leading spaces
name="${name%"${name##*[![:space:]]}"}" # removes trailing spaces
echo "[$name]" # [alopez]That idiom for trimming spaces is ugly but it is the standard in pure Bash; save it as a trim() function in your library.
- Uppercase and lowercase
Four operators, available since Bash 4:
city="valencia"
echo "${city^}" # Valencia → first letter to uppercase
echo "${city^^}" # VALENCIA → all to uppercase
c="MADRID"
echo "${c,}" # mADRID → first letter to lowercase
echo "${c,,}" # madrid → all to lowercaseThe most frequent use is normalizing before comparing, so the user can type VALENCIA, valencia or Valencia: city="${1,,}"; city="${city^}" always yields Valencia. It can be restricted to certain characters with a pattern (${s^^[aeiou]} uppercases only the vowels), although that is rare.
- Concatenation
Bash has no concatenation operator: you just write the strings one after another.
base="/var/log/veloz"; file="app.log"
path="$base/$file" # /var/log/veloz/app.log
prefix="report"
name="${prefix}_$(date +%F).txt" # the braces separate the name from the text after itThe braces in ${prefix}_ are mandatory here: without them Bash would look for a variable called prefix_. And to accumulate text in a loop:
list=""
for city in Valencia Sevilla Bilbao; do list+="$city, "; done # += appends at the end
echo "${list%, }" # Valencia, Sevilla, Bilbao (removes the trailing comma)+= on a string concatenates; on an array (04-03) it appends elements; on a declare -i variable it adds. The same operator, three behaviors depending on the type.
- Comparison and pattern matching
Inside [[ ]], the == operator does more than compare: if the right-hand side is unquoted, it is interpreted as a glob pattern.
f="app.log.3.gz"
[[ "$f" == app.log.* ]] # TRUE → pattern match
[[ "$f" == *.gz ]] # TRUE → ends in .gz
[[ "$f" == "app.log.*" ]] # FALSE → with quotes, it is literal
[[ "$f" == ?pp* ]] # TRUE → ? is any single character
pattern="*.gz"
[[ "$f" == $pattern ]] # TRUE → the unquoted variable acts as a pattern
[[ "$f" == "$pattern" ]] # FALSE → with quotes, it compares against the text "*.gz"This is the only situation in the course where dropping the quotes is deliberate and correct: when you want the variable's content to work as a pattern. In any other context, quotes.
Other string operators in [[ ]], already seen in 03-03: !=, < and > (alphabetical order according to the locale), -z (empty) and -n (non-empty).
For matches a glob cannot express —"three digits followed by a letter", "a valid IP"— you need the =~ operator with regular expressions. That is the full topic of 05-04; for now, if the pattern can be written with * and ?, use ==.
- Splitting a string into an array
You already saw this in 04-03; we bring it back here because it is a string operation in its own right:
line="E1001,2026-08-03,Valencia,alopez,delivered,24.50"
IFS=',' read -r -a fields <<< "$line" # the recommended form
echo "${fields[2]}" # Valencia
IFS=',' ; fields=( $line ) ; unset IFS # shorter, but applies globbing: dangerousThe second variant fails if a field contains *, because globbing will expand it into file names. The inverse operation, joining an array into a string, is done with "${arr[*]}" and the right IFS:
cities=(Valencia Sevilla Bilbao)
IFS=' | '; echo "${cities[*]}"; unset IFS # Valencia | Sevilla | BilbaoNote: only the first character of IFS is used as the separator in ${arr[*]}, so the example above joins with a space. For multi-character separators, use printf or a loop.
printf: widths, alignment and decimals
printf: widths, alignment and decimalsYou already know from 03-06 that printf is preferable to echo. Here we exploit its specifiers to build tables:
| Specifier | Meaning | Example → output |
|---|---|---|
%s |
String | printf '%s' hello → hello |
%-10s |
String, width 10, left-aligned | hello |
%10s |
String, width 10, right-aligned | hello |
%5d |
Integer, width 5, right-aligned | 128 |
%05d |
Integer, zero-padded | 00128 |
%.2f |
Decimal with 2 digits (rounds) | 24.50 |
%8.2f |
Decimal, width 8, 2 digits | 24.50 |
%% |
A literal percent sign | % |
printf '%-10s %8s %13s %9s\n' CITY SHIPMENTS ISSUES AMOUNT
printf '%-10s %8d %13d %8.2f€\n' Valencia 128 11 3204.55
# CITY SHIPMENTS ISSUES AMOUNT
# Valencia 128 11 3204.55€Three printf behaviors worth knowing: it reuses the format while arguments remain, so printf '%s\n' "${cities[@]}" prints each city on its own line with no loop needed; the width can come from a variable using *, as in printf '%-*s\n' 15 "$city"; and %.2f rounds, it does not truncate (printf '%.2f' 2.005 gives 2.01), accepting decimals even though Bash cannot compute them, because the formatting is done by printf and not by the shell's arithmetic (04-06). With accented characters the width is counted in characters if the locale is UTF-8; if your columns wobble, check LANG.
- Expansions versus
sed, cut, basename
sed, cut, basenameEverything in this lesson can also be done with external tools. The difference is the cost:
| Task | With an external tool | With an expansion |
|---|---|---|
| File name | basename "$path" |
"${path##*/}" |
| Directory | dirname "$path" |
"${path%/*}" |
| Strip extension | basename "$f" .csv |
"${f%.csv}" |
| Field 3 of a CSV | cut -d, -f3 <<< "$l" |
read -r -a c <<< "$l"; "${c[2]}" |
| Substitute text | sed 's/,/;/g' <<< "$s" |
"${s//,/;}" |
| To uppercase | tr a-z A-Z <<< "$s" |
"${s^^}" |
Every left-hand column launches a new process: the kernel does a fork, loads the binary, sets up a pipe and waits. That is a few milliseconds... multiplied by the number of iterations.
while IFS= read -r l; do echo "$l" | cut -d, -f3; done < shipments.csv # 5,000 lines: ~12 s
while IFS=, read -r _ _ city _; do echo "$city"; done < shipments.csv # the same: ~0.3 sThe rule of thumb: inside a loop, expansion; outside a loop, whichever tool reads better. And if the job is to process a whole file at once, awk in a single pass almost always wins (06-01); we will come back to this when we measure for real in 08-02. One nuance in favor of basename/dirname: they handle the odd cases (paths ending in /, ., ..) according to the standard, whereas "${path%/*}" on a file.txt with no slashes returns file.txt and not .. For paths you control, the expansion is perfect; for arbitrary user paths, use the tools.
- Integrating example
An app.log analyzer that extracts the fields with expansions and formats a table with printf, with no cut, no awk and no sed:
# log_summary — Table of errors per hour from app.log.
# Usage: log_summary <file> <date> Returns: 0 | 3 if the file is not readable
log_summary() {
local file="${1:?}" report_date="${2:?}" line rest hour level h
declare -A by_hour
[[ -r "$file" ]] || return 3
while IFS= read -r line; do
[[ "$line" == "$report_date"* ]] || continue # glob pattern: starts with the date
rest="${line#* }" # removes "2026-08-03 "
hour="${rest%%:*}" # "10"
level="${line#*[}"; level="${level%%]*}" # "ERROR"
[[ "$level" == "ERROR" ]] && (( by_hour["$hour"]++ ))
done < "$file"
printf '%-6s %8s\n' HOUR ERRORS
for h in $(printf '%s\n' "${!by_hour[@]}" | sort); do
printf '%02d:00 %8d\n' "$(( 10#$h ))" "${by_hour[$h]}"
done
}Output: a HOUR / ERRORS table with lines like 10:00 14. Notes on the new parts:
[[ "$line" == "$report_date"* ]]: the variable is quoted and the*is outside, so the date is compared literally and anything is accepted after it. It is the correct idiom for "starts with".${rest%%:*}cuts at the first:, leaving the hour. With%instead of%%we would have cut at the last one and got10:15.$(( 10#$h ))forces base-10 interpretation. An hour like08would be read as invalid octal and raise an error; it is a classic trap that we dismantle completely in 04-06.
And in daily-report.sh, the formatting function fits on one line: city_row() { printf '%-10s %8d %13d %10.2f€\n' "${1^}" "$2" "$3" "$4"; }. The ${1^} normalizes the city's initial and the fixed widths guarantee the columns line up whatever the data looks like.
Common Mistakes and Tips
- Forgetting the space in
${s: -3}. Without it,${s:-3}is "default value", a different expansion that will not raise an error but will return something else. - Confusing
#/##with%/%%. Always test with anechobefore putting it in the script; the difference between the shortest and the longest match breaks paths with several dots. - Believing that the patterns are regexes.
${s#[0-9]+}does not work:+is not a quantifier in globbing. For regexes,=~in 05-04. - Quoting the pattern in
[[ $f == "*.gz" ]]. It turns it literal and always yields false. The pattern goes unquoted; the variable, quoted. - Applying
${s^^}on Bash 3.2 (macOS). It does not exist; there you have to usetr. - Tip: when a chain of trims becomes unreadable (
${${x#a}%b}is not even valid in Bash), use named intermediate variables. Three clear lines are worth more than one cryptic one.
Exercises
Exercise 1. Given path="/srv/veloz/data/shipments-2026-08-03.csv", obtain with expansions: the file name, the directory, the name without extension and the date 2026-08-03 it carries inside.
Exercise 2. Write normalize_city() that accepts any combination of case (SEVILLA, sevilla, SeViLla) and always returns Sevilla; if the city is not one of the four, return code 1.
Exercise 3. From the line 2026-08-03 10:15:22 [WARN] delivery queue above the threshold, extract date, time, level and message into four variables without using external tools, and print them with printf in aligned columns.
Solutions
Solution 1.
path="/srv/veloz/data/shipments-2026-08-03.csv"
file="${path##*/}" # shipments-2026-08-03.csv
directory="${path%/*}" # /srv/veloz/data
base="${file%.csv}" # shipments-2026-08-03
file_date="${base#shipments-}" # 2026-08-03
printf '%s | %s | %s | %s\n' "$file" "$directory" "$base" "$file_date"Solution 2.
normalize_city() {
local input="${1:?}" c
input="${input,,}" # everything to lowercase
for c in Valencia Sevilla Bilbao Madrid; do
[[ "$input" == "${c,,}" ]] && { printf '%s\n' "$c"; return 0; }
done
return 1
}
normalize_city "SeViLla" # SevillaNormalizing both sides to lowercase and comparing is more robust than playing with ${c^}, because it returns the exact canonical form from the list.
Solution 3.
line="2026-08-03 10:15:22 [WARN] delivery queue above the threshold"
log_date="${line%% *}" # up to the first space
rest="${line#* }" # removes the date
hour="${rest%% *}" # up to the next space
level="${line#*\[}"; level="${level%%\]*}"
message="${line#*\] }"
printf '%-12s %-10s %-7s %s\n' "$log_date" "$hour" "$level" "$message"
# 2026-08-03 10:15:22 WARN delivery queue above the thresholdEach line uses the minimum operator needed: %% for "up to the first one from the right", # for "the first one from the left". Zero processes launched.
Conclusion
You no longer need cut, tr or sed for routine string work. ${#s} measures, ${s:i:n} cuts by position, #/##/%/%% trim by pattern from each end, / and // substitute, ^/, change case, += concatenates, [[ $s == pattern* ]] compares against globs, IFS splits a line into an array, and printf with widths and %.2f produces tables that line up. All inside the same process, which in a loop over thousands of lines is the difference between seconds and minutes.
One structural piece is still missing. When in exercise 2 you walked four cities with a loop to see whether a string was valid, you were using the wrong tool: comparing a value against a closed list of alternatives is exactly what case does, the construct that in 04-05 will finally get the formal treatment we only used in passing in 03-05 to parse options. With it we will give daily-report.sh subcommands (summary, detail, cities) and build veloz-menu.sh, an interactive toolkit menu with select.
Bash Programming Course
Module 1: Introduction to Bash
- What Is Bash?
- Setting Up Your Environment
- Basic Command Line Navigation
- Understanding the Shell
- Finding Help: man, help and --help
Module 2: Basic Bash Commands
- File and Directory Operations
- Text Processing Commands
- File Permissions and Ownership
- Redirection and Piping
- Wildcards and Path Expansion
- History and Keyboard Shortcuts
Module 3: Scripting Fundamentals
- Creating and Running a Script
- Variables and Constants
- Basic Operators
- Conditional Statements
- Arguments and User Input
- Quoting, Expansion and Substitution
Module 4: Intermediate Scripting
- Loops in Bash
- Functions in Bash
- Arrays and Associative Arrays
- String Manipulation
- The case Statement and Interactive Menus
- Arithmetic and Numeric Calculations
Module 5: Advanced Scripting Techniques
- Advanced File Operations
- Process Management
- Error Handling and Debugging
- Regular Expressions
- Advanced I/O: Descriptors and Here-Documents
- Modular Scripts and Reusable Libraries
Module 6: Working with External Tools
Module 7: Automation and Scheduling
- Cron Jobs
- Automating Tasks
- Backup and Restore Scripts
- Monitoring and Logging
- Services and Timers with systemd
- Remote Automation with SSH
Module 8: Best Practices and Optimization
- Writing Readable Code
- Optimizing Bash Scripts
- Security Considerations
- Version Control with Git
- Static Analysis with ShellCheck and shfmt
- Automated Testing with Bats
- Portability: POSIX sh versus Bashisms
