We ended the previous lesson with an uncomfortable diagnosis: loops had removed the repetition between runs of the script, but inside the file we were still writing the same validation block three times and the same formatting printf four times. In Bash there are no classes, no modules, no objects: the function is the only unit of reuse that exists. Anything you want to write once and use many times —validate, format, log, compute— has to be a function. In this lesson you will learn to define them, to pass them data, to get results back (which is not what it looks like) and to document them; and at the end you will refactor daily-report.sh so that it stops being a wall of code and becomes a set of named pieces.

1. The two syntaxes and which to prefer
2. Where they are defined: definitions on top, main at the bottom
3. Parameters: functions are mini-scripts
4. local, and the bug waiting for you if you forget it
5. return returns an exit code, not data
6. How to really return data
7. Functions that fail, and recursion
8. Functions versus scripts and aliases
9. Documenting a function
10. Refactoring daily-report.sh

Bash accepts two ways of declaring a function, plus a third that mixes them:

We will use name() { ... } throughout the course. It is the one ShellCheck expects (08-05) and the one that keeps working if one day the script has to run under sh (08-07).

Naming rules: letters, digits and underscores; by convention, lowercase with _. Do not put parentheses when calling it: you invoke it like any other command, greet, not greet().

Bash reads the script top to bottom. A function only exists after the interpreter has gone past its definition:

From this comes the structural pattern you will use in all your scripts from now on:

Concrete advantages of this skeleton: the reader finds the program flow in main and not by digging through 300 lines; nothing runs until the last line, so the definition order between functions stops mattering (when main runs, they are all loaded); and main "$@" forwards the script's arguments to the function, with the quotes around $@ that 03-05 declared mandatory.

Here is the surprise for anyone coming from other languages: parameters are not declared in the signature. A function receives arguments exactly like a script: $1, $2, $#, "$@", shift.

Two surprising subtleties: $0 does not change inside a function (it is still the script name; for the function name there is ${FUNCNAME[0]}), and $1 are the arguments of the function, not those of the script, so if you need them inside you have to pass them along with my_function "$@".

By default, all Bash variables are global, including the ones you assign inside a function. This produces one of the hardest bugs in the language to track down:

The function has destroyed a variable of the main program without warning. The fix is to declare local on every internal working variable:

Details about local: it is only valid inside a function; it takes several at once (local a b c=0); its scope is dynamic, so a function called from another sees the caller's local variables (do not rely on it). And beware of local var=$(command): the $? left behind is local's, always 0, not the command's; if you need to check it, split it into local var; var=$(command) || return 1.

Golden rule: if a variable is not declared local, it is because you deliberately want it to outlive the function.

return N ends the function and sets $? to N. It is the same mechanism as the exit codes from 03-01, with the same limitations: an integer between 0 and 255, where 0 means success.

Since the function returns a code, it can be used directly in an if, in an &&, in a while… just like any command. That is why check functions are given question-like names (csv_usable, is_valid_city) and return 0/1.

What you cannot do is return 3.5, return "Valencia" or return 1000 (it is truncated to 1000 % 256 = 232). If you omit return, the function returns the code of the last command executed.

There are three techniques, in order of preference:

a) Write to stdout and capture with $( ). This is the idiomatic form:

Its cost: $( ) creates a subshell, and the function cannot write anything else to stdout —progress messages included— without contaminating the result. That is why informational messages always go to stderr, as you will see in log_info below.

b) Assign to an agreed global variable (result=...). Fast and with no subshell, but it couples the function to a specific name.

c) Nameref with local -n (Bash 4.3+). The caller decides the name of the output variable and the function writes into it indirectly:

local -n is the scoped version of declare -n. It is the technique you will use to "return" arrays (04-03), where $( ) would lose the separation between elements. Trap: the local name cannot match the one the caller passes or Bash raises a circular reference error; that is why it is prefixed with _.

Chaining functions with &&, || and $? works just like with external commands: validate_env || exit $? propagates the function's code to the operating system, and csv_usable && process_csv chains. Careful: exit inside a function kills the whole script, not just the function. That is correct in abort functions like die(), but in a reusable function prefer return and let the caller decide.

Bash supports recursion —factorial() { local n="$1"; (( n <= 1 )) && { echo 1; return; }; echo $(( n * $(factorial $(( n - 1 )) ) )); }— but each level creates a subshell because of the $( ), so it is slow; the limit is set by FUNCNEST. In operations scripts, a loop is almost always clearer and faster.

Practical translation: aliases are personal keyboard shortcuts (alias ll='ls -l'), functions are reuse inside a program, and scripts are the unit invoked from outside —cron, systemd, another script—. daily-report.sh is a script; validate_env is one of its functions.

A function with no header forces you to read its body to know how it is used. Adopt this format from today:

Four fixed sections —usage, arguments, output, return code— and one line of description. It is cheap to write and removes half the questions about the script.

We apply everything above. This is the script's skeleton after the refactor (we omit the option parsing, which you already have from 03-05):

What we have gained: a single place that validates (if tomorrow we have to check the configuration file, you touch validate_env and nothing else); messages separated from data, because log_info writes to stderr so that daily-report.sh > report.txt saves a clean table while the warnings stay on screen —the stream separation from 02-04 applied with judgment—; a four-line loop instead of four identical blocks, so adding Zaragoza means adding one word to CITIES; and a main that fits on one screen and reads like the statement of the problem.

The || true in count_errors deserves an explanation: grep -c returns code 1 when it counts 0 matches, which in a script with set -e (05-03) would abort the program. || true forces a 0 code without altering the output.

These functions are generic and you will want them in other toolkit scripts too. In 05-06 you will move them to ~/veloz-ops/lib/common.sh and load them with source, turning the toolkit into a real library.

• Calling the function with parentheses: city_summary("Valencia") is not a syntax error, it is something else and will fail in strange ways. You call it as city_summary Valencia.
• Forgetting local. The most expensive bug in this lesson. Declare everything internal local, always.
• Mixing messages and result on stdout. If the function is captured with $( ), any progress echo ends up inside the result. Messages → stderr.
• Believing that return returns data. It returns a 0-255 code. return $(wc -l < file) with 300 lines would return 44.
• Defining a function after using it. Definitions on top, main "$@" at the end.
• Tip: name functions with a verb (validate_, count_, generate_) and check functions as predicates (is_valid, exists_). The name should make reading the body unnecessary.

Exercise 1. Write is_valid_city() that takes a city and returns 0 if it is one of the four operational ones and 1 otherwise, printing nothing. Use it in an if.

Exercise 2. Write shipments_by() that takes a courier and writes its number of shipments to stdout, and show that it is captured with $( ). Add the full documentation header.

Exercise 3. Fix this function, which has two defects:

Solution 1.

In 04-05 you will see that a case does this in three lines, and in 04-03 that an associative array does it in one.

Solution 2.

Solution 3. The two defects are that sum is global and that return cannot return a total (it is truncated modulo 256):

By the way: ${amount%.*} trims the decimal part —an expansion you will master in 04-04— because Bash does not add decimals, something you will solve properly in 04-06.

Functions turn a script into a program. They are defined with name() { }, placed all on top with a main "$@" as the last line, receive arguments like a script through $1 and "$@", protect their internal state with local, signal success or failure with return (a code, never data) and deliver results by writing to stdout so the caller can capture them with $( ) —or through a nameref when that is not enough—. With validate_env, count_errors, city_summary and the two logging functions, daily-report.sh now has a skeleton.

What it lacks is memory. Our main already uses "${CITIES[@]}" without having explained what that syntax is, and the loop re-reads the whole CSV once per city because it has nowhere to accumulate a counter for each one. That is arrays (04-03): indexed lists and, better still, key→value maps that let you count issues by city and by courier in a single pass, replacing the old sort | uniq -c we have been dragging along since Module 2.