The security audit in 08-03 changed twelve things in the toolkit, and the optimization lesson changed just as many before that. Right now, if watchdog.sh starts failing on Tuesday at dawn, nobody can say what was touched, when or why, nor how to get back to the version that worked. This lesson puts ~/veloz-ops/ under version control. It is not a Git course: it is Git seen from the concrete need of an operations team that maintains five scripts, a library and a configuration file with credentials that must never be pushed.

1. Why backup.sh.bak.old.v2 is not version control
2. The five concepts you need
3. Putting ~/veloz-ops/ under control
4. Seeing what changed: git diff and git log
5. What is NOT versioned: .gitignore and the configuration
6. If you already pushed a secret
7. Commit messages that are useful in operations
8. Undoing with judgment
9. Branches for testing without touching production
10. Tags and deploying to the fleet
11. Remotes and private repositories
12. A real pre-commit hook
13. A minimal workflow for a small team

The pattern is universal: before touching a script you make a copy with a suffix. A year later the directory holds backup.sh.bak, backup.sh.old, backup.sh.v2, backup.sh.20260715 and backup.sh.GOOD. And none of those copies answers the questions that matter: what exactly changed between any two of them, who changed it, when, why and which one is running right now on srv-veloz-02. Nor do they let you know whether Tuesday's change and Thursday's are independent.

Git answers all five questions and adds two capabilities the copies do not give you: testing a change in parallel without touching what is in production, and undoing one specific change without undoing the later ones.

The staging area is what confuses people most at first and it has a very practical reason: it lets you touch four files and make two separate commits, one with the security fix and another with the performance improvement. In operations that is worth gold, because it lets you revert one without the other.

Before the first git add you have to decide what does not go in, so the right thing is to write the .gitignore first (section 5) and then add:

git add moves changes to the staging area; git commit records them in the history. In git status --short, A is added, M modified, D deleted and ?? untracked. One detail that saves surprises: Git stores the execute bit, so the chmod +x of bin/ travels with the repository.

The distinction between the two diff commands is the most asked question when starting out: the first shows what you have not yet added with git add; the second, what is about to go into the commit. Always review git diff --staged before writing the message: it is the natural moment to discover you left a set -x in place. To investigate a particular file, git log -p bin/watchdog.sh shows its history line by line and git blame bin/watchdog.sh says which commit introduced each line: the tool you will really use when something has been broken for months.

This is the critical part. etc/veloz-ops.conf holds the token we just protected in 08-03; pushing it makes it visible to the whole team, forever and in every clone.

Instead, a template is versioned that documents the configuration without revealing anything:

When installing the toolkit on a new server you copy the template, fill it in and adjust permissions. An important warning: .gitignore only affects untracked files. If you already did git add etc/veloz-ops.conf, adding it to .gitignore does not take it out: you need git rm --cached etc/veloz-ops.conf, which removes it from the repository and leaves it on disk.

It happens. The order of the actions is not negotiable:

1. Rotate the secret immediately. Generate a new one and invalidate the old one. It is the only step that actually mitigates; the rest is cleanup.
2. Take it out of the current repository: git rm --cached, add it to .gitignore, commit.
3. Rewrite the history with git filter-repo (today's recommended tool) or BFG Repo-Cleaner, which remove the file from every commit.
4. Tell the team: the rewrite changes every commit identifier and anyone with a clone will have to redo it.

And the underlying warning: rewriting the history is not enough if the repository has already been shared. Anyone could have cloned it, the platform could have indexed it, there are copies in caches and in backups. That is why step 1 is neither optional nor later.

A message does not describe the diff — the diff already does that — but what changed and why. In operations, the "why" is usually an incident:

Practical convention: an imperative first line under 72 characters, a blank line and a body with the context.

Six months from now, that commit explains why a retry loop exists that at first sight looks unnecessary. One commit per logical change: do not mix the performance refactor with the security fix even if you did both the same afternoon.

Four commands for four situations. Confusing them is the main source of lost work:

Golden rule: git revert for what others have already seen; git reset --hard only for your own, unpublished work. revert leaves a record that there was a change and that it was withdrawn, which in operations is valuable information. And if the last commit's message is wrong, git commit --amend fixes it as long as you have not pushed it.

You want to rewrite watchdog.sh's threshold logic, but that script runs every five minutes on three servers. A branch gives you a parallel space:

When the change is tested it is integrated with git merge dynamic-thresholds and the branch is deleted with git branch -d. If meanwhile somebody touched the same lines on master, Git reports a conflict and marks the file:

Resolving it means editing the file leaving the correct version — often a combination of both — deleting the three marker lines and running git add and git commit. There is no magic: a human decides. With Bash scripts it is also worth running bash -n (05-03) on the resolved file before committing, because a badly resolved conflict easily leaves one fi too many.

A tag marks a point in history with a stable name, and in operations it tells you which version is running on each server:

Deploying to srv-veloz-01/02/03 combines this with what we saw in 07-06:

Option A lets you query the deployed version with git describe on each machine; option B avoids installing Git and credentials in production. Note the exclusion of etc/veloz-ops.conf: the configuration is local to each server and must not be overwritten from the repository.

A local repository is not a backup: if the disk is lost, the history is lost.

git pull is internally fetch plus merge; if you prefer to see what is coming before integrating it, git fetch followed by git log HEAD..origin/master is an excellent habit in operations. And a warning that is not superfluous: operations code goes in private repositories. Even if it contains no secrets — and it must not — it reveals server names, internal paths, thresholds and system structure; it is free reconnaissance for an attacker. Authentication by SSH key (07-06), not by password.

A hook is a script Git runs automatically at a certain moment. pre-commit runs before creating the commit and, if it exits with a code other than 0, aborts it:

Four barriers: syntax with bash -n, static analysis with ShellCheck, tests with Bats and a simple secret search. --diff-filter=ACM limits the check to added, copied or modified files — there is no point analyzing one that is being deleted — and it only looks at the staged ones, not the whole tree. Two limitations you have to know: hooks live in .git/hooks/ and are not versioned, so they have to be installed in every clone (which is why many teams keep the hook in tools/ and link it with an install script), and git commit --no-verify skips them. The hook is a safety net for slip-ups, not an access control.

For two or three operations people this is enough: git pull before starting; a short branch for the change (git switch -c fix-api-timeout); small commits with messages that cite the incident; git push -u origin <branch> and a review request to another person; merge to master, tag if it is deployable and deploy to the fleet; delete the branch.

The second-to-last step is the most important on the list, and it is not a tool: it is a person. Review by a colleague is the best quality control there is, better than ShellCheck, better than tests and better than your own discipline. It finds what tools cannot see: that the threshold is badly chosen, that the script assumes a directory that only exists on your machine, that the rm -rf is not as constrained as it looks. Ten minutes of review save a night on call.

• Starting with git add . and no .gitignore. It is the most common way to push a secret. The .gitignore comes before the first commit.
• Believing .gitignore removes already-tracked files. It does not: git rm --cached is what takes them out.
• Deleting the secret and not rotating it. It is still in the history, in the clones and in the caches.
• git reset --hard on already published commits. It breaks everyone else's repository; for what is published, git revert.
• Tip: git stash for the interruption. It saves half-done work, you handle the incident and get it back with git stash pop.
• Tip: git log -S 'veloz_api_get' searches for the commit in which a string appeared or disappeared; it is the fast way to date a behavior change.

Exercise 1. You discover that etc/veloz-ops.conf, with the API token, has been versioned for three commits in a repository shared with two colleagues. List in order the actions and the concrete commands.

Exercise 2. Write the toolkit's .gitignore, justifying each exclusion, and explain what is versioned instead.

Exercise 3. You have merged into master a change in watchdog.sh (commit 3b8d201) that causes false alerts, and there are already two later commits from a colleague. How do you undo it without losing their work?

Solution 1. First, rotate the token in the API and invalidate the old one: it is in three clones and on the Git server, and it is the only thing that stops the real exposure. Then:

Then rewrite the history with git filter-repo --path etc/veloz-ops.conf --invert-paths (or BFG), warning the two colleagues beforehand that they will have to clone again because every commit identifier changes. And document the incident, remembering that the rewrite does not replace rotation: if the repository was shared, you have to assume the token is public.

Solution 2. The one in section 5. etc/veloz-ops.conf, *.key, *.pem and .netrc contain credentials; logs/ and *.log are regenerable output that would also grow the repository without limit; *.tmp are mktemp temporaries; data/ and *.csv are business data with personal information (08-03), which belong in the backup and not in version control. etc/veloz-ops.conf.example is versioned because it documents which variables have to be defined without revealing their values.

Solution 3. With git revert, which creates a new commit undoing exactly the changes in 3b8d201 and leaves the two later ones untouched:

git reset --hard would be a serious mistake: it would also throw away your colleague's commits and, since they are already published, would force a push --force that would break their repository. If the revert produces a conflict because the later commits touched the same lines, it is resolved by hand as in section 9 and finished with git revert --continue.

For an operations toolkit Git solves what .bak copies do not: what changed, who, when and why, and how to go back without dragging everything else along. Five concepts — repository, staging area, commit, branch and remote — are enough to work: git init, git add, git commit -m, git status --short, git diff and git diff --staged to review before committing, git log --oneline --graph to read the history and git blame to find out who introduced that line. The first thing you write is not a commit but the .gitignore, which leaves out logs/, temporaries, data and above all etc/veloz-ops.conf with its credentials, versioning etc/veloz-ops.conf.example instead; and if a secret is already inside, the order is rotate first, then git rm --cached and then git filter-repo or BFG, knowing that rewriting the history repairs nothing if the repository has already been shared. Messages describe the change and its motive — the incident that caused it — with one commit per logical change. To undo you have to choose well: git restore discards edits, git restore --staged unstages, git revert undoes what is published leaving a record, and git reset --hard is only used on your own unpublished commits. Branches (git switch -c) let you rewrite watchdog.sh without touching what runs every five minutes, with merging and manual conflict resolution; tags (git tag -a v1.2) fix which version is deployed and combine with the fleet deployment from 07-06, either by git checkout on each server or by rsync excluding .git and the local configuration. The remote — private and with an SSH key — is at once a backup and a meeting point. And the pre-commit hook automates the barriers: bash -n, ShellCheck, Bats and a secret search, with the double warning that hooks are not versioned and that --no-verify skips them. Above all of that remains the most effective quality control, which is not a tool: having another person read your change before it reaches production.

The pre-commit hook we just wrote invokes two commands we do not know yet. Lesson 08-05 deals with the first one: ShellCheck, the static analyzer that finds the errors this course has taught you to avoid — and quite a few more — together with shfmt for the uniform formatting that 08-01 left promised. We will go through the most frequent warnings, connecting each one to the lesson where the problem was explained, we will learn to silence a warning with judgment rather than for convenience, and we will run ShellCheck over the five scripts and lib/common.sh to classify and fix everything that shows up.