Two questions from Vallmar's council department had been left unanswered since the first day of this module, and neither of them is fixed with an index.
The first was asked by the culture councillor at the follow-up meeting: "who can look up the phone numbers and email addresses of the twelve thousand members?". Nobody knew how to answer, which is the worst possible answer. The second was asked by the finance officer: "what would happen if the server's disk died tonight?". Somebody said "we have backups", and when asked when the last restore had been tested, silence fell.
This lesson answers both, and closes the module and the course's theoretical block.
There is one idea worth putting up front, because it organizes everything that follows: the database is the last frontier. If somebody gets through the firewall, tricks the application or finds a credential in a repository, the only thing left between that person and the data of twelve thousand Vallmar residents are the database's permissions. And if the building burns down, the only thing left between the library and starting from scratch are the backups —tested, not merely existing—.
We will look at authentication and authorization with roles and GRANT; the principle of least privilege applied to three concrete BiblioRed roles; views and row-level security so that each branch sees only its own; SQL injection and why parameterized queries are the only defense that works; encryption in transit and at rest; auditing; the handling of personal data; and the complete backup block, where the WAL from 06-01 reappears turned into the tool that lets you recover the database at the instant before the disaster.
Contents
- Why database security is different from application security
- Authentication: roles, passwords and
pg_hba.conf - Authorization:
GRANT,REVOKEand the privilege catalog - Least privilege: BiblioRed's three roles
- The danger of connecting as a superuser
- Views for hiding sensitive columns
- Row-level security: each branch sees its own
- SQL injection: how it happens and how it is prevented
- Encryption in transit, at rest and the special case of passwords
- Auditing: access logging and a change table
- Personal data: minimization, pseudonymization and retention
- Backups: logical and physical
- Full, differential and incremental
- WAL archiving and point-in-time recovery
- The 3-2-1 rule and verifying your backups
- RPO and RTO applied to BiblioRed
- BiblioRed's backup plan, annotated
- The first minutes after an accidental deletion
- The minimum equivalent in SQLite
- Why database security is different from application security
It is common to think that security is solved in the application layer: if only authorized staff see the member listing screen, the data is protected. It is a dangerous idea, and listing the paths that reach the database without going through the application is enough to see it:
| Path | Does it go through the application's logic? |
|---|---|
A developer with psql from their laptop |
No |
| A nightly maintenance script | No |
| A reporting tool connected over ODBC | No |
| A backup copied onto a laptop | No |
| An SQL injection through the catalog search | Yes, but bypassing the logic |
| An intern with the credentials from the configuration file | No |
Six paths, and five of them never see a single line of the application's code. Hence the principle:
Access controls must be where the data is. The application can add convenience and context, but the guarantee has to live in the database.
It is exactly the same argument we used in 04-04 for constraints and in 06-02 for the reading club's capacity: a rule that cannot be broken must live where it cannot be bypassed. Here the rule is "front desk staff cannot export members' email addresses".
A system's security layers, from the outside in:
graph LR
A[Network and firewall] --> B[TLS in transit]
B --> C[Authentication:<br/>who are you?]
C --> D[Authorization:<br/>what can you do?]
D --> E[Row-level security:<br/>on which rows?]
E --> F[Encryption at rest]
F --> G[(Data)]
D --> H[Auditing:<br/>what have you done?]
The two central questions of this lesson are the third and fourth boxes: authentication (who are you?) and authorization (what can you do?). They are constantly confused and they are different things.
- Authentication: roles, passwords and
pg_hba.conf
pg_hba.confRoles: a single entity for users and groups
PostgreSQL does not distinguish between "user" and "group": it has roles. A role with the LOGIN attribute can connect; a role without it serves as a permission group. CREATE USER is simply shorthand for CREATE ROLE ... LOGIN.
-- Group role: it bundles permissions, it does not connect
CREATE ROLE bibliored_frontdesk NOLOGIN;
-- Login role: one specific person
CREATE ROLE u_alsina LOGIN PASSWORD 'a-long-and-unique-password'
VALID UNTIL '2027-01-01';
-- Group membership
GRANT bibliored_frontdesk TO u_alsina;Attributes worth knowing:
| Attribute | What it grants |
|---|---|
LOGIN |
Can connect |
SUPERUSER |
Bypasses every permission check |
CREATEDB |
Can create databases |
CREATEROLE |
Can create and modify other roles |
INHERIT (default) |
Automatically inherits the permissions of the roles it belongs to |
NOINHERIT |
Must activate them explicitly with SET ROLE |
CONNECTION LIMIT n |
Maximum simultaneous connections |
VALID UNTIL |
Password expiry date |
Inspecting what is there:
List of roles
Role name | Attributes | Member of
----------------------+------------------------------------+----------------------
app_bibliored | | {bibliored_app}
bibliored_app | Cannot login | {}
bibliored_frontdesk | Cannot login | {}
bibliored_management | Cannot login | {}
postgres | Superuser, Create role, Create DB | {}
u_alsina | Password valid until 2027-01-01 | {bibliored_frontdesk}pg_hba.conf: who can connect, from where and how
Before checking the password, PostgreSQL consults the pg_hba.conf file (host-based authentication). It is a list of rules evaluated from top to bottom, and the first one that matches is applied. If none matches, the connection is rejected.
# TYPE DATABASE USER ADDRESS METHOD local all postgres peer host biblioredb bibliored_frontdesk 10.20.0.0/16 scram-sha-256 host biblioredb app_bibliored 10.20.5.11/32 scram-sha-256 hostssl biblioredb bibliored_management 0.0.0.0/0 scram-sha-256 host all all 0.0.0.0/0 reject
The methods, with an assessment of each:
| Method | What it does | Use it? |
|---|---|---|
scram-sha-256 |
Password with challenge-response; the password never travels over the network | Yes. It is the default since PostgreSQL 10 and the recommended one |
md5 |
Old method, with known weaknesses | Only for compatibility with old clients; migrate |
peer |
Checks the operating system user (local socket connections only) | Yes, for administration tasks on the server itself |
cert |
TLS client certificate | Yes, in environments with certificate management |
ldap, gss |
Delegation to the corporate directory | Yes, in organizations with a directory |
trust |
Accepts anyone without checking anything | NO |
reject |
Always denies | Yes, as a final rule |
About
trust. It literally means "anyone arriving by this route gets in as whichever user they claim to be, with no password". Its only legitimate use is a local development instance on your own machine, with no real data and no port open to the outside. A linehost all all 0.0.0.0/0 truston a server is equivalent to publishing the database on the internet with no door. It shows up more often than anybody would like to admit, almost always because somebody put it there "just for a moment, to test something" and nobody removed it.
Note the hostssl line for management: it forces the connection to be encrypted. And the last line, reject, turns the list into a deny-by-default policy, which is how every access control list should be written.
After editing the file:
sudo systemctl reload postgresql
# or, without system permissions, from psql as a superuser:
# SELECT pg_reload_conf();Checking which rules are active:
line_number | type | database | user_name | address | auth_method
-------------+---------+----------------+-----------------------+-------------+---------------
80 | local | {all} | {postgres} | | peer
81 | host | {biblioredb} | {bibliored_frontdesk} | 10.20.0.0 | scram-sha-256
82 | host | {biblioredb} | {app_bibliored} | 10.20.5.11 | scram-sha-256
83 | hostssl | {biblioredb} | {bibliored_management}| 0.0.0.0 | scram-sha-256
84 | host | {all} | {all} | 0.0.0.0 | reject
- Authorization:
GRANT, REVOKE and the privilege catalog
GRANT, REVOKE and the privilege catalogOnce the role is authenticated, the next question is what it can do. PostgreSQL's model is hierarchical: to reach a table you have to pass through the database and the schema.
graph TD
A["Database<br/>CONNECT privilege"] --> B["Schema<br/>USAGE privilege"]
B --> C["Table<br/>SELECT / INSERT / UPDATE / DELETE"]
B --> D["Sequence<br/>USAGE / SELECT"]
B --> E["Function<br/>EXECUTE"]
C --> F["Column<br/>SELECT (col) / UPDATE (col)"]
It is the number one mistake when configuring permissions: granting SELECT on the tables and forgetting USAGE on the schema. Without USAGE, the role cannot even see that the table exists.
Privilege catalog
| Privilege | Applies to | Allows |
|---|---|---|
CONNECT |
Database | Connecting to it |
CREATE |
Database, schema | Creating schemas / objects |
USAGE |
Schema, sequence, type | Accessing the schema's objects; using the sequence |
SELECT |
Table, view, column | Reading |
INSERT |
Table, column | Inserting |
UPDATE |
Table, column | Modifying |
DELETE |
Table | Deleting rows |
TRUNCATE |
Table | Emptying the table |
REFERENCES |
Table, column | Creating foreign keys that point at it |
TRIGGER |
Table | Creating triggers |
EXECUTE |
Function, procedure | Running it |
Syntax
GRANT SELECT, INSERT ON loans TO bibliored_frontdesk;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO bibliored_management;
GRANT USAGE ON SCHEMA public TO bibliored_frontdesk;
REVOKE DELETE ON loans FROM bibliored_frontdesk;Two essential advanced forms:
Column-level privileges. You can authorize reading some columns and not others:
GRANT SELECT (member_id, first_name, last_name, join_date, branch_id, active)
ON members TO bibliored_frontdesk;That role will be able to read a member's name but not their email. If it tries:
Default privileges for future objects. A GRANT ON ALL TABLES affects the tables that exist today. The table somebody creates tomorrow will not be included, and that is a classic silent failure:
The PUBLIC role problem
PostgreSQL has a pseudo-role called PUBLIC that everybody belongs to. Historically, PUBLIC had CREATE on the public schema, which let any user create tables there. Since PostgreSQL 15 that is no longer the case, but on older installations —or migrated ones— it is worth checking and fixing:
Inspecting the privileges granted:
Schema | Name | Type | Access privileges
--------+---------+-------+---------------------------------------------------------------
public | members | table | bibliored_app=arwd/postgres +
| | | bibliored_management=r/postgres +
| | | bibliored_frontdesk=r(member_id,first_name,last_name)/postgresThe letters: r = SELECT, a = INSERT, w = UPDATE, d = DELETE, x = REFERENCES, U = USAGE.
- Least privilege: BiblioRed's three roles
Principle of least privilege. Each role receives exactly the permissions it needs for its function, not one more, and only while it needs them.
BiblioRed needs three profiles. We define them first in plain language, because a permission you cannot explain in one sentence is usually badly thought out:
| Role | Who it is | What it needs | What it must NOT be able to do |
|---|---|---|---|
bibliored_frontdesk |
Staff at the four branches | Lend, take returns, enroll members, collect fines, register people for events | Read members' emails and phone numbers; delete anything; see other branches |
bibliored_management |
Management and the council department, for reports | Read everything aggregated and statistical | Write absolutely anything; read contact details |
bibliored_app |
The public web application | Query the catalog, manage the authenticated member's reservations and registrations | Touch fines, payments, complete members, or any other member's data |
The complete script
-- =========================================================
-- 1) Closed by default: nobody has anything they are not given
-- =========================================================
REVOKE ALL ON DATABASE biblioredb FROM PUBLIC;
REVOKE ALL ON SCHEMA public FROM PUBLIC;
REVOKE ALL ON ALL TABLES IN SCHEMA public FROM PUBLIC;
-- =========================================================
-- 2) Group roles (no LOGIN: they are permission containers)
-- =========================================================
CREATE ROLE bibliored_frontdesk NOLOGIN;
CREATE ROLE bibliored_management NOLOGIN;
CREATE ROLE bibliored_app NOLOGIN;
GRANT CONNECT ON DATABASE biblioredb
TO bibliored_frontdesk, bibliored_management, bibliored_app;
GRANT USAGE ON SCHEMA public
TO bibliored_frontdesk, bibliored_management, bibliored_app;
-- =========================================================
-- 3) FRONT DESK: runs the day to day, without contact details
-- =========================================================
GRANT SELECT, INSERT, UPDATE ON loans, reservations, registrations
TO bibliored_frontdesk;
GRANT SELECT, UPDATE (status, branch_id) ON copies
TO bibliored_frontdesk;
GRANT SELECT ON materials, materials_book, materials_dvd,
materials_magazine, materials_audiobook, dvd_subtitles,
authors, branches, rooms, event_types, events, books
TO bibliored_frontdesk;
-- Members: enrollment and modification, but WITHOUT reading email
GRANT SELECT (member_id, first_name, last_name, join_date, branch_id, active)
ON members TO bibliored_frontdesk;
GRANT INSERT ON members TO bibliored_frontdesk;
GRANT UPDATE (first_name, last_name, email, branch_id, active)
ON members TO bibliored_frontdesk;
-- Fines and payments: issue and collect, never delete
GRANT SELECT, INSERT, UPDATE ON fines TO bibliored_frontdesk;
GRANT SELECT, INSERT ON payments TO bibliored_frontdesk;
-- Sequences: without USAGE you cannot insert into tables with identity columns
GRANT USAGE ON ALL SEQUENCES IN SCHEMA public TO bibliored_frontdesk;
-- =========================================================
-- 4) MANAGEMENT: read only, and without contact details
-- =========================================================
GRANT SELECT ON ALL TABLES IN SCHEMA public TO bibliored_management;
-- Access to the sensitive parts is withdrawn and replaced by the view from section 6
REVOKE SELECT ON members, member_phones FROM bibliored_management;
GRANT SELECT ON v_members_public TO bibliored_management;
-- Future objects: so they do not slip through by oversight
ALTER DEFAULT PRIVILEGES IN SCHEMA public
GRANT SELECT ON TABLES TO bibliored_management;
-- =========================================================
-- 5) WEB APPLICATION: minimum surface
-- =========================================================
GRANT SELECT ON materials, materials_book, materials_dvd,
materials_magazine, materials_audiobook, dvd_subtitles,
authors, branches, copies, rooms, event_types, books
TO bibliored_app;
GRANT SELECT ON v_events_published TO bibliored_app;
GRANT SELECT, INSERT, UPDATE ON reservations, registrations TO bibliored_app;
GRANT SELECT ON loans TO bibliored_app;
GRANT SELECT (member_id, first_name, last_name, email, branch_id, active)
ON members TO bibliored_app;
GRANT USAGE ON ALL SEQUENCES IN SCHEMA public TO bibliored_app;
-- What the application may NOT touch, stated explicitly
REVOKE ALL ON fines, payments, event_reports FROM bibliored_app;
-- =========================================================
-- 6) Real login roles
-- =========================================================
CREATE ROLE u_alsina LOGIN PASSWORD 'xxxxxxxxxxxx' VALID UNTIL '2027-01-01';
CREATE ROLE u_pereda LOGIN PASSWORD 'xxxxxxxxxxxx' VALID UNTIL '2027-01-01';
CREATE ROLE u_management LOGIN PASSWORD 'xxxxxxxxxxxx' VALID UNTIL '2027-01-01';
CREATE ROLE app_bibliored LOGIN PASSWORD 'xxxxxxxxxxxx' CONNECTION LIMIT 40;
GRANT bibliored_frontdesk TO u_alsina, u_pereda;
GRANT bibliored_management TO u_management;
GRANT bibliored_app TO app_bibliored;Testing it
A permission that has not been tested is an assumption. The check is done by impersonating the role:
SET ROLE bibliored_frontdesk;
SELECT first_name, last_name FROM members WHERE member_id = 14; -- must work
SELECT email FROM members WHERE member_id = 14; -- must fail
DELETE FROM loans WHERE loan_id = 88301; -- must fail
RESET ROLE;first_name | last_name ------------+------------ Marta | Alsina ERROR: permission denied for table members ERROR: permission denied for table loans
Two expected errors and one correct query: the permissions do what the document says.
There is also a function to check it without running anything:
SELECT has_table_privilege('bibliored_frontdesk', 'fines', 'DELETE') AS can_delete_fines,
has_column_privilege('bibliored_frontdesk', 'members', 'email', 'SELECT') AS can_read_email;
- The danger of connecting as a superuser
It is the most widespread bad practice and the easiest to fix.
When an application connects with a superuser role —postgres in most installations—, all the work of the previous sections is nullified. A superuser bypasses every permission check, including the row-level security of section 7.
What an attacker who gets hold of the application's connection can do:
With bibliored_app |
With postgres (superuser) |
|---|---|
| Read the catalog and the reservations | Read everything, including fines and payments |
| Cannot drop tables | DROP TABLE loans; |
Cannot touch pg_hba.conf |
Can create users and give itself permanent access |
| Cannot read the server's files | COPY ... FROM PROGRAM runs operating system commands |
That last row turns a medium-severity SQL injection into a total compromise of the server.
The checklist:
- The application's connection string never uses
postgresor any role withSUPERUSER. - The tables' owner is an administration role different from the one the application uses.
- Passwords live in a secrets manager or in the service's environment variables, never in the code repository.
- Every person has their own login role. A shared account makes the auditing of section 10 impossible.
Verifying there are no extra superusers:
One. If three or four show up, there is work to do.
- Views for hiding sensitive columns
The column privileges of section 3 work well, but they have a practical drawback: SELECT * fails, and many reporting tools use it. The elegant alternative is a view.
A view runs with the permissions of whoever created it, not of whoever queries it. That makes it possible to give access to a subset of the data without giving access to the underlying table.
CREATE VIEW v_members_public AS
SELECT member_id,
first_name,
last_name,
join_date,
branch_id,
active
FROM members;
-- The role has NO permission on members, but it does on the view
REVOKE ALL ON members FROM bibliored_management;
GRANT SELECT ON v_members_public TO bibliored_management;Check:
member_id | first_name | last_name | join_date | branch_id | active
-----------+------------+-------------+------------+-----------+--------
14 | Marta | Alsina | 2019-03-11 | 1 | t
15 | Iván | Pereda | 2021-09-02 | 2 | t
16 | Nuria | Bastos | 2023-01-24 | 3 | tSELECT * works, and the emails and phone numbers are out of reach.
Other useful views of the same kind in BiblioRed:
-- Public events catalog: no internal notes or reports
CREATE VIEW v_events_published AS
SELECT e.event_id, e.title, t.name AS type, r.name AS room,
e.start_time, e.end_time, e.offered_seats
FROM events e
JOIN event_types t ON t.event_type_id = e.event_type_id
JOIN rooms r ON r.room_id = e.room_id
WHERE e.published AND e.status = 'scheduled';
-- Masked contact details, for technical support
CREATE VIEW v_members_contact_masked AS
SELECT member_id,
first_name,
last_name,
regexp_replace(email, '(.).*(@.*)', '\1***\2') AS masked_email,
branch_id
FROM members; member_id | first_name | last_name | masked_email | branch_id
-----------+------------+-----------+--------------------+-----------
14 | Marta | Alsina | m***@example.org | 1A warning about masking: it exists so that support staff can verify an email address a member reads out over the phone, not to anonymize. Masking is not anonymization, and in section 11 we will see the difference.
Technical note: by default views are SECURITY INVOKER as far as the row-level security of the next section is concerned, but they run with the owner's permissions with respect to the tables. If you need the view to apply the querying user's row policies, declare it with WITH (security_invoker = true), available since PostgreSQL 15.
- Row-level security: each branch sees its own
Views hide columns. To hide rows —so that North branch staff do not see South branch loans— PostgreSQL offers row-level security (RLS).
With RLS, each table can carry policies that act as an implicit and mandatory
WHERE, applied by the management system to every query from the affected roles.
Step 1: enable RLS
Careful: with RLS enabled and no policy at all, nobody sees anything. The default behavior is to deny, which is the correct one.
Step 2: set the session context
The policy needs to know which branch the user is at. The application communicates it with a session parameter when opening the connection:
Step 3: create the policies
-- Front desk staff only see their own branch's loans
CREATE POLICY pol_loans_branch ON loans
FOR ALL
TO bibliored_frontdesk
USING (
EXISTS (
SELECT 1 FROM copies c
WHERE c.copy_id = loans.copy_id
AND c.branch_id = current_setting('app.current_branch')::int
)
);
-- Management sees every loan, with no row restriction
CREATE POLICY pol_loans_management ON loans
FOR SELECT
TO bibliored_management
USING (true);A policy's clauses:
| Clause | What it controls |
|---|---|
USING (expr) |
Which rows are visible (SELECT, UPDATE, DELETE) |
WITH CHECK (expr) |
Which rows can be created or left behind (INSERT, UPDATE) |
FOR |
Which operations it applies to (ALL, SELECT, INSERT, UPDATE, DELETE) |
TO |
Which roles |
Without WITH CHECK, a role could insert rows it then cannot see, which is usually a bug. The complete policy for members:
ALTER TABLE members ENABLE ROW LEVEL SECURITY;
CREATE POLICY pol_members_branch ON members
FOR ALL
TO bibliored_frontdesk
USING (branch_id = current_setting('app.current_branch')::int)
WITH CHECK (branch_id = current_setting('app.current_branch')::int);Check
The same query, the same role, different results: the management system is applying the filter of its own accord. And it is impossible to bypass it from SQL:
The three warnings about RLS
- Superusers and table owners bypass RLS. If the application connects as the tables' owner, the policies are not applied. You have to force it with
ALTER TABLE members FORCE ROW LEVEL SECURITY;. - It has a performance cost. The policy becomes an extra condition on every query, and policies with subqueries like the one on
loanscan change the execution plan. Check it withEXPLAIN ANALYZE(06-03). - The session parameter must be set reliably by the application and must not be alterable by the end user. If the user can influence
app.current_branch, the policy protects nothing.
- SQL injection: how it happens and how it is prevented
It is the oldest and best-known vulnerability of database-backed applications, and it still shows up every year in incident lists. Its cause is always the same: mixing code and data in the same string of text.
How it happens
BiblioRed's catalog search builds the query by concatenating what the user types:
# VULNERABLE CODE — never write this
term = request.args.get("q")
sql = "SELECT material_id, title FROM materials WHERE title LIKE '%" + term + "%'"
cursor.execute(sql)With a normal search, q = map, the resulting query is correct:
Now a visitor types into the search box:
The query that reaches the server is:
SELECT material_id, title FROM materials WHERE title LIKE '%' UNION SELECT member_id, email FROM members --%' material_id | title
-------------+------------------------------
907 | The Map of Time
14 | marta.alsina@example.org
15 | ivan.pereda@example.org
16 | nuria.bastos@example.org
...The twelve thousand email addresses of Vallmar's members, in the catalog's public search box. With no password, no tools and leaving no more trace than a line in the query log.
And that is just reading. What an injection allows, in general:
| Objective | Example |
|---|---|
| Read any accessible table | UNION SELECT over members, fines, payments |
| Modify or delete data | '; UPDATE fines SET status='paid'; -- |
| Bypass authentication | ' OR '1'='1 in a login form |
| Extract the schema | Queries against information_schema |
| Deny service | pg_sleep(60) on every request |
| Run system commands | COPY ... FROM PROGRAM, only if the connection is a superuser |
That last row explains why sections 4 and 5 are part of the defense against injection: with bibliored_app properly restricted, the same injection cannot read fines, cannot delete anything, and cannot touch the operating system. The damage of an injection is exactly equal to the connection's permissions.
The main defense: parameterized queries
A parameterized query sends the SQL text and the values along separate paths. The management system receives the query's structure first and the data afterwards, so it is impossible for a piece of data to be interpreted as code. There is nothing to escape because there is nothing to mix.
# CORRECT CODE
term = request.args.get("q")
sql = "SELECT material_id, title FROM materials WHERE title ILIKE %s"
cursor.execute(sql, ('%' + term + '%',))With the same attack, the result is:
The database has searched, literally, for materials whose title contains the string ' UNION SELECT member_id, email FROM members --. There are none. The attack has turned into a search with no results, which is exactly what it should be.
In direct SQL, inside psql or a function, the equivalent is PREPARE:
PREPARE search_material (text) AS
SELECT material_id, title FROM materials WHERE title ILIKE '%' || $1 || '%';
EXECUTE search_material ('map');Why escaping by hand is not enough
It is tempting to think it is enough to "strip the quotes". It is not, for five reasons:
| Problem | Explanation |
|---|---|
| You have to remember every time | One forgotten place out of two hundred is enough for the defense not to exist |
| It depends on the encoding | Certain multibyte encodings allow sequences to be built that survive escaping |
| Numbers do not carry quotes | WHERE member_id = + input is not protected by escaping quotes |
| It does not cover identifiers | A dynamic column or table name needs different treatment (quote_ident) |
| It is your own code on the critical path | Any bug in the escaping function opens the whole hole |
A parameterized query has none of these problems because it escapes nothing: it separates the paths.
When the SQL must be dynamic
Sometimes the variable part is the name of a sort column, and that cannot be parameterized. The solution is not to escape, it is to validate against an allowlist:
SORT_COLUMNS = {"title": "title", "year": "publication_year", "author": "author_id"}
col = SORT_COLUMNS.get(request.args.get("sort"), "title") # if absent, default value
sql = f"SELECT material_id, title FROM materials ORDER BY {col} LIMIT %s"
cursor.execute(sql, (20,))The user's input never reaches the SQL: it is only used as a key for choosing among fixed values written by you.
Complementary defenses
None of them replaces parameterization; all of them reduce the damage:
- Least privilege (section 4): make sure the connection cannot read what it has no business reading.
- Input validation: check that an identifier is an integer, that a date is a date.
- Never show the database's error message to the end user. An
ERROR: column "members.email" does not existis a free map of the schema. - Log SQL errors: a burst of syntax errors from the same address is an attack in progress.
- Encryption in transit, at rest and the special case of passwords
In transit: TLS
Without encryption, queries and their results travel across the network in the clear. Anybody with access to the link can read the members' email addresses as they are transmitted.
# postgresql.conf ssl = on ssl_cert_file = '/etc/ssl/certs/biblioredb.crt' ssl_key_file = '/etc/ssl/private/biblioredb.key'
And in pg_hba.conf, hostssl instead of host to force the connection to be encrypted. On the client side:
The sslmode modes, which matter more than they seem:
| Mode | Encrypts | Verifies the certificate | Verifies the server name |
|---|---|---|---|
disable |
No | — | — |
require |
Yes | No | No |
verify-ca |
Yes | Yes | No |
verify-full |
Yes | Yes | Yes |
require encrypts but does not check who it is talking to, so it does not protect against a man in the middle. The correct configuration in production is verify-full.
Checking a connection's status:
ssl | version | cipher -----+---------+------------------------- t | TLSv1.3 | TLS_AES_256_GCM_SHA384
At rest
Two levels, with different purposes:
| Level | How | Protects against | Does not protect against |
|---|---|---|---|
| Disk / volume (LUKS, provider encryption) | Transparent to PostgreSQL | Physical theft of the disk, hardware decommissioning, backups on lost media | Anything that happens while the server is running |
Column (pgcrypto) |
Explicit encryption of specific values | Direct reading of the files or of a backup | It requires key management, and it prevents indexing and searching by that column |
Disk encryption is cheap, transparent and should always be on. Column encryption is a surgical tool: in BiblioRed there is nothing that justifies it —no card numbers or health data are stored—, and applying it to email would make searching by email impossible and would require guarding a key whose loss would be equivalent to losing the data.
Rule: always encrypt the disk; encrypt columns only when you can name the exact piece of data, the exact risk and who holds the key.
Passwords: the exception that is not encrypted
Members' passwords for the web portal deserve a section of their own because the mistake here is serious and frequent.
| Practice | Verdict |
|---|---|
| Storing the password in the clear | Unacceptable |
| Storing it reversibly encrypted | Unacceptable: whoever has the key has all of them |
Storing an MD5 or SHA-1 of the password |
Unacceptable: they are broken with precomputed tables |
Storing an unsalted SHA-256 |
Insufficient: fast to brute-force |
| Storing the result of a key derivation function with a salt (bcrypt, scrypt, Argon2) | Correct |
A password is not encrypted: it is transformed with a one-way function, deliberately slow and salted. You do not need to be able to recover it; you only need to be able to verify it. That is why serious systems offer "reset your password" and never "remind me of my password".
CREATE EXTENSION IF NOT EXISTS pgcrypto;
-- When enrolling or changing the password
UPDATE members
SET password_hash = crypt('the-members-password', gen_salt('bf', 12))
WHERE member_id = 14;
-- When verifying it
SELECT member_id
FROM members
WHERE member_id = 14
AND password_hash = crypt('the-typed-password', password_hash);If the password is wrong, the query returns zero rows. The 12 parameter in gen_salt('bf', 12) is the cost: each unit doubles the computation time, which slows down brute-force attacks without bothering the legitimate user.
An important note: the above applies to the passwords of members in the application. The passwords of PostgreSQL roles are managed by the server itself with SCRAM-SHA-256, and nothing special needs doing beyond using that method in pg_hba.conf.
- Auditing: access logging and a change table
Authentication, authorization and encryption answer "who can?". Auditing answers "who has done what, and when?", which is the question asked after an incident, and also the one any serious review of personal-data processing demands.
Server logging
# postgresql.conf log_connections = on log_disconnections = on log_statement = 'ddl' # 'none' | 'ddl' | 'mod' | 'all' log_min_duration_statement = 1000 log_line_prefix = '%m [%p] %u@%d from %h '
| Parameter | What it logs | Cost |
|---|---|---|
log_connections / log_disconnections |
Who connects and from where | Very low |
log_statement = 'ddl' |
Schema changes | Low. The recommended minimum |
log_statement = 'mod' |
Additionally, every write | Medium |
log_statement = 'all' |
Absolutely everything | High, and it logs personal data in plain text |
That last point deserves attention: turning on log_statement = 'all' on a database with personal data means members' emails and phone numbers end up written into log files, which often have less protection and more copies than the database itself. It is a clear case of a security measure that creates a privacy problem.
Audit table with triggers
To know who modified which row and when, the standard pattern is an audit table fed by a trigger —the TRIGGERs we introduced in 05-04—.
CREATE TABLE members_audit (
audit_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
member_id INTEGER NOT NULL,
operation TEXT NOT NULL CHECK (operation IN ('INSERT','UPDATE','DELETE')),
occurred_at TIMESTAMPTZ NOT NULL DEFAULT now(),
db_user TEXT NOT NULL DEFAULT current_user,
ip_address INET,
data_before JSONB,
data_after JSONB
);
CREATE INDEX idx_members_audit_member ON members_audit (member_id, occurred_at DESC);
CREATE FUNCTION fn_audit_members() RETURNS TRIGGER AS $$
BEGIN
INSERT INTO members_audit (member_id, operation, ip_address, data_before, data_after)
VALUES (
coalesce(NEW.member_id, OLD.member_id),
TG_OP,
inet_client_addr(),
CASE WHEN TG_OP IN ('UPDATE','DELETE') THEN to_jsonb(OLD) END,
CASE WHEN TG_OP IN ('INSERT','UPDATE') THEN to_jsonb(NEW) END
);
RETURN NULL; -- AFTER trigger: the returned value is ignored
END;
$$ LANGUAGE plpgsql SECURITY DEFINER;
CREATE TRIGGER trg_audit_members
AFTER INSERT OR UPDATE OR DELETE ON members
FOR EACH ROW EXECUTE FUNCTION fn_audit_members();Let us try it out:
UPDATE members SET email = 'marta.alsina.new@example.org' WHERE member_id = 14;
SELECT operation, occurred_at, db_user,
data_before ->> 'email' AS email_before,
data_after ->> 'email' AS email_after
FROM members_audit
WHERE member_id = 14
ORDER BY occurred_at DESC LIMIT 1;operation | occurred_at | db_user | email_before | email_after -----------+-------------------------------+------------+--------------------------+------------------------------- UPDATE | 2026-08-02 13:14:52.118+02 | u_alsina | marta.alsina@example.org | marta.alsina.new@example.org
Four design decisions worth understanding:
SECURITY DEFINERmakes the function run with the permissions of whoever created it, so the front desk role can generate audit records without having write permission on the audit table. That is exactly what you want: to be able to write to the log but not to erase it.current_useridentifies the database role. That is why it matters that each person has their own login role (section 5): with a shared account, this column says nothing useful.JSONBfor the before and after avoids having to rebuild the audit table every time themembersschema changes. It is a legitimate use ofjsonbof the kind 04-04 called "controlled escape hatches".- The cost: auditing doubles the writes and grows a table nobody queries daily. It is applied to the tables with sensitive data —
members,fines,payments—, not to all of them.
And a final warning: the audit table also contains personal data, including data that has been deleted from the original table. It needs the same restrictive permissions and the same retention policy as the data it audits.
- Personal data: minimization, pseudonymization and retention
⚠️ Important warning. What follows describes technical mechanisms for handling personal data in a database. It is not legal advice. Compliance with the General Data Protection Regulation (GDPR) and with the applicable legislation in each jurisdiction —including the specific obligations of a public administration such as Vallmar town council— must be reviewed by a compliance professional or the organization's data protection officer. Decisions about what data may be collected, on what legal basis, for how long and with what measures, are legal and organizational decisions, not technical ones. This lesson teaches how to implement what is decided, not what should be decided.
That said, there are four techniques every database professional should know.
Minimization
The most effective measure is not having the data.
Going over BiblioRed's schema with that question in mind:
| Data | What is it used for? | Decision |
|---|---|---|
members.email |
Due-date notices and available reservations | Kept: there is a clear function |
member_phones |
Calls about serious overdues | Kept, reviewing whether several per member are needed |
| Full date of birth | Only to know whether it is a child or adult card | Replaceable by the year, or by an is_minor flag |
| Full postal address | Nothing in the current system | Remove |
| ID card number | Verification at in-person enrollment | Verify and do not store, or store only a check |
Every piece of data that is not there cannot leak, does not have to be encrypted, does not have to be audited and does not have to be deleted.
Pseudonymization
Pseudonymizing means replacing the direct identifiers with a reference that does not identify anybody on its own, while keeping the mapping somewhere else and with different protections. It is reversible with the additional information.
In BiblioRed, the usage statistics table does not need to know who each member is:
CREATE TABLE usage_stats (
pseudonym TEXT NOT NULL,
stat_date DATE NOT NULL,
branch_id INTEGER NOT NULL REFERENCES branches(branch_id),
material_type TEXT NOT NULL,
loans INTEGER NOT NULL
);
-- The pseudonym is stable (it allows time series) and not invertible without the salt
INSERT INTO usage_stats (pseudonym, stat_date, branch_id, material_type, loans)
SELECT encode(digest(l.member_id::text || current_setting('app.stats_salt'), 'sha256'), 'hex'),
l.loan_date, c.branch_id, m.type, count(*)
FROM loans l
JOIN copies c ON c.copy_id = l.copy_id
JOIN materials m ON m.material_id = c.material_id
GROUP BY 1, 2, 3, 4;The key is the salt (app.stats_salt): without it, a sha256 of a numeric identifier is inverted by trying the 12,000 possible values in less than a second. A pseudonym without a salt is not a pseudonym.
Anonymization for test environments
Anonymizing means transforming the data so that it is no longer possible to identify the person, not even with additional information. Unlike pseudonymization, it is irreversible.
The practical case is the development environment. Copying the production database onto a developer's laptop —which is what half of all organizations do— means distributing the data of twelve thousand residents across uncontrolled machines, with no guaranteed disk encryption and no auditing.
An anonymization script for BiblioRed's test environment:
-- RUN ONLY ON THE TEST COPY. Never on production.
BEGIN;
UPDATE members SET
first_name = 'Member' || member_id,
last_name = 'Surname' || member_id,
email = 'member' || member_id || '@example.org';
UPDATE member_phones SET
number = '600' || lpad((100000 + member_id)::text, 6, '0');
-- Amounts and dates are kept: they are needed to test for real
-- The audit tables contain old data: they are emptied
TRUNCATE members_audit;
COMMIT;Three rules for an anonymization to be worth anything:
- It runs as part of the restore process, automatically, never by hand. A manual step will be forgotten.
- The shape of the data is preserved —lengths, distributions, volume— or the test environment will stop resembling production and the execution plans of 06-03 will not be comparable.
- You check that no identifiable data is left in secondary tables: auditing, logs,
event_reports, free-text fields where somebody noted "call the son's number 6XX".
That last point is the one that fails most often. Free-text comment fields are a well of personal data that no column-based anonymization detects.
Retention
Data kept forever is a risk kept forever.
-- Example of a technical policy: loan history more than 5 years old
-- (the specific period is a legal decision, not a technical one)
UPDATE loans
SET member_id = NULL
WHERE return_date < CURRENT_DATE - INTERVAL '5 years'
AND member_id IS NOT NULL;Notice that the loan is not deleted: it is unlinked from the member. The library keeps its circulation statistics —how many times each material was lent, at which branch, in which month—, which is what it needs for its management, and stops keeping who read what. It is the kind of solution that technology can indeed contribute to a retention decision.
This does require, of course, that loans.member_id accept nulls and that the foreign keys allow it, which is a design decision worth taking beforehand, in module 4, and not when the retention policy arrives.
- Backups: logical and physical
We move on to the council department's second question. Let us start with the statement that organizes the whole block:
An untested backup is not a backup: it is a hope.
PostgreSQL offers two families of backup, and they do not compete: they complement each other.
Logical backup: pg_dump and pg_restore
It generates a file with the statements needed to rebuild the data: CREATE TABLE, COPY, CREATE INDEX.
# Custom format (compressed, selective restore) — the recommended one
pg_dump -h localhost -U postgres -d biblioredb \
-F c -Z 6 -f /backups/biblioredb_2026-08-02.dump
# Schema only, for version control
pg_dump -d biblioredb --schema-only -f /backups/schema_2026-08-02.sql
# Only a few tables
pg_dump -d biblioredb -t members -t loans -F c -f /backups/partial.dumpOn success, pg_dump prints nothing: it only speaks up if there are problems. To see the progress on large databases use the -v option.
Restoring:
# The whole database into a new one
createdb -U postgres biblioredb_restored
pg_restore -U postgres -d biblioredb_restored -j 4 /backups/biblioredb_2026-08-02.dump
# A single table, which is where the custom format shines
pg_restore -U postgres -d biblioredb -t members /backups/biblioredb_2026-08-02.dump
# See what it contains without restoring anything
pg_restore -l /backups/biblioredb_2026-08-02.dump | head -20; ; Archive created at 2026-08-02 03:00:14 CEST ; dbname: biblioredb ; TOC Entries: 214 ; Compression: 6 ; Format: CUSTOM ; 215; 1259 16482 TABLE public members postgres 216; 1259 16490 TABLE public loans postgres ...
Important: pg_dump does not copy the roles or the passwords, which are global to the server. They are needed separately:
Forgetting this file is a classic: the data is restored and nobody can get in because no role exists.
Physical backup: pg_basebackup
It copies the files of the data directory as they are, at block level.
Logical backup (pg_dump) |
Physical backup (pg_basebackup) |
|
|---|---|---|
| What it copies | SQL statements to rebuild | The cluster's files |
| Granularity | One table, one schema or the whole database | The whole cluster, no exceptions |
| Portability | Across different versions and architectures | Only the same major version and architecture |
| Size | Smaller (no indexes, compressed) | Larger (includes indexes) |
| Backup speed | Slow on large databases | Fast |
| Restore speed | Slow: it re-runs everything and rebuilds indexes | Fast: it is copying files |
| Does it allow point-in-time recovery? | No | Yes, with WAL archiving |
| Typical use | Migrations, per-table backups, version changes | Disaster recovery |
The correct answer for BiblioRed is "both", and section 17 spells out the plan.
- Full, differential and incremental
The classic classification, applicable to any backup system:
| Type | What it copies | Space | Backup time | Restore time |
|---|---|---|---|---|
| Full | Everything | Maximum | Maximum | Minimum: a single file |
| Differential | What changed since the last full | Medium, growing | Medium | Medium: full + last differential |
| Incremental | What changed since the last backup of any type | Minimum | Minimum | Maximum: full + all the incrementals |
The trade-off is always the same: space and backup time against restore time and complexity. And there is a factor that decides more than the numbers:
The more files a restore needs, the more likely it is that one will fail. An incremental chain of thirty links breaks if number seventeen is missing.
In PostgreSQL, the practical translation of this scheme is:
- The full one is the
pg_basebackup. - The role of the incrementals is played by WAL archiving, which is continuous rather than periodic. And it is a better solution than classic incrementals, because it allows you to restore not just to the moment of a backup, but to any instant.
(PostgreSQL 17 also added native incremental backups with pg_basebackup --incremental, useful on very large databases; WAL archiving is still the basis of the scheme.)
- WAL archiving and point-in-time recovery
Here the write-ahead log from lesson 06-01 reappears, and its second life is as important as the first.
You will remember the mechanism: before modifying a data page, PostgreSQL writes an entry into the WAL describing the change. That log therefore contains the complete history of every modification since the moment the base backup was taken.
WAL archiving. If we keep a base backup and all the WAL segments generated since then, we can rebuild the database at any instant after the base backup, by applying the log up to the desired point. This is point-in-time recovery (PITR).
graph LR
A["03:00<br/>Full base backup<br/>(pg_basebackup)"] --> B["03:00 -> 11:47<br/>WAL segments<br/>archived non-stop"]
B --> C["11:47:03<br/>DELETE FROM members<br/>with no WHERE"]
C --> D["11:52<br/>The problem<br/>is spotted"]
D --> E["Restore:<br/>base backup + WAL<br/>up to 11:46:59"]
Configuration
# postgresql.conf wal_level = replica archive_mode = on archive_command = 'test ! -f /wal_archive/%f && cp %p /wal_archive/%f' archive_timeout = 300
| Parameter | What it does |
|---|---|
wal_level = replica |
Generates enough information in the WAL for recovery and replicas |
archive_mode = on |
Turns archiving on |
archive_command |
Command that copies each segment to a safe place |
archive_timeout = 300 |
Forces a segment to be closed every 5 minutes even if it is not full. It bounds the maximum loss |
That archive_timeout deserves attention because it is what sets the worst case: without it, a half-filled 16 MB segment might not be archived for hours, and that work would be lost. With 300 seconds, the maximum loss is bounded to five minutes.
And an important operational warning: if archive_command fails, PostgreSQL keeps the segments and the disk fills up. An archive_command pointing at an unreachable destination is a sure way to bring the server down within hours. Keep an eye on it:
SELECT archived_count, last_archived_wal, last_archived_time,
failed_count, last_failed_time
FROM pg_stat_archiver; archived_count | last_archived_wal | last_archived_time | failed_count | last_failed_time
----------------+--------------------------+-----------------------------+--------------+------------------
88412 | 000000010000000000000023 | 2026-08-02 13:15:02.118+02 | 0 |failed_count = 0 and a recent last_archived_time: archiving is working. These two columns should be on the monitoring dashboard.
Point-in-time restore
A real scenario: at 11:47 somebody runs the following in production, believing they are in the test environment:
At 11:52 it is spotted. The steps:
# 1) Stop the server. No hurry justifies skipping this.
sudo systemctl stop postgresql
# 2) Move the current data directory aside. Do NOT delete it: it may contain
# what was written between 11:47 and 11:52, and it will be needed to reconcile.
sudo mv /var/lib/postgresql/17/main /var/lib/postgresql/17/main_incident
# 3) Restore the base backup
sudo -u postgres mkdir -p /var/lib/postgresql/17/main
sudo -u postgres tar -xzf /backups/base_2026-08-02/base.tar.gz \
-C /var/lib/postgresql/17/main
# 4) State how far to apply the log
sudo -u postgres tee -a /var/lib/postgresql/17/main/postgresql.auto.conf <<'EOF'
restore_command = 'cp /wal_archive/%f %p'
recovery_target_time = '2026-08-02 11:46:59+02'
recovery_target_action = 'promote'
EOF
sudo -u postgres touch /var/lib/postgresql/17/main/recovery.signal
# 5) Start up: PostgreSQL will apply the WAL up to the given instant
sudo systemctl start postgresqlIn the server log:
LOG: starting point-in-time recovery to 2026-08-02 11:46:59+02 LOG: restored log file "000000010000000000000021" from archive LOG: restored log file "000000010000000000000022" from archive LOG: recovery stopping before commit of transaction 90412, time 2026-08-02 11:47:03.882+02 LOG: redo done at 0/22F1A8C0 LOG: selected new timeline ID: 2 LOG: archive recovery complete LOG: database system is ready to accept connections
Notice the decisive line: recovery stopping before commit of transaction 90412. That transaction 90412 is the DELETE. Recovery stops just before committing it.
The twelve thousand members are back. Only the operations of those five minutes between 11:47 and 11:52 have been lost, and they are rebuilt by hand from the directory set aside in step 2 and from the front desk's paper receipts.
Other possible recovery targets, besides recovery_target_time:
| Parameter | Stops recovery at |
|---|---|
recovery_target_time |
An instant |
recovery_target_xid |
A specific transaction (useful if you know it from the log) |
recovery_target_lsn |
An exact WAL position |
recovery_target_name |
A point marked beforehand with pg_create_restore_point('before_migration') |
That last one is pure gold before a schema migration:
Management tools
Doing all this by hand is feasible, but in production people use tools that manage retention, verification and a backup catalog: pgBackRest, Barman and WAL-G are the three usual ones. They all implement the same underlying mechanism we have just seen. Knowing the mechanism is what lets you understand the tool, and not the other way round.
- The 3-2-1 rule and verifying your backups
The 3-2-1 rule
3 copies of the data · on 2 different types of media · with 1 copy off site.
Applied to BiblioRed:
| Element | Implementation |
|---|---|
| Copy 1 | The production data, on the council's server |
| Copy 2 | Daily backup on the council's network storage |
| Copy 3 | Encrypted backup on the contracted external provider's storage |
| 2 media | The server's local disk + network storage / external storage |
| 1 off site | The copy at the external provider, in another city |
Modern additions to the rule, which experience with ransomware has made essential:
- 1 immutable copy: storage that does not allow modification or deletion for a set period. Without this, an attacker with access to the server encrypts or deletes the backups too, which is exactly what happens in ransomware incidents.
- 0 errors in verification: no backup counts until it has been restored successfully.
Verification: an untested backup is not a backup
This is the point where most organizations fail. The backup process is configured, it is seen to produce files, and nobody ever restores until the day of the disaster, which is the worst moment to discover that the file was truncated, that the roles were missing, or that the process had been backing up an empty database for seven months.
An automatic verification script, run weekly:
#!/bin/bash
# verify_backup.sh — restores the latest backup and checks that it makes sense
set -euo pipefail
BACKUP=$(ls -t /backups/biblioredb_*.dump | head -1)
TEST_DB="verification_$(date +%Y%m%d)"
echo "Verifying: $BACKUP"
createdb "$TEST_DB"
pg_restore -d "$TEST_DB" -j 4 "$BACKUP"
# Content checks: restoring without an error is not enough
MEMBERS=$(psql -tAc "SELECT count(*) FROM members" "$TEST_DB")
LOANS=$(psql -tAc "SELECT count(*) FROM loans" "$TEST_DB")
LATEST=$(psql -tAc "SELECT max(loan_date) FROM loans" "$TEST_DB")
echo "Members: $MEMBERS | Loans: $LOANS | Latest loan: $LATEST"
if [ "$MEMBERS" -lt 10000 ] || [ "$LOANS" -lt 2000000 ]; then
echo "ERROR: the backup does not contain the expected volumes"
dropdb "$TEST_DB"
exit 1
fi
# The date of the latest loan must be yesterday's or today's
if [[ "$LATEST" < $(date -d 'yesterday' +%Y-%m-%d) ]]; then
echo "ERROR: the backup is old; the process may have stopped"
dropdb "$TEST_DB"
exit 1
fi
dropdb "$TEST_DB"
echo "Verification successful"Verifying: /backups/biblioredb_2026-08-02.dump Members: 12000 | Loans: 2841077 | Latest loan: 2026-08-01 Verification successful
The volume and date checks are the heart of the script. A backup that restores without errors but contains an empty database restores perfectly, and is worth nothing. The classic mistake —backing up the wrong database, or one that stopped being used— is only caught by counting rows.
And once a year, a full drill: restore on a different server, start the application against the restored copy, and time how long it takes from the phone call to the service being up. That stopwatch is the real RTO, and it is almost always three times the one that had been estimated.
- RPO and RTO applied to BiblioRed
Two acronyms that organize any conversation about continuity, because they turn "we want to be safe" into two numbers that can be designed and budgeted for.
| Acronym | Name | Question it answers | Measured in |
|---|---|---|---|
| RPO | Recovery Point Objective | How much data can we afford to lose? | Working time lost |
| RTO | Recovery Time Objective | How long can the service be down? | Time unavailable |
Applied to BiblioRed's operations:
| Operation | Tolerable RPO | Tolerable RTO | Why |
|---|---|---|---|
| Loans and returns | 5 minutes | 2 hours | The front desk can write on paper for a while, but it cannot lose loans: those are books nobody will know the whereabouts of |
Fine collection (payments) |
0 | 2 hours | It is public money. A lost payment is a complaint |
| Member enrollment | 1 hour | 4 hours | It can be redone from the paper form |
| Event registrations | 1 hour | 4 hours | Annoying, recoverable |
| Statistics and reports | 24 hours | 3 days | It does not affect the service |
The technical decisions come straight out of the table:
| Requirement | Technical consequence |
|---|---|
| 5-minute RPO | WAL archiving with archive_timeout = 300. A daily backup alone would give a 24-hour RPO |
| RPO of 0 on payments | Synchronous replica (synchronous_commit = remote_apply), or accept that the card reader's paper slip is the fallback |
| 2-hour RTO | A verified physical backup and a written, rehearsed procedure. Restoring 2.8 million rows with pg_restore can take more than two hours |
| 2-hour RTO with margin | A standby replica ready to promote (the replication of 03-01), which cuts the RTO to minutes |
Notice the logic: first you decide how much can be lost and how long you can wait; then you choose the technology. Doing it the other way round —building the infrastructure and seeing what RPO comes out— is like designing the schema without knowing the domain.
- BiblioRed's backup plan, annotated
With everything above, this is the concrete plan:
| When | What | Where | Retention | Covers |
|---|---|---|---|---|
| Continuous | WAL archiving (archive_timeout = 300) |
Network storage + external copy | 35 days | 5-min RPO; PITR |
| Daily 03:00 | Full pg_dump -F c |
Network storage | 14 days | Per-table restore; migrations |
| Daily 03:30 | pg_dumpall --globals-only |
Alongside the daily backup | 14 days | Roles and passwords |
| Weekly, Sunday 02:00 | Full pg_basebackup |
Network storage + encrypted external copy | 8 weeks | Base for PITR; low RTO |
| Monthly, 1st | Full backup | Immutable external storage | 12 months | Ransomware; auditing |
| Weekly, Monday 06:00 | Automatic verification (script from section 15) | Test server | Log 12 months | That the backups are usable |
| Annual | Full restore drill | Alternative server | Report | Measure the real RTO |
And the decisions behind it, one by one:
Why both logical and physical backups. The physical one gives the low RTO (restoring is copying files) and is the only possible basis for PITR. The logical one allows restoring a single table —which is what is needed 90% of the time, because the typical disaster is not a dead disk, it is a badly written DELETE— and is the only one that works for migrating to a different major version.
Why the WAL is retained for 35 days and the weekly base for 8 weeks. The WAL is only useful if the corresponding base backup exists. Retaining 35 days of WAL with no bases older than 35 days would be useless; conversely, retaining bases without their WAL prevents PITR. WAL retention must cover, with a margin, that of the oldest base backup you want to use for PITR.
Why an immutable copy. Because an attacker who compromises the server and has the backup credentials will delete or encrypt the backups too. The monthly immutable copy is the last resort, and its one-month RPO is terrible but infinitely better than nothing.
Why verification is weekly and not monthly. Because the typical failure of a backup process is silent —a permission that changed, a full disk, a path that stopped existing— and one week is the maximum acceptable time to detect it.
Why an annual drill, if there is already weekly verification. Because they are different things. Verification checks that the backup file is good; the drill checks that the procedure works with people involved: that somebody knows where the document is, that the external storage credentials are still valid, that the person who wrote it is not the only one who knows how to do it, and how long it really takes.
What is missing from this plan. It is worth saying too: there is no standby replica. With the 2-hour RTO of section 16 it is not essential, but it is the first investment that would be needed if the library decided to lower that objective. The primary-standby replication of 03-01 is exactly that piece.
- The first minutes after an accidental deletion
The moment when most damage is done is the one immediately following the mistake, when somebody tries to fix it in a hurry. This is the procedure, and it is worth having printed out.
Minute 0 — Stop. Do not run anything else. Do not try to "insert the data back". Every subsequent write complicates the reconciliation and may overwrite recoverable information.
Minute 1 — Is the transaction still open? If the DELETE was run inside an uncommitted BEGIN, the solution is a ROLLBACK and that is that (06-01). Check who has what open:
SELECT pid, usename, state, now() - xact_start AS duration, left(query, 60) AS query
FROM pg_stat_activity
WHERE state IN ('idle in transaction', 'active')
ORDER BY xact_start;pid | usename | state | duration | query -------+----------+---------------------+-----------------+------------------------------------ 41902 | u_alsina | idle in transaction | 00:03:12.881021 | DELETE FROM members
There it is: the transaction is still open. A ROLLBACK in that session resolves the whole incident. It is worth checking this before anything else.
Minute 2 — Isolate. If it is already committed, prevent new writes while a decision is made:
-- Cut off the application's access without stopping the server
REVOKE CONNECT ON DATABASE biblioredb FROM bibliored_app, bibliored_frontdesk;
-- Close those roles' existing sessions
SELECT pg_terminate_backend(pid)
FROM pg_stat_activity
WHERE datname = 'biblioredb'
AND usename IN ('app_bibliored','u_alsina','u_pereda');Minute 3 — Determine the exact scope. Which table, how many rows, at what time, with which user. The audit table from section 10 answers all of this:
SELECT operation, min(occurred_at) AS from_, max(occurred_at) AS to_,
db_user, count(*) AS rows
FROM members_audit
WHERE occurred_at > now() - INTERVAL '1 hour'
GROUP BY operation, db_user;operation | from_ | to_ | db_user | rows -----------+-----------------------------+-----------------------------+------------+------- DELETE | 2026-08-02 11:47:02.118+02 | 2026-08-02 11:47:04.882+02 | u_alsina | 12000
Minute 5 — Decide the recovery route:
| Situation | Route |
|---|---|
| Transaction still open | ROLLBACK |
Few rows and there is an audit trail with data_before |
Reinsert from the audit table |
| A whole table, and the daily backup will do | pg_restore -t members into an auxiliary database, and copy the rows over |
| Wide scope, or data later than the backup is needed | PITR to the previous instant (section 14) |
The audit-table option, when it applies, is the most surgical:
INSERT INTO members (member_id, first_name, last_name, email, join_date, branch_id, active)
SELECT (data_before ->> 'member_id')::int,
data_before ->> 'first_name',
data_before ->> 'last_name',
data_before ->> 'email',
(data_before ->> 'join_date')::date,
(data_before ->> 'branch_id')::int,
(data_before ->> 'active')::boolean
FROM members_audit
WHERE operation = 'DELETE'
AND occurred_at BETWEEN '2026-08-02 11:47:00+02' AND '2026-08-02 11:47:10+02';Minute 30 — Afterwards. Restore the permissions, check consistency with the control queries from 05-03, and write down what happened. The post-incident report is not looking for culprits: it is looking for why it was possible. Almost always the answer is one of these three: the connection had more permissions than it needed, there was no way to tell the production environment from the test one, or there was no procedure for bulk operations.
Three preventive measures come out of that:
-- 1) The front desk should not be able to delete anything
REVOKE DELETE ON members, loans, fines, payments FROM bibliored_frontdesk;
-- 2) Make the psql prompt shout in production
-- (in ~/.psqlrc on the production server)
\set PROMPT1 '%[%033[1;31m%]PRODUCTION%[%033[0m%] %/=# '
-- 3) A restore point before any bulk operation
SELECT pg_create_restore_point('before_history_purge');
- The minimum equivalent in SQLite
To close, the reduced version of the same problem.
SQLite has no users, no roles and no GRANT: the permissions are those of the file in the operating system. Whoever can read the file reads everything; whoever can write it writes everything. The whole first half of this lesson has no equivalent, and that is an important reason not to use SQLite in a multi-user system with personal data.
Backups do have an equivalent, and it is simple:
# CORRECT: hot, consistent backup with the database in use
sqlite3 biblioredb.db ".backup '/backups/biblioredb_2026-08-02.db'"
# CORRECT: logical dump, equivalent to pg_dump
sqlite3 biblioredb.db ".dump" > /backups/biblioredb_2026-08-02.sql
# CORRECT: check the file's integrity
sqlite3 biblioredb.db "PRAGMA integrity_check;"And the central warning:
Copying the
.dbfile withcpwhile the database is in use produces a corrupt backup, because it may capture the file halfway through a transaction, and in WAL mode it leaves out the-walfile with the recent changes. Copying the file directly is only safe with the database closed and no connection open.
.backup is safe hot: it uses SQLite's internal backup mechanism, which guarantees a consistent image. It is a one-line difference that separates a valid backup from a useless one.
Common Mistakes and Tips
Connecting the application as a superuser. It nullifies the entire permission setup and turns any injection into a compromise of the server. It is the most serious mistake and the easiest to fix.
Granting SELECT on the tables and forgetting USAGE on the schema. The role will see nothing and you will get a confusing error report. It is the number one failure when configuring permissions for the first time.
Using GRANT ... ON ALL TABLES and believing it covers the future. It only affects existing tables. Without ALTER DEFAULT PRIVILEGES, the table created tomorrow will be inaccessible or —worse, depending on the setup— accessible to the wrong people.
Leaving trust in pg_hba.conf. Even "just for a moment, to test something". That moment lasts three years.
Using sslmode=require and believing it is secure. It encrypts, but it does not verify who it is talking to. Use verify-full.
Escaping quotes by hand instead of parameterizing. One forgotten place out of two hundred is enough. Parameterized queries do not have that failure mode.
Showing the database's error to the end user. It gives away the schema. Log the full error internally and show a generic message.
Copying the production database onto a development laptop without anonymizing. It is the most common data leak and the one least perceived as such. Anonymize as an automatic part of the restore process.
Turning on log_statement = 'all' on a database with personal data. Emails and phone numbers end up in log files with less protection and more copies than the database.
Having backups and never having restored one. It is this lesson's mistake. An untested backup is not a backup. Verify weekly and automatically, and run a full drill once a year.
Not backing up the roles. pg_dump does not include them. Without pg_dumpall --globals-only, you will restore the data and be unable to get in.
Not watching pg_stat_archiver. If archive_command fails, the WAL piles up, the disk fills and the server stops. failed_count belongs on a dashboard.
Confusing masking with anonymizing. m***@example.org is still personal data in a record that contains the name and the branch. Anonymizing means replacing, not covering up.
Final tip: write the recovery procedure down and keep it outside the system it protects. A restore document stored only on the server that has burned down is a joke that is only funny before the fire. On paper, in the external storage, and with the necessary credentials held by at least two people.
Exercises
Exercise 1: A fourth role with least privilege
Vallmar's culture department hires an external firm to analyze library usage over six months. The firm needs: to read the loans, copies, materials, branches and events; it must not be able to identify any member; it must not be able to write anything; and its access must expire automatically on 31 December 2026.
Write the complete SQL: the role, the permissions, any view you need, and the check that it cannot reach the personal data.
Exercise 2: Diagnose and fix a configuration
A technician has left this configuration on BiblioRed's production server. Identify all the security problems, ordered by severity, and write the fix for each one.
CREATE ROLE app_bibliored LOGIN PASSWORD 'bibliored2026' SUPERUSER;
GRANT ALL ON ALL TABLES IN SCHEMA public TO PUBLIC;Exercise 3: Design the backup plan for a new case
Vallmar's library network adds a musical instrument lending service, with its own instruments database. Its characteristics: 400 instruments, 900 users, around 30 operations a day, and the deposit amounts (up to €300 per instrument) are recorded in a deposits table.
Define the RPO and the RTO of each type of operation, justify them, and write the backup plan that follows from them. Explain how it differs from BiblioRed's plan in section 17 and why.
Solutions
Solution 1
-- =========================================================
-- 1) A view with no identifiable member data at all
-- =========================================================
CREATE VIEW v_loans_analytics AS
SELECT l.loan_id,
-- Stable pseudonym, not invertible without the salt
encode(digest(l.member_id::text || current_setting('app.analytics_salt'), 'sha256'), 'hex')
AS member_pseudonym,
l.loan_date,
l.due_date,
l.return_date,
c.branch_id,
c.material_id,
m.type AS material_type
FROM loans l
JOIN copies c ON c.copy_id = l.copy_id
JOIN materials m ON m.material_id = c.material_id;
-- =========================================================
-- 2) Group role with minimum permissions
-- =========================================================
CREATE ROLE bibliored_analytics NOLOGIN;
GRANT CONNECT ON DATABASE biblioredb TO bibliored_analytics;
GRANT USAGE ON SCHEMA public TO bibliored_analytics;
GRANT SELECT ON v_loans_analytics,
copies, materials, branches,
v_events_published, event_types, rooms
TO bibliored_analytics;
-- Explicit and verifiable: nothing sensitive
REVOKE ALL ON members, member_phones, fines, payments,
registrations, members_audit, loans
FROM bibliored_analytics;
-- So that a future object is not granted to it by oversight
ALTER DEFAULT PRIVILEGES IN SCHEMA public
REVOKE ALL ON TABLES FROM bibliored_analytics;
-- =========================================================
-- 3) Login role with an expiry date and a limit
-- =========================================================
CREATE ROLE u_consultant LOGIN
PASSWORD 'a-long-randomly-generated-password'
VALID UNTIL '2026-12-31 23:59:59+01'
CONNECTION LIMIT 3;
GRANT bibliored_analytics TO u_consultant;And in pg_hba.conf, restricting the origin and forcing TLS:
Check:
SET ROLE bibliored_analytics;
SELECT count(*) FROM v_loans_analytics; -- must work
SELECT email FROM members LIMIT 1; -- must fail
SELECT * FROM loans LIMIT 1; -- must fail (member_id in the clear)
INSERT INTO loans (member_id) VALUES (14); -- must fail
RESET ROLE;count --------- 2841077 ERROR: permission denied for table members ERROR: permission denied for table loans ERROR: permission denied for table loans
Key points of the solution:
loansis revoked and access is given only to the view. GrantingSELECTonloanswould have handed overmember_idin the clear, which combined with the dates allows specific people to be re-identified with little effort.VALID UNTILmakes the access expire on its own. Trusting somebody to remember to revoke it in December is trusting too much.CONNECTION LIMIT 3stops a badly configured analytics tool from opening two hundred connections and affecting the front desk service.hostsslwith the specific address limits access to the consultancy's network and forces encryption.- Compliance note: the pseudonym makes re-identification harder, but a data set with dates, branch and materials may still be re-identifiable by combination. Whether this treatment is sufficient and on what legal basis it is disclosed to a third party is a question the council's data protection officer must validate, not the technical team.
Solution 2
Problem 1 (critical) — SUPERUSER on the application's connection.
It nullifies every permission and every row policy, and turns any injection into operating system command execution through COPY ... FROM PROGRAM.
ALTER ROLE app_bibliored NOSUPERUSER;
ALTER ROLE app_bibliored CONNECTION LIMIT 40;
GRANT bibliored_app TO app_bibliored;Problem 2 (critical) — GRANT ALL ... TO PUBLIC.
PUBLIC includes every present and future role. Anybody who manages to connect has full control over every table, including DELETE and TRUNCATE.
REVOKE ALL ON ALL TABLES IN SCHEMA public FROM PUBLIC;
REVOKE ALL ON SCHEMA public FROM PUBLIC;
REVOKE ALL ON DATABASE biblioredb FROM PUBLIC;
-- And apply the role script from section 4Problem 3 (critical) — SQL injection by concatenation.
sql = "SELECT member_id, first_name, last_name FROM members WHERE last_name = %s"
cursor.execute(sql, (last_name,))Besides, the SELECT * is removed: the query should ask only for the columns it uses, which avoids exposing email by accident and helps the Index Only Scan of 06-03.
Problem 4 (critical) — local all all trust.
Any operating system user on the server gets in as any role, including postgres, with no password.
Problem 5 (serious) — host all all 0.0.0.0/0.
The database accepts connections from anywhere on the internet, to any database and with any user.
hostssl biblioredb bibliored_frontdesk 10.20.0.0/16 scram-sha-256 hostssl biblioredb app_bibliored 10.20.5.11/32 scram-sha-256 host all all 0.0.0.0/0 reject
Problem 6 (serious) — md5 instead of scram-sha-256.
A method with known weaknesses. The method is changed and the passwords are reset, because changing the method does not convert the existing ones on its own:
Problem 7 (serious) — No TLS.
Every line was host, not hostssl: the traffic travels in the clear. Fixed in problem 5, plus ssl = on in postgresql.conf.
Problem 8 (moderate) — A weak and predictable password.
bibliored2026 is guessed on the first attempt of a targeted attack. Use a long randomly generated password, stored in a secrets manager, never in the code repository, with planned rotation.
Recommended order of intervention: first close down pg_hba.conf (problems 4, 5, 6 and 7), which stops the improper access immediately; then remove SUPERUSER and PUBLIC (1 and 2), which limit the damage; and in parallel fix the code (3), which requires a deployment. And as soon as the urgent part is closed: review the logs to determine whether this configuration was exploited, which is an incident that must be reported through the organization's channels.
Solution 3
RPO and RTO per operation:
| Operation | RPO | RTO | Justification |
|---|---|---|---|
Recording deposits (deposits) |
0 | 4 hours | It is citizens' money, up to €300 per instrument. A lost deposit is a complaint and a problem for the municipal auditor |
| Lending and returning instruments | 1 hour | 8 hours | 30 operations a day: in an hour you lose 2 or 3 operations, rebuildable from the paper receipt |
| User enrollment | 24 hours | 24 hours | It is redone from the enrollment form |
| Instrument catalog | 24 hours | 24 hours | It changes very little |
Resulting backup plan:
| When | What | Where | Retention |
|---|---|---|---|
| Continuous | WAL archiving, archive_timeout = 600 |
Network storage + external | 35 days |
| Daily 03:00 | Full pg_dump -F c |
Network storage | 30 days |
| Daily 03:15 | pg_dumpall --globals-only |
Alongside the daily backup | 30 days |
| Weekly | pg_basebackup |
Network storage + encrypted external | 8 weeks |
| Monthly | Full backup | Immutable external storage | 12 months |
| Weekly | Automatic verification with a restore | Test server | Log 12 months |
| Annual | Full drill | Alternative server | Report |
Differences from BiblioRed's plan and why:
-
archive_timeout = 600instead of 300. With 30 operations a day, one WAL segment every 10 minutes is more than enough and generates far fewer files to archive and retain. The effective 10-minute RPO is still better than the 1-hour requirement for lending. -
Logical backup retention of 30 days instead of 14. The database is tiny —900 users and 400 instruments—, so 30 compressed daily backups take up a few megabytes. When space is not a constraint, you retain more, because the error that takes longest to detect is silent logical corruption: somebody modified some data badly three weeks ago and nobody noticed.
-
A more relaxed RTO (4-8 hours against 2). The instrument service is not critical to the libraries' daily operation and can run on paper for a working day. That makes any investment in a standby replica unnecessary.
-
The RPO of 0 on
depositsdoes not force a synchronous replica here. Unlike BiblioRed's fine payments —which arrive by card reader and leave no receipt at the library—, an instrument deposit is collected with a paper receipt signed by the user. That receipt is the RPO 0 fallback, and it is far cheaper than the equivalent infrastructure. It is a good example that a continuity requirement is not always solved with technology. -
The same verification and drill regime. This point is not relaxed because of the database's size. The probability of a backup process failing silently does not depend on how many rows there are, and a small database usually has less supervision, not more. It is exactly where automatic verification is most needed.
Conclusion
The two questions from Vallmar's council department now have answers, and both are concrete answers.
Who can look up members' phone numbers and email addresses? The bibliored_app role, because the web application needs to send the due-date notices, and nobody else. Front desk staff can modify an email address but not read it in a listing. Management accesses v_members_public, where those columns do not exist. The external consultancy works on v_loans_analytics with salted pseudonyms. And every change to members is recorded in members_audit with the role, the originating address and the previous value. The answer is no longer "I don't know": it is a GRANT script that can be read, checked with has_column_privilege and audited.
What would happen if the disk died tonight? The latest pg_basebackup would be restored, the archived WAL would be applied up to the last available segment, and the service would come back with a maximum loss of five minutes and in under two hours. And that statement is not a hope: it is a written procedure, verified automatically every Monday with a script that counts rows and checks dates, and rehearsed in full once a year with a stopwatch.
Along the way we have gone through the complete layers. Authentication, with roles that are both users and groups, scram-sha-256 as the method and that pg_hba.conf that is read from top to bottom and must end in reject. Authorization, with its hierarchy of database, schema and table —and the USAGE everybody forgets—, column-level privileges, ALTER DEFAULT PRIVILEGES for objects that do not exist yet, and the principle of least privilege turned into three concrete, testable roles. Views for hiding columns and row-level security for hiding rows, with the warning that neither protects against a superuser. SQL injection, with the catalog search returning twelve thousand email addresses and the parameterized query turning the same attack into a search with no results, because it escapes nothing: it separates the paths. Encryption in transit with verify-full and not with require, at rest at the disk level, and passwords that are not encrypted but derived with a salt and a cost. Auditing with SECURITY DEFINER, which allows writing to the log without being able to erase it. And personal data, with minimization being the only measure that cannot fail, pseudonymization that without a salt is not pseudonymization, anonymization that must be an automatic step of the restore process, and the reminder —worth repeating— that these are technical mechanisms and that regulatory compliance is decided by a compliance professional, not by the development team.
And the backup block, where the WAL from 06-01 has had its second life: what in that lesson was the mechanism guaranteeing a COMMIT's durability against a power cut has here become the complete history of modifications that lets you stop recovery just before transaction 90412, the one that deleted twelve thousand members at 11:47:03. We have seen logical and physical backups with their different and complementary roles, the 3-2-1 rule with its modern additions of immutability, RPO and RTO as the two numbers that turn "we want to be safe" into engineering decisions, a backup plan annotated decision by decision, and the first-minutes procedure, which starts with the most counterintuitive and most effective thing of all: stop and check whether the transaction is still open.
With this, module 6 closes and, with it, the course's theoretical block. It is worth looking back at the whole journey. We started in module 1 with an overflowing spreadsheet and opened up the DBMS to see its pieces from the inside. In module 2 we learned the relational model and SQL as far as aggregates and referential integrity. In module 3 we left the relational world to understand NoSQL, the CAP theorem and eventual consistency, and came back knowing what is gained and what is paid on each side. In module 4 we designed BiblioRed's schema from the entity-relationship diagram to the last CHECK. In module 5 we put it through a formal examination, found a real flaw in fines and justified every denormalization in writing. And in module 6 we stopped looking at the schema and started looking at the system in operation: the transactions that happen in full or do not happen at all, the concurrency that breaks correct code as soon as there are two people, the indexes that turned fourteen seconds into forty milliseconds, and the security and backups that separate a database from an accident waiting to happen.
What is left of the course is no longer new theory: it is doing it with your hands. Module 7, Practical Exercises, is where all of the above turns into skill, and it is organized along the same journey: SQL exercises over the BiblioRed schema you now know inside out (07-01); schema design exercises, with new domains you will have to model from scratch (07-02); normalization exercises, with tables hiding functional dependencies you now know how to spot (07-03); and advanced query and transaction exercises, where the two parallel psql sessions, the isolation levels and the execution plans of this module will come back (07-04). After that, module 8 will go through three complete case studies —relational, non-relational and polyglot persistence— and module 9 will gather the books, courses and tools to carry on with on your own. Open the two terminals, keep the BiblioRed schema to hand, and see you in the first exercise.
Database Fundamentals
Module 1: Introduction to Databases
- Basic Database Concepts
- Types of Databases
- History and Evolution of Databases
- Database Management Systems and Architecture
Module 2: Relational Databases
- The Relational Model
- The SQL Language
- Basic SQL Operations
- Multi-Table Queries: JOINs and Subqueries
- Data Aggregation and Grouping
- Referential Integrity
Module 3: Non-Relational Databases
- Introduction to NoSQL
- Types of NoSQL Databases
- Data Modeling in NoSQL
- Comparing Relational and Non-Relational Databases
Module 4: Schema Design
- Schema Design Principles
- Entity-Relationship (ER) Diagrams
- Transforming ER Diagrams into Relational Schemas
- Data Types and Constraints
Module 5: Normalization
Module 6: Transactions, Performance, and Security
- Transactions and ACID Properties
- Concurrency and Isolation Levels
- Indexes and Query Optimization
- Security, Permissions, and Backups
Module 7: Practical Exercises
- SQL Exercises
- Schema Design Exercises
- Normalization Exercises
- Advanced Query and Transaction Exercises
Module 8: Case Studies
- Case Study: Relational Database
- Case Study: Non-Relational Database
- Case Study: Polyglot Persistence
