Reservalia's pipeline already builds a reproducible artifact, but building is not the same as working: tsc can compile without complaint code that returns the wrong answer. Automated tests are what turn the pipeline's green into a statement with content. In this lesson we will look at what types of test exist and which ones make sense to run on every pull request and which do not, because throwing everything in is the fastest route to a 40-minute pipeline that the team ends up ignoring. We will write real code with Vitest: a unit test of the appointment availability logic and an integration test against the PostgreSQL from the services: block we brought up in 02-02. We will talk about coverage without turning it into a target, we will devote a whole section to the greatest destroyer of trust in a pipeline — flaky tests — and we will close with parallelisation and with the criterion of what exactly blocks a merge. What we will not touch here are linters and static analysis, which are lesson 02-05.
Contents
- The test pyramid applied to the pipeline
- What runs on every pull request and what does not
- A unit test of appointment availability
- An integration test against PostgreSQL
- Code coverage: a signal, not a target
- Flaky tests and the quarantine policy
- Parallelisation and
matrix - Reservalia's
testjob and what blocks the merge - Common Mistakes and Tips
- Exercises
- Conclusion
- The test pyramid applied to the pipeline
The test pyramid says something very simple: many fast, cheap tests at the bottom, few slow, expensive ones at the top. The reason is not aesthetic, it is economic: each level is roughly an order of magnitude slower and more fragile than the one below.
flowchart TD
E["E2E · a few dozen · minutes"] --> C["Contract · dozens · seconds"]
C --> I["Integration · hundreds · seconds"] --> U["Unit · thousands · milliseconds"]
| Type | What it verifies | What it needs | Typical speed | Fragility |
|---|---|---|---|---|
| Unit | An isolated function or class | Nothing external | 1-10 ms | Very low |
| Integration | Several pieces together: code + database | A real PostgreSQL | 50-500 ms | Low |
| Contract | That api and web still understand each other |
A shared schema | 10-100 ms | Low |
| End-to-end (E2E) | A complete user journey | The whole system deployed | 5-60 s | High |
At Reservalia the translation is direct. Unit: given opening hours of 09:00 to 14:00 with a midday break, which 30-minute slots are free? Integration: when an appointment is inserted, does the database constraint stop another one being created on top of it? Contract: is the Appointment type returned by apps/api the one apps/web expects? — in a monorepo with shared types, a good part of this is done by tsc for free. E2E: a customer opens the public site, picks a business, a day and a time, and receives a confirmation email.
- What runs on every pull request and what does not
This is the most important design decision in the test pipeline, and it is governed by a simple formula: value of the information ÷ time it costs to obtain it.
| Test | On every PR? | Blocks the merge? | Reason |
|---|---|---|---|
| Unit | Yes | Yes | Seconds; they catch most logic errors |
| Integration | Yes | Yes | A couple of minutes; they catch what unit tests cannot see |
| Contract | Yes | Yes | Cheap and they stop you breaking the other application |
| Critical E2E (2-3 journeys) | Yes | Yes | It is the flow that makes money: booking an appointment |
| Full E2E (~40 journeys) | No | No | 25 minutes; they run overnight against main |
| Performance, load and security | No | No | The subject of lessons 04-04 and 04-03 |
Reservalia agrees that the set that blocks a PR must fit in 10 minutes, rule 5 of the agreement in 02-01. The full E2E journeys run in a separate workflow, scheduled with schedule in the small hours: if something breaks there, it shows up first thing as an incident, not as a blocked PR.
The "let us put everything in the PR" trap. It is a decision that looks prudent and is paid for in trust: when the pipeline takes 40 minutes, the team starts merging without waiting, re-running without looking and treating red as background noise. A smaller test set that is respected protects you more than a huge one that is ignored.
- A unit test of appointment availability
The logic lives in apps/api/src/domain/schedule.ts, with this signature — openingHours is the business's working day, busy are appointments and breaks, and durationMin the service duration:
export interface Interval { start: string; end: string } // "HH:MM"
export function calculateSlots(
openingHours: Interval, busy: Interval[], durationMin: number,
): Interval[] { /* ... */ }And this is the test, in apps/api/tests/unit/schedule.test.ts:
import { describe, it, expect } from 'vitest';
import { calculateSlots } from '../../src/domain/schedule';
describe('calculateSlots', () => {
const workday = { start: '09:00', end: '11:00' };
it('returns every slot when the schedule is empty', () => {
const slots = calculateSlots(workday, [], 30);
expect(slots).toHaveLength(4); // 9:00, 9:30, 10:00, 10:30
expect(slots[0]).toEqual({ start: '09:00', end: '09:30' });
});
it('excludes the interval of an existing appointment', () => {
const slots = calculateSlots(workday, [{ start: '09:30', end: '10:00' }], 30);
expect(slots.map(s => s.start)).toEqual(['09:00', '10:00', '10:30']);
});
it('does not offer a slot that overlaps with the break', () => {
const breakTime = [{ start: '09:45', end: '10:15' }]; // ← the real bug case
expect(calculateSlots(workday, breakTime, 30).map(s => s.start))
.toEqual(['09:00', '10:30']);
});
it('does not offer a slot that runs past the working day', () => {
expect(calculateSlots({ start: '09:00', end: '09:40' }, [], 30))
.toEqual([{ start: '09:00', end: '09:30' }]);
});
});Four things that make this a good test:
describegroups anditdescribes a behaviour in natural language. When it fails, the test name already tells you what has broken without opening the code.- There is no clock and no database. All the data are literals in the test itself. That is why it takes milliseconds and cannot be flaky.
- The third case is Diego's real bug: a 30-minute slot starting at 9:30 overlaps a break that begins at 9:45. Every bug fix should start with a test that reproduces it; that way the pipeline guarantees it does not come back.
- The edge cases are covered: empty schedule, partial occupancy, overlap and a slot that does not fit. It runs with
npm run test:unit --workspace apps/api, which underneath isvitest run tests/unit.
- An integration test against PostgreSQL
Unit tests cannot see the database's constraints: the application calculating slots correctly does not stop two simultaneous requests creating two overlapping appointments, and only PostgreSQL can guarantee that.
// apps/api/tests/integration/appointments.test.ts
import { describe, it, expect, beforeEach, afterAll } from 'vitest';
import { Pool } from 'pg';
import { createAppointment } from '../../src/routes/appointments';
const pool = new Pool({ connectionString: process.env.DATABASE_URL }); // 1
beforeEach(async () => {
await pool.query('TRUNCATE appointments, businesses RESTART IDENTITY CASCADE'); // 2
await pool.query(`INSERT INTO businesses (id, name, opening_time, closing_time)
VALUES (1, 'Sol Hair Salon', '09:00', '14:00')`);
});
afterAll(async () => { await pool.end(); }); // 3
describe('createAppointment', () => {
it('saves a valid appointment', async () => {
const appointment = await createAppointment(pool, {
businessId: 1, start: '2026-03-02T10:00:00+01:00', durationMin: 30,
});
const { rows } = await pool.query('SELECT * FROM appointments WHERE id = $1', [appointment.id]);
expect(rows).toHaveLength(1);
expect(rows[0].business_id).toBe(1);
});
it('rejects an appointment that overlaps an existing one', async () => {
await createAppointment(pool, { businessId: 1, start: '2026-03-02T10:00:00+01:00', durationMin: 30 });
await expect(
createAppointment(pool, { businessId: 1, start: '2026-03-02T10:15:00+01:00', durationMin: 30 }),
).rejects.toThrow('overlap'); // 4
});
});DATABASE_URLcomes from the environment, it is not written in the code. Locally thedocker-compose.ymlprovides it; in CI, theservices:block from 02-02. The test is identical in both places. The schema is created beforehand by runningnpm run migrate, so the pipeline verifies along the way that the migrations work (lesson 04-06).beforeEachwithTRUNCATEis the key to isolation: every test starts from a known base. Without it, execution order changes the result, and that is exactly a flaky test.afterAllcloses the pool. If you do not, the Vitest process never finishes and the job hangs untiltimeout-minutes. (4) The second case can only be verified here: the rejection is produced by a PostgreSQL exclusion constraint, not by the application code.
Locally, docker compose up -d followed by npm run test:integration --workspace apps/api is all you need.
- Code coverage: a signal, not a target
Coverage measures what percentage of the code was executed during the tests. It is generated with npm run test --workspace apps/api -- --coverage and configured like this in apps/api/vitest.config.ts:
coverage: {
provider: 'v8',
reporter: ['text', 'lcov', 'json-summary'], // console, tooling, summary
exclude: ['**/migrations/**', '**/*.d.ts', 'tests/**'], // what carries no signal
thresholds: { lines: 70, functions: 70, branches: 60 },
}To publish it as a job summary, $GITHUB_STEP_SUMMARY is a special file: whatever you write into it appears on the workflow page, with no external tooling.
- name: Publish the coverage summary
if: always() # even if some test fails
run: |
echo "### Coverage for apps/api" >> $GITHUB_STEP_SUMMARY
npx nyc report --reporter=text-summary >> $GITHUB_STEP_SUMMARYNow the uncomfortable part. Coverage measures what runs, not what is checked. A test like it('does not break', () => { calculateSlots(workday, [], 30); }) — without a single expect — gives 100% coverage of that function while verifying absolutely nothing.
That is why the threshold is a signal: it is there to detect that a new module has landed with no tests at all, not to certify quality. How to use it well:
- Set the threshold at the current level, not at an ideal. If you are at 68% today, set 68 and raise it when you exceed it naturally. An unreachable threshold ends up being switched off.
- Watch the coverage of new code, not the global figure — it is the "clean new code" quality gate idea from 02-05 — and exclude what carries no signal: migrations, configuration, types.
- Never turn coverage into a team target. It is a textbook case of Goodhart's law (lesson 01-05): as soon as the number is rewarded, tests with no assertions appear that raise it while verifying nothing.
- Flaky tests and the quarantine policy
A flaky test is one that, with no change to the code, sometimes passes and sometimes fails. It is the most corrosive problem a pipeline can have, because it destroys the meaning of red: if red might just be "bad luck", nobody investigates it again.
6.1. Why they appear
| Cause | Example in Reservalia | How it gets fixed |
|---|---|---|
| Real time | A test uses new Date() and fails at midnight or in another time zone |
Inject the date or freeze the clock |
| Execution order | One test leaves appointments behind that another finds | TRUNCATE in beforeEach |
| Concurrency | Two tests running in parallel share the same database | A database or schema per process |
| Fixed waits | await sleep(500) trusting that it is enough |
Wait for the condition, not for the clock |
| External resources | A test calls a real service or asks for port 3000 | A test double, a dynamic port |
6.2. How to detect them and what to do with them
The simplest way is repetition: npx vitest run tests/integration/appointments.test.ts --repeat 20 gives away a test that fails one time in twenty. In the pipeline, a nightly workflow that runs the whole suite several times against the same commit finds the flaky ones before a colleague finds them at six in the evening; recording each failure (test, date, commit) lets you see which ones reoffend.
Reservalia's quarantine policy. When a flaky test is detected, the same day: it is marked as skipped with an explicit reference to the issue — it.skip('sends the reminder 24 h beforehand [FLAKY · RES-412]', ...); an issue is opened with links to the run that failed and the one that passed; it is assigned to somebody with a deadline, because without a deadline quarantine turns into a graveyard; and if nobody fixes it within two weeks, it is deleted, since a test disabled indefinitely gives a false sense of coverage.
And blind retries are a trap. Many tools offer retry: 3. It is tempting and it is a mistake, for three reasons: it hides real concurrency bugs that will also happen in production, with real customers; it masks the deterioration, because a test failing 2 times out of 3 still passes the pipeline; and it punishes time, since retries multiply the duration of the worst cases. If you still need them as a temporary measure, do it with a metric: record how many tests needed a retry and treat that number as debt to be reduced.
- Parallelisation and
matrix
matrixThere are two ways of speeding things up, and they are not the same. Parallelism inside the job is done by Vitest by default, running files in several processes: it is free for unit tests, but for integration tests it requires each process to have its own data space, or you will be back to the previous section's problem.
Splitting across jobs (sharding) divides the suite into chunks that run on different runners. Each job pays its own start-up cost (checkout + npm ci), so it only pays off when the suite takes several minutes:
test-unit:
runs-on: ubuntu-22.04
strategy:
fail-fast: false # one red shard must not cancel the others
matrix:
shard: [1, 2, 3] # → three jobs in parallel
steps:
# ... checkout, setup-node and npm ci ...
- run: npx vitest run tests/unit --shard=${{ matrix.shard }}/3matrix for several Node versions. The same mechanism serves to test on several environments at once: all you need is matrix: { node: ['20.11.0', '22.4.0'] } and to pass node-version: ${{ matrix.node }} to setup-node. Reservalia pins Node 20.11.0, so it does not need this today; a library declaring support for several versions would. Beware of combinatorial growth: 3 versions × 3 operating systems is 9 jobs, and 8 of them will tell you nothing new.
- Reservalia's
test job and what blocks the merge
test job and what blocks the mergeWe replace the provisional test job from 02-02 with its definitive version, ordered to give fast feedback:
test:
name: Tests
runs-on: ubuntu-22.04
timeout-minutes: 15
services:
postgres: # the same block as in lesson 02-02,
image: postgres:16.3 # with its pg_isready healthcheck
env: { POSTGRES_USER: reservalia, POSTGRES_PASSWORD: ci, POSTGRES_DB: reservalia_test }
ports: ['5432:5432']
options: >-
--health-cmd "pg_isready -U reservalia -d reservalia_test"
--health-interval 5s --health-timeout 3s --health-retries 10
env:
TZ: Europe/Madrid
DATABASE_URL: postgres://reservalia:ci@localhost:5432/reservalia_test
steps:
# ... checkout, setup-node and npm ci ...
- name: Unit tests # ~40 s · stops early if something obvious fails
run: npm run test:unit --workspaces --if-present
- name: Migrations on the test database
run: npm run migrate --workspace apps/api
- name: Integration tests # ~2 min
run: npm run test:integration --workspace apps/api
- name: Coverage
if: always()
run: npm run test --workspace apps/api -- --coverageThe order unit → migrations → integration is not accidental: cheap things first. If a unit test fails, the job stops in 40 seconds instead of three minutes. What blocks the merge at Reservalia: the full test check (unit, migrations and integration), the quality check from the next lesson and the build check. What does not block it: coverage below the threshold — it is reported in the summary and commented on in review — and the full nightly E2E suite. The technical configuration of those required checks is lesson 02-07.
Common Mistakes and Tips
Mistake 1: tests that depend on the system clock. new Date() inside the logic makes it impossible to test on a Monday something that only happens on a Saturday. Pass the date in as a parameter: as well as making the test stable, it improves the design. Mistake 2: not isolating state between integration tests; without TRUNCATE in beforeEach, tests contaminate each other and the order decides the result, which is the number one factory of flakiness.
Mistake 3: turning coverage into a target. The predictable result is tests with no assertions that raise the number while verifying nothing. Mistake 4: putting the full E2E suite in every PR, the fastest way to reach 40 minutes and lose the team's trust. Mistake 5: normalising retries, because retry: 3 hides concurrency problems that will indeed show up in production, where there are no retries.
Tip 1: every bug fix starts with a test that reproduces it. It is the best source of useful tests there is, because it covers exactly what the system has already proved it cannot do. Tip 2: when a test fails, read it before you read the code — the fault is often in the test — and measure the time your tests take: vitest --reporter=verbose will tell you the five slowest, and fixing those five usually halves the total time.
Exercises
Exercise 1
Classify each test (unit, integration, contract or E2E) and say whether it should block a PR:
- That
calculateSlotsrespects a midday break. - That a customer can book from the public site and receives the email.
- That the database constraint prevents two overlapping appointments.
- That the JSON from
GET /appointments/:idmatches theAppointmenttype fromshared-types. - That the API responds in under 200 ms at 500 requests per second.
Exercise 2
This test fails in CI roughly one time in five, always at night. Identify two problems and rewrite it.
it('creates the appointment for tomorrow', async () => {
const tomorrow = new Date(Date.now() + 24 * 60 * 60 * 1000);
await createAppointment(pool, { businessId: 1, start: tomorrow.toISOString(), durationMin: 30 });
const { rows } = await pool.query('SELECT * FROM appointments');
expect(rows).toHaveLength(1);
});Exercise 3
The team proposes: "let us raise the coverage threshold to 95% and that way we guarantee quality". Give three technical arguments against it and one concrete alternative.
Solutions
Solution 1. (1) Unit, blocks: milliseconds, pure logic. (2) E2E, blocks only if it is one of the 2-3 critical journeys — and it is: booking an appointment is the flow that generates revenue; the rest of the E2E tests run overnight. (3) Integration, blocks: only the database can guarantee it. (4) Contract, blocks: it is cheap and it stops you breaking apps/web. (5) Performance, does not block: slow and noisy on a shared runner, and it is the subject of 04-04.
Solution 2. The two problems: (a) it uses Date.now(), so at 23:30 Madrid time "tomorrow" falls on a different day in UTC and the calculation shifts — hence the "always at night"; (b) it does not isolate state, because SELECT * FROM appointments counts all the rows, including those left behind by other tests, and toHaveLength(1) ends up depending on execution order.
beforeEach(async () => pool.query('TRUNCATE appointments RESTART IDENTITY CASCADE'));
it('creates the appointment for the given day', async () => {
const start = '2026-03-02T10:00:00+01:00'; // a fixed, explicit date
const appointment = await createAppointment(pool, { businessId: 1, start, durationMin: 30 });
const { rows } = await pool.query('SELECT * FROM appointments WHERE id = $1', [appointment.id]);
expect(rows).toHaveLength(1); // query scoped to this appointment
});And in the pipeline, TZ: Europe/Madrid removes the entire class of time-related problems.
Solution 3. Three arguments: (1) coverage measures execution, not verification, so 95% can be reached with tests that have not a single assertion; (2) the last stretch, from 80% to 95%, usually consists of error handlers and defensive branches whose testing cost is high and whose value is low, and that time is not spent testing the business logic properly; (3) an unreachable threshold ends up switched off or dodged with exclusions, which loses even the signal that was working. Alternative: set the global threshold at the current value (so it cannot drop) and apply a quality gate on new code — for example, 80% coverage on the lines the PR adds or modifies — together with the requirement that every bug fix comes with the test that reproduces it.
Conclusion
Reservalia now knows whether its code works, and it knows fast:
- The test pyramid orders the effort: thousands of unit tests in milliseconds, hundreds of integration tests against a real PostgreSQL, a few dozen contract tests and very few E2E ones. And not everything goes in every PR: unit, integration, contract and two or three critical E2E tests block the merge; the full E2E suite, performance and security run separately, on the principle that the blocking set fits in 10 minutes.
- The unit tests of
calculateSlotstouch neither clock nor database, and one of them reproduces the real midday break bug. The integration ones use theDATABASE_URLfrom the environment, isolate themselves withTRUNCATEinbeforeEachand verify the one thing the code cannot guarantee on its own: the database's overlap constraint. - Coverage is generated with
--coverage, published to$GITHUB_STEP_SUMMARYand interpreted as a signal: a realistic threshold, attention to new code and never a team target. - Flaky tests have identifiable causes — time, order, concurrency, fixed waits, external resources — they are detected by repeating the run and they are managed with a quarantine that has an owner and a deadline. Retrying blindly hides problems that will indeed happen in production.
- Parallelisation by shards and the version
matrixspeed things up, but every job pays its own start-up cost. And the definitivetestjob orders the work from cheap to expensive — unit, migrations, integration, coverage — on top of the PostgreSQL 16.3services:block and withTZ: Europe/Madrid.
There is a whole family of problems left that no test detects: code that works but is inconsistent, unreadable or needlessly complicated. In the next lesson, Code Quality and Static Analysis, we will look at why that belongs in the pipeline rather than in human review, at exactly how Prettier, ESLint and tsc --noEmit differ, with examples of what each one catches that the others do not, at what a quality gate is and why the criterion should be "clean new code" instead of paying off all the debt at once. And we will add the quality job to ci.yml.
CI/CD Course: Continuous Integration and Deployment
Module 1: Introduction to CI/CD
- Basic CI/CD Concepts
- Benefits of CI/CD
- Popular CI/CD Tools
- The Course Project: the Application We Are Going to Automate
- DORA Metrics: How Software Delivery Is Measured
Module 2: Continuous Integration (CI)
- Introduction to Continuous Integration
- Setting Up a CI Environment
- Build Automation
- Automated Testing
- Code Quality and Static Analysis
- Artifacts, Versioning and Promotion
- Integration with Version Control
Module 3: Continuous Deployment (CD)
- Introduction to Continuous Deployment
- Deployment Automation
- Infrastructure as Code and Reproducible Environments
- Deployment Strategies
- Feature Flags, Rollback and Failure Recovery
- Monitoring and Feedback
Module 4: Advanced CI/CD Practices
- CI/CD Pipelines
- Dependency Management
- Security in CI/CD
- Scalability and Performance
- Pipeline as Code: Templates, Reuse and Testing the Pipeline
- Databases in the Pipeline: Safe Migrations
Module 5: Implementing CI/CD in Real Projects
- Case Study: Web Project
- Case Study: Mobile Application
- Case Study: Microservices
- Case Study: Modernising a Legacy Project
Module 6: Tools and Technologies
- Jenkins
- GitLab CI/CD
- CircleCI
- Travis CI
- Docker and Kubernetes
- GitHub Actions in Depth
- Comparison and Criteria for Choosing a Tool
Module 7: Practical Exercises
- Exercise 1: Setting Up a Basic Pipeline
- Exercise 2: Integrating Automated Tests
- Exercise 3: Deploying to a Production Environment
- Exercise 4: Monitoring and Feedback
- Exercise 5: Hardening the Pipeline with Security and Secrets
- Final Project: A Complete End-to-End Pipeline
