The Project | About Us | Contribute | Donations | License
HOME My courses Completed
◄ Previous
◄

Exercise 2: Integrating Automated Tests

Next ►
►

The pipeline from 07-01 is green, but its signal is weak: seven unit tests over a pure function say absolutely nothing about whether the /api/slots endpoint responds correctly, whether the server starts, or whether persistence stores what it claims to store. A green pipeline with insufficient tests looks exactly the same as one with good tests, and that is its peculiar danger. In this lab you are going to turn that weak signal into a strong one: you will extend Mini-Reservalia with a real persistence layer, write the three layers of the pyramid on top of it, measure coverage and publish it in the run summary, put a threshold in place that breaks the build, run everything across a matrix of Node versions and in two parallel shards to watch the time drop, and —the most instructive part of the lesson— manufacture a flaky test on purpose to watch it fail intermittently and apply the quarantine policy from 02-04 to it.

None of these pieces is optional in a real project. The three layers tell you what is broken and at which level; coverage tells you where you are not looking; the matrix protects you from "it works on my version"; sharding is what keeps the suite tolerable when it goes from 7 tests to 700; and the flaky policy is the only thing that stops the team learning to ignore red.

Contents

  1. Objective, prerequisites and starting point
  2. The persistence layer: interface, memory and SQLite
  3. The server on top of the repository
  4. Layer 1: unit tests with genuine edge cases
  5. Layer 2: integration tests against real persistence
  6. Layer 3: the end-to-end test against the running process
  7. Coverage: measuring it, publishing it and putting a threshold on it
  8. A matrix of Node versions
  9. Sharding: splitting the suite in two
  10. The complete ci.yml
  11. Manufacturing a flaky test and putting it in quarantine
  12. Reports as artifacts and annotations in the PR
  13. Final verification
  14. Common Mistakes and Tips
  15. Exercises
  16. Conclusion

  1. Objective, prerequisites and starting point

Objective. By the end you will have a three-layer suite over Mini-Reservalia running in four parallel jobs (2 Node versions × 2 shards), with coverage measured, published and gated by a threshold that breaks the build, plus a quarantine policy applied to a real flaky test.

Prerequisites. Having completed 07-01: the mini-reservalia repository on GitHub, with a three-job ci.yml, cache, protected main and the badge in the README.

Starting point.

mini-reservalia/
├── .github/workflows/ci.yml
├── src/availability.js
├── src/server.js
├── scripts/build.js
├── test/availability.test.js
├── eslint.config.js
├── package.json
└── package-lock.json

Work on a branch from the start, because main is protected:

git checkout main && git pull
git checkout -b automated-testing

  1. The persistence layer: interface, memory and SQLite

Until now the diary was a Map inside server.js itself. That makes persistence impossible to test and conflates two responsibilities. We are going to extract a repository with two interchangeable implementations: one in memory (fast, for the unit tests) and one over SQLite (real, for the integration ones).

Why SQLite and not PostgreSQL as the main path: SQLite is a file, it needs no service, it starts in microseconds and it behaves the same on your laptop and on the runner. The PostgreSQL variant —which is what the real Reservalia uses— goes in a note at the end of the section.

npm install better-sqlite3

It is the project's first production dependency. From now on npm ci installs a native module, which will make the cache from 07-01 considerably more worthwhile.

Note. Node 22.5+ ships node:sqlite out of the box (still experimental). If your project is only ever going to run on Node 22+, you can avoid the dependency. Here we use better-sqlite3 because our matrix includes Node 20 and because a native module is a more realistic case for talking about caches and dependency auditing in 07-05.

2.1 src/repository.js — the interface and the in-memory implementation

// src/repository.js
// Mini-Reservalia persistence contract + in-memory implementation.
//
// Every implementation exposes the same interface:
//   listAppointments(date) -> {start, end, customer}[]  sorted by start
//   createAppointment({date, start, end, customer}) -> created appointment (with id)
//   deleteAll() -> void
//   close() -> void

const DATE_PATTERN = /^\d{4}-\d{2}-\d{2}$/;
const TIME_PATTERN = /^([01]\d|2[0-3]):([0-5]\d)$/;

/** Validates and normalises an appointment before storing it. Throws if invalid. */
export function validateAppointment(appointment) {
  if (!DATE_PATTERN.test(appointment?.date ?? '')) {
    throw new TypeError('invalid date: YYYY-MM-DD expected');
  }
  if (!TIME_PATTERN.test(appointment?.start ?? '') || !TIME_PATTERN.test(appointment?.end ?? '')) {
    throw new TypeError('start and end must be in HH:MM format');
  }
  if (appointment.end <= appointment.start) {
    throw new RangeError('the end must come after the start');
  }
  return {
    date: appointment.date,
    start: appointment.start,
    end: appointment.end,
    customer: String(appointment.customer ?? 'anonymous').slice(0, 80),
  };
}

export class MemoryRepository {
  #byDate = new Map();
  #nextId = 1;

  listAppointments(date) {
    const appointments = this.#byDate.get(date) ?? [];
    return [...appointments].sort((a, b) => a.start.localeCompare(b.start));
  }

  createAppointment(data) {
    const appointment = { id: this.#nextId++, ...validateAppointment(data) };
    const list = this.#byDate.get(appointment.date) ?? [];
    list.push(appointment);
    this.#byDate.set(appointment.date, list);
    return appointment;
  }

  deleteAll() {
    this.#byDate.clear();
    this.#nextId = 1;
  }

  close() {
    /* nothing to close */
  }
}

2.2 src/repository-sqlite.js

// src/repository-sqlite.js
// Implementation of the same contract over SQLite.

import Database from 'better-sqlite3';
import { validateAppointment } from './repository.js';

const SCHEMA = `
  CREATE TABLE IF NOT EXISTS appointments (
    id       INTEGER PRIMARY KEY AUTOINCREMENT,
    date     TEXT NOT NULL,
    start    TEXT NOT NULL,
    end      TEXT NOT NULL,
    customer TEXT NOT NULL DEFAULT 'anonymous'
  );
  CREATE INDEX IF NOT EXISTS idx_appointments_date ON appointments(date);
`;

export class SqliteRepository {
  #db;
  #stmtList;
  #stmtInsert;

  /** @param {string} path ':memory:' for an ephemeral database, or a file. */
  constructor(path = ':memory:') {
    this.#db = new Database(path);
    this.#db.pragma('journal_mode = WAL');
    this.#db.exec(SCHEMA); // minimal migration; 04-06 explains why this is versioned for real
    this.#stmtList = this.#db.prepare(
      'SELECT id, date, start, end, customer FROM appointments WHERE date = ? ORDER BY start',
    );
    this.#stmtInsert = this.#db.prepare(
      'INSERT INTO appointments (date, start, end, customer) VALUES (@date, @start, @end, @customer)',
    );
  }

  listAppointments(date) {
    return this.#stmtList.all(date);
  }

  createAppointment(data) {
    const appointment = validateAppointment(data);
    const info = this.#stmtInsert.run(appointment);
    return { id: Number(info.lastInsertRowid), ...appointment };
  }

  deleteAll() {
    this.#db.exec('DELETE FROM appointments');
  }

  close() {
    this.#db.close();
  }
}

/**
 * Repository factory based on the connection URL.
 * This is what lets the same binary run with memory in the fast tests and with
 * SQLite in production, without a single "if we are in test" branch. The
 * configuration comes from the environment (12-factor), as in 03-02.
 */
export async function createRepository(url = process.env.DATABASE_URL ?? 'memory:') {
  if (url === 'memory:') {
    const { MemoryRepository } = await import('./repository.js');
    return new MemoryRepository();
  }
  if (url.startsWith('sqlite:')) {
    return new SqliteRepository(url.slice('sqlite:'.length));
  }
  throw new Error(`Unsupported data source: ${url}`);
}

Two decisions that apply to any project:

  • Validation lives in a single place (validateAppointment), shared by both implementations. If it were duplicated, the two implementations would behave differently with bad data and the integration tests would pass while production failed.
  • prepare outside the methods. Prepared statements avoid SQL concatenation. This is not just about performance: it is what makes SQL injection impossible by construction. In 07-05 we will introduce a badly written query on purpose to watch CodeQL detect it.

The real equivalent in Reservalia, and the PostgreSQL variant. Reservalia uses PostgreSQL on RDS. In CI, the ci.yml from 02-02 brings up a PostgreSQL as a runner service:

  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16-alpine
        env:
          POSTGRES_PASSWORD: test
          POSTGRES_DB: reservalia_test
        ports: ['5432:5432']
        # Without this health check, the steps start before
        # Postgres accepts connections: the classic intermittent failure.
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 5s --health-timeout 5s --health-retries 10
    env:
      DATABASE_URL: postgres://postgres:test@localhost:5432/reservalia_test
    steps:
      - uses: actions/checkout@v4
      # ... npm ci, migrations, npm test

If you want to do it that way, write a PostgresRepository with the same interface and everything else in this lesson works unchanged. The cost is +20-30 s per job and one more service that can fail; that is why the main path here is SQLite.

  1. The server on top of the repository

Replace the in-memory diary in src/server.js with the injected repository, and add POST /api/appointments so that data can be created from the integration tests.

// src/server.js  (07-02 version)
import http from 'node:http';
import { fileURLToPath } from 'node:url';
import { calculateSlots } from './availability.js';
import { createRepository } from './repository-sqlite.js';

export const VERSION = process.env.APP_VERSION ?? 'dev';
export const PORT = Number(process.env.PORT ?? 3000);

export const DEFAULT_OPENING_HOURS = [
  { start: '09:00', end: '14:00' },
  { start: '16:00', end: '20:00' },
];

const DATE_PATTERN = /^\d{4}-\d{2}-\d{2}$/;

function respondJson(res, code, body) {
  const text = JSON.stringify(body);
  res.writeHead(code, {
    'content-type': 'application/json; charset=utf-8',
    'content-length': Buffer.byteLength(text),
  });
  res.end(text);
}

async function readBody(req, maxBytes = 8192) {
  const chunks = [];
  let total = 0;
  for await (const chunk of req) {
    total += chunk.length;
    if (total > maxBytes) throw new RangeError('body too large');
    chunks.push(chunk);
  }
  if (total === 0) return {};
  return JSON.parse(Buffer.concat(chunks).toString('utf8'));
}

export function createServer({ repository, openingHours = DEFAULT_OPENING_HOURS } = {}) {
  if (!repository) throw new Error('createServer requires a repository');

  return http.createServer(async (req, res) => {
    const url = new URL(req.url, `http://${req.headers.host ?? 'localhost'}`);

    try {
      if (req.method === 'GET' && url.pathname === '/health') {
        return respondJson(res, 200, {
          status: 'ok',
          version: VERSION,
          uptimeSec: Math.round(process.uptime()),
        });
      }

      if (req.method === 'GET' && url.pathname === '/api/slots') {
        const date = url.searchParams.get('date');
        const duration = Number(url.searchParams.get('duration') ?? 30);
        if (!date || !DATE_PATTERN.test(date)) {
          return respondJson(res, 400, { error: 'The "date" parameter is required (YYYY-MM-DD)' });
        }
        if (!Number.isInteger(duration) || duration <= 0) {
          return respondJson(res, 400, { error: 'Invalid "duration" parameter' });
        }
        const appointments = repository.listAppointments(date);
        const slots = calculateSlots(openingHours, appointments, duration);
        return respondJson(res, 200, { date, duration, total: slots.length, slots });
      }

      if (req.method === 'POST' && url.pathname === '/api/appointments') {
        const body = await readBody(req);
        const appointment = repository.createAppointment(body);
        return respondJson(res, 201, appointment);
      }

      return respondJson(res, 404, { error: 'Route not found' });
    } catch (error) {
      // Validation errors -> 400; everything else -> 500. Without leaking the stack.
      const isValidation = error instanceof TypeError || error instanceof RangeError || error instanceof SyntaxError;
      if (!isValidation) console.error('Unhandled error:', error);
      return respondJson(res, isValidation ? 400 : 500, {
        error: isValidation ? error.message : 'Internal error',
      });
    }
  });
}

if (process.argv[1] === fileURLToPath(import.meta.url)) {
  const repository = await createRepository();
  createServer({ repository }).listen(PORT, () => {
    console.log(`Mini-Reservalia ${VERSION} listening on http://localhost:${PORT}`);
  });
}

Try it locally before moving on:

DATABASE_URL='sqlite:/tmp/mini.db' npm start &
curl -s -X POST localhost:3000/api/appointments \
  -H 'content-type: application/json' \
  -d '{"date":"2026-03-02","start":"10:00","end":"10:30","customer":"Ana"}'
# {"id":1,"date":"2026-03-02","start":"10:00","end":"10:30","customer":"Ana"}

curl -s "localhost:3000/api/slots?date=2026-03-02&duration=30" | head -c 200
# {"date":"2026-03-02","duration":30,"total":17,"slots":[{"start":"09:00",...
kill %1

  1. Layer 1: unit tests with genuine edge cases

The pyramid from 02-04, applied to this project:

Layer What it tests How many Speed File
Unit calculateSlots, validateAppointment — pure logic, no I/O Many µs test/availability.test.js
Integration The real SQLite repository; HTTP routes against that repository Some ms test/repository.test.js, test/api.test.js
End-to-end The real process running, over HTTP, no tricks Very few s test/e2e.test.js

Extend test/availability.test.js with the edge cases that really hurt in production:

// test/availability.test.js  (07-02 extension)
import test, { describe } from 'node:test';
import assert from 'node:assert/strict';
import { calculateSlots, toMinutes, toTime } from '../src/availability.js';

const MORNING = { start: '09:00', end: '12:00' };
const SPLIT = [
  { start: '09:00', end: '14:00' },
  { start: '16:00', end: '20:00' },
];

describe('time conversions', () => {
  test('toMinutes converts valid times', () => {
    assert.equal(toMinutes('00:00'), 0);
    assert.equal(toMinutes('09:30'), 570);
    assert.equal(toMinutes('23:59'), 1439);
  });

  test('toMinutes rejects invalid formats', () => {
    for (const bad of ['9:00', '25:00', '09:60', '', '0900', 900, null]) {
      assert.throws(() => toMinutes(bad), TypeError, `should reject ${JSON.stringify(bad)}`);
    }
  });

  test('toTime is the inverse of toMinutes', () => {
    for (const time of ['00:00', '07:05', '13:45', '23:59']) {
      assert.equal(toTime(toMinutes(time)), time);
    }
  });
});

describe('calculateSlots: base cases', () => {
  test('a day with no appointments is sliced end to end', () => {
    assert.deepEqual(calculateSlots(MORNING, [], 60), [
      { start: '09:00', end: '10:00' },
      { start: '10:00', end: '11:00' },
      { start: '11:00', end: '12:00' },
    ]);
  });

  test('an appointment splits the day into two blocks', () => {
    assert.deepEqual(calculateSlots(MORNING, [{ start: '10:00', end: '11:00' }], 60), [
      { start: '09:00', end: '10:00' },
      { start: '11:00', end: '12:00' },
    ]);
  });

  test('the leftover remainder does not produce a short slot', () => {
    const slots = calculateSlots(MORNING, [], 50);
    assert.equal(slots.length, 3);
    assert.equal(slots.at(-1).end, '11:30');
  });
});

describe('calculateSlots: edge cases', () => {
  test('two OVERLAPPING appointments merge and leave no phantom slot', () => {
    // 10:00-11:00 and 10:30-11:30 -> busy 10:00-11:30, no slot between them.
    const slots = calculateSlots({ start: '09:00', end: '13:00' }, [
      { start: '10:00', end: '11:00' },
      { start: '10:30', end: '11:30' },
    ], 30);
    assert.deepEqual(slots.map((s) => s.start), ['09:00', '09:30', '11:30', '12:00', '12:30']);
  });

  test('two CONSECUTIVE appointments that touch leave no zero-length slot', () => {
    const slots = calculateSlots({ start: '09:00', end: '12:00' }, [
      { start: '10:00', end: '10:30' },
      { start: '10:30', end: '11:00' },
    ], 30);
    assert.deepEqual(slots.map((s) => s.start), ['09:00', '09:30', '11:00', '11:30']);
  });

  test('UNSORTED appointments give the same result as sorted ones', () => {
    const unsorted = [{ start: '11:00', end: '11:30' }, { start: '09:30', end: '10:00' }];
    const sorted = [...unsorted].sort((a, b) => a.start.localeCompare(b.start));
    assert.deepEqual(calculateSlots(MORNING, unsorted, 30), calculateSlots(MORNING, sorted, 30));
  });

  test('an appointment CROSSING CLOSING TIME trims the block without overflowing it', () => {
    // Closing at 14:00, appointment 13:45-14:30. No slot may go past 13:45.
    const slots = calculateSlots({ start: '09:00', end: '14:00' }, [{ start: '13:45', end: '14:30' }], 30);
    assert.ok(slots.every((s) => s.end <= '13:45'), `slot outside opening hours: ${JSON.stringify(slots.at(-1))}`);
    assert.equal(slots.at(-1).end, '13:30');
  });

  test('an appointment BEFORE OPENING has no effect', () => {
    const slots = calculateSlots(MORNING, [{ start: '07:00', end: '08:00' }], 60);
    assert.equal(slots.length, 3);
  });

  test('an appointment COVERING the whole block leaves the day with no slots', () => {
    assert.deepEqual(calculateSlots(MORNING, [{ start: '08:00', end: '15:00' }], 30), []);
  });

  test('SPLIT OPENING HOURS: no slot crosses the lunch break', () => {
    const slots = calculateSlots(SPLIT, [], 60);
    assert.ok(!slots.some((s) => s.start < '14:00' && s.end > '14:00'), 'there is a slot crossing the break');
    assert.equal(slots.length, 5 + 4);
  });

  test('SPLIT OPENING HOURS with one appointment in each block', () => {
    const slots = calculateSlots(SPLIT, [
      { start: '10:00', end: '11:00' },
      { start: '17:00', end: '18:00' },
    ], 60);
    assert.deepEqual(slots.map((s) => s.start), ['09:00', '11:00', '12:00', '13:00', '16:00', '18:00', '19:00']);
  });

  test('a duration longer than the block produces no slots', () => {
    assert.deepEqual(calculateSlots(MORNING, [], 240), []);
  });

  test('invalid parameters throw, they do not return empty', () => {
    assert.throws(() => calculateSlots(MORNING, [], 0), RangeError);
    assert.throws(() => calculateSlots(MORNING, [], 12.5), RangeError);
    assert.throws(() => calculateSlots({ start: '14:00', end: '09:00' }, [], 30), RangeError);
  });
});

Notice the pattern in the closing-time test: assert.ok(slots.every(...)) with a message that includes the value that failed. An assert.ok(condition) with no message produces AssertionError: The expression evaluated to a falsy value, which tells you nothing; with a message, the pipeline log gives you the diagnosis without having to reproduce it locally. That is a difference of two minutes of writing and twenty of debugging.

npm test
# ℹ tests 17 / pass 17 / fail 0

  1. Layer 2: integration tests against real persistence

Two files: one for the repository and one for the HTTP routes.

5.1 test/repository.test.js

The interesting thing here is that the same battery runs against both implementations. That is a contract test: it guarantees that memory and SQLite are interchangeable, which is exactly what we assume when injecting one or the other.

// test/repository.test.js
import test, { describe, beforeEach, after } from 'node:test';
import assert from 'node:assert/strict';
import { MemoryRepository } from '../src/repository.js';
import { SqliteRepository } from '../src/repository-sqlite.js';

const IMPLEMENTATIONS = [
  ['memory', () => new MemoryRepository()],
  ['sqlite', () => new SqliteRepository(':memory:')],
];

for (const [name, factory] of IMPLEMENTATIONS) {
  describe(`repository contract: ${name}`, () => {
    let repo;

    beforeEach(() => {
      repo?.close();
      repo = factory();
    });

    after(() => repo?.close());

    test('a day with no appointments returns an empty list', () => {
      assert.deepEqual(repo.listAppointments('2026-03-02'), []);
    });

    test('createAppointment returns the appointment with a numeric id', () => {
      const appointment = repo.createAppointment({ date: '2026-03-02', start: '10:00', end: '10:30', customer: 'Ana' });
      assert.equal(typeof appointment.id, 'number');
      assert.equal(appointment.customer, 'Ana');
    });

    test('appointments are returned SORTED by start time', () => {
      repo.createAppointment({ date: '2026-03-02', start: '17:00', end: '18:00', customer: 'C' });
      repo.createAppointment({ date: '2026-03-02', start: '09:00', end: '09:30', customer: 'A' });
      repo.createAppointment({ date: '2026-03-02', start: '12:00', end: '12:30', customer: 'B' });
      assert.deepEqual(repo.listAppointments('2026-03-02').map((a) => a.customer), ['A', 'B', 'C']);
    });

    test('the appointments of one day do NOT mix with those of another', () => {
      repo.createAppointment({ date: '2026-03-02', start: '10:00', end: '10:30' });
      repo.createAppointment({ date: '2026-03-03', start: '11:00', end: '11:30' });
      assert.equal(repo.listAppointments('2026-03-02').length, 1);
      assert.equal(repo.listAppointments('2026-03-03').length, 1);
    });

    test('the default customer is "anonymous"', () => {
      const appointment = repo.createAppointment({ date: '2026-03-02', start: '10:00', end: '10:30' });
      assert.equal(appointment.customer, 'anonymous');
    });

    test('rejects invalid data in BOTH implementations', () => {
      assert.throws(() => repo.createAppointment({ date: '2/3/2026', start: '10:00', end: '10:30' }), TypeError);
      assert.throws(() => repo.createAppointment({ date: '2026-03-02', start: '10:00', end: '09:00' }), RangeError);
      assert.throws(() => repo.createAppointment({ date: '2026-03-02', start: '10', end: '11' }), TypeError);
    });

    test('deleteAll leaves the repository clean', () => {
      repo.createAppointment({ date: '2026-03-02', start: '10:00', end: '10:30' });
      repo.deleteAll();
      assert.deepEqual(repo.listAppointments('2026-03-02'), []);
    });
  });
}

5.2 test/api.test.js — the routes against real persistence

// test/api.test.js
// Integration: real HTTP server + real SQLite repository, on an ephemeral port.
import test, { describe, before, after, beforeEach } from 'node:test';
import assert from 'node:assert/strict';
import { createServer } from '../src/server.js';
import { SqliteRepository } from '../src/repository-sqlite.js';

let server;
let repository;
let base;

before(async () => {
  repository = new SqliteRepository(':memory:');
  server = createServer({ repository });
  // Port 0 = the system assigns a free one. Never pin a port in the
  // tests: two parallel jobs on the same runner would clash (EADDRINUSE).
  await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve));
  base = `http://127.0.0.1:${server.address().port}`;
});

after(async () => {
  await new Promise((resolve) => server.close(resolve));
  repository.close();
});

beforeEach(() => repository.deleteAll()); // isolation between tests

describe('GET /health', () => {
  test('responds 200 with status ok and a version', async () => {
    const response = await fetch(`${base}/health`);
    assert.equal(response.status, 200);
    const body = await response.json();
    assert.equal(body.status, 'ok');
    assert.ok(typeof body.version === 'string');
    assert.ok(Number.isFinite(body.uptimeSec));
  });
});

describe('GET /api/slots', () => {
  test('with no appointments it returns the whole day (9 slots of 60 min)', async () => {
    const body = await (await fetch(`${base}/api/slots?date=2026-03-02&duration=60`)).json();
    assert.equal(body.total, 9); // 5 in the morning + 4 in the afternoon
  });

  test('reflects an appointment created through the API', async () => {
    await fetch(`${base}/api/appointments`, {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify({ date: '2026-03-02', start: '10:00', end: '11:00', customer: 'Ana' }),
    });
    const body = await (await fetch(`${base}/api/slots?date=2026-03-02&duration=60`)).json();
    assert.equal(body.total, 8);
    assert.ok(!body.slots.some((s) => s.start === '10:00'), 'the booked slot still shows up');
  });

  test('with no date it responds 400 with a useful message', async () => {
    const response = await fetch(`${base}/api/slots`);
    assert.equal(response.status, 400);
    assert.match((await response.json()).error, /date/i);
  });

  test('with a malformed date it responds 400', async () => {
    assert.equal((await fetch(`${base}/api/slots?date=02-03-2026`)).status, 400);
  });

  test('with an invalid duration it responds 400', async () => {
    assert.equal((await fetch(`${base}/api/slots?date=2026-03-02&duration=-5`)).status, 400);
    assert.equal((await fetch(`${base}/api/slots?date=2026-03-02&duration=abc`)).status, 400);
  });
});

describe('POST /api/appointments', () => {
  test('creates the appointment and responds 201 with the id', async () => {
    const response = await fetch(`${base}/api/appointments`, {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify({ date: '2026-03-02', start: '10:00', end: '10:30', customer: 'Ana' }),
    });
    assert.equal(response.status, 201);
    assert.ok((await response.json()).id > 0);
  });

  test('rejects an invalid appointment with 400 and does NOT persist it', async () => {
    const response = await fetch(`${base}/api/appointments`, {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify({ date: '2026-03-02', start: '11:00', end: '10:00' }),
    });
    assert.equal(response.status, 400);
    assert.deepEqual(repository.listAppointments('2026-03-02'), []);
  });

  test('malformed JSON responds 400, not 500', async () => {
    const response = await fetch(`${base}/api/appointments`, {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: '{this is not json',
    });
    assert.equal(response.status, 400);
  });
});

describe('unknown routes', () => {
  test('respond 404', async () => {
    assert.equal((await fetch(`${base}/does-not-exist`)).status, 404);
  });
});

Three rules these tests embody that you can take to any project:

  1. Port 0. Never pin a port. Two parallel shards on the same runner with port 3000 pinned step on each other and produce an intermittent EADDRINUSE: you have just created a flaky test by accident.
  2. A beforeEach that cleans up. Isolation between tests is not optional. Without it, execution order matters, and the order changes when you shard.
  3. The negative test checks the side effect. rejects an invalid appointment ... and does NOT persist it does not only look at the status code: it looks at whether anything was written. A 400 that also stores the row is a bug a lazy test does not catch.

  1. Layer 3: the end-to-end test against the running process

The integration tests import the server as a module. That leaves out everything that happens when the process really starts: the environment variables, createRepository, the argv[1] guard, the listen. An end-to-end test launches the binary exactly as production will launch it.

// test/e2e.test.js
// End-to-end: starts the REAL process with `node src/server.js`, with its
// environment-based configuration, and talks to it over HTTP like a client would.
import test, { describe, before, after } from 'node:test';
import assert from 'node:assert/strict';
import { spawn } from 'node:child_process';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';

const PORT = 3100 + Number(process.env.PORT_OFFSET ?? 0);
const BASE = `http://127.0.0.1:${PORT}`;

let child;
let directory;

/** Polls until /health responds 200, with a time limit. */
async function waitForHealth(attempts = 40, waitMs = 250) {
  for (let i = 1; i <= attempts; i++) {
    try {
      const response = await fetch(`${BASE}/health`);
      if (response.ok) return;
    } catch {
      /* not listening yet: retry */
    }
    await new Promise((r) => setTimeout(r, waitMs));
  }
  throw new Error(`The server did not respond within ${(attempts * waitMs) / 1000}s`);
}

before(async () => {
  directory = await mkdtemp(join(tmpdir(), 'mini-reservalia-e2e-'));
  child = spawn(process.execPath, ['src/server.js'], {
    env: {
      ...process.env,
      PORT: String(PORT),
      DATABASE_URL: `sqlite:${join(directory, 'e2e.db')}`,
      APP_VERSION: 'e2e-test',
    },
    stdio: ['ignore', 'pipe', 'pipe'],
  });
  // Forward the child's output: without this, a startup failure is invisible.
  child.stdout.on('data', (d) => process.stdout.write(`[server] ${d}`));
  child.stderr.on('data', (d) => process.stderr.write(`[server:err] ${d}`));
  await waitForHealth();
});

after(async () => {
  child?.kill('SIGTERM');
  await rm(directory, { recursive: true, force: true });
});

describe('complete booking flow', () => {
  test('/health reports the version injected by the environment', async () => {
    const body = await (await fetch(`${BASE}/health`)).json();
    assert.equal(body.version, 'e2e-test');
  });

  test('booking reduces the available slots and persists between requests', async () => {
    const date = '2026-04-15';
    const slotsBefore = await (await fetch(`${BASE}/api/slots?date=${date}&duration=60`)).json();

    const created = await fetch(`${BASE}/api/appointments`, {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify({ date, start: '11:00', end: '12:00', customer: 'Diego' }),
    });
    assert.equal(created.status, 201);

    const slotsAfter = await (await fetch(`${BASE}/api/slots?date=${date}&duration=60`)).json();
    assert.equal(slotsAfter.total, slotsBefore.total - 1);
    assert.ok(!slotsAfter.slots.some((s) => s.start === '11:00'));
  });
});

With that, the suite now looks like this:

npm test
# ℹ tests 34
# ℹ pass 34
# ℹ fail 0
# ℹ duration_ms 1850

A note on Playwright. This E2E test exercises the API. An end-to-end test of the interface —click on the calendar, pick a slot, confirm— requires a real browser, and there the tool is Playwright. 05-01 covers it in the Reservalia context: npx playwright test in CI with --reporter=html, cached browsers and failure traces as artifacts. We do not repeat it here because it would triple the pipeline time without teaching anything new about CI.

  1. Coverage: measuring it, publishing it and putting a threshold on it

Node has coverage built in:

node --test --experimental-test-coverage test/

At the end of the report you will see a per-file table with lines, branches and functions. For the pipeline we need three more things: a machine-readable format, a human-readable summary and a threshold.

Add this to package.json:

"scripts": {
  "lint": "eslint .",
  "test": "node --test test/",
  "test:coverage": "node --test --experimental-test-coverage --test-reporter=lcov --test-reporter-destination=reports/lcov.info --test-reporter=spec --test-reporter-destination=stdout test/",
  "coverage:check": "node scripts/coverage.js",
  "build": "node scripts/build.js",
  "start": "node src/server.js"
}

And create scripts/coverage.js, which reads the LCOV, writes the summary in Markdown and fails if it drops below the threshold:

// scripts/coverage.js
// Reads reports/lcov.info, publishes a summary and applies the thresholds.
// No dependencies: the LCOV format is four labels.
//
//   SF:<file>      start of file
//   LF/LH          lines found / hit
//   BRF/BRH        branches found / hit
//   FNF/FNH        functions found / hit
//   end_of_record

import { readFile, appendFile } from 'node:fs/promises';

const LINES_THRESHOLD = Number(process.env.LINES_THRESHOLD ?? 85);
const BRANCHES_THRESHOLD = Number(process.env.BRANCHES_THRESHOLD ?? 75);

const content = await readFile('reports/lcov.info', 'utf8').catch(() => {
  console.error('reports/lcov.info does not exist. Run this first: npm run test:coverage');
  process.exit(2);
});

const files = [];
let current = null;
for (const line of content.split('\n')) {
  const [label, value] = line.split(':');
  if (label === 'SF') current = { file: value, LF: 0, LH: 0, BRF: 0, BRH: 0 };
  else if (current && ['LF', 'LH', 'BRF', 'BRH'].includes(label)) current[label] = Number(value);
  else if (label === 'end_of_record' && current) {
    files.push(current);
    current = null;
  }
}

const total = files.reduce(
  (acc, f) => ({ LF: acc.LF + f.LF, LH: acc.LH + f.LH, BRF: acc.BRF + f.BRF, BRH: acc.BRH + f.BRH }),
  { LF: 0, LH: 0, BRF: 0, BRH: 0 },
);

const pct = (part, whole) => (whole === 0 ? 100 : (part / whole) * 100);
const lines = pct(total.LH, total.LF);
const branches = pct(total.BRH, total.BRF);

const rows = files
  .filter((f) => f.file.includes('/src/'))
  .map((f) => `| \`${f.file.replace(process.cwd() + '/', '')}\` | ${pct(f.LH, f.LF).toFixed(1)} % | ${pct(f.BRH, f.BRF).toFixed(1)} % |`)
  .sort();

const mark = (value, threshold) => (value >= threshold ? '✅' : '❌');

const summary = [
  '## Coverage',
  '',
  `**Lines: ${lines.toFixed(1)} %** ${mark(lines, LINES_THRESHOLD)} (threshold ${LINES_THRESHOLD} %)  `,
  `**Branches: ${branches.toFixed(1)} %** ${mark(branches, BRANCHES_THRESHOLD)} (threshold ${BRANCHES_THRESHOLD} %)`,
  '',
  '| File | Lines | Branches |',
  '|---|---|---|',
  ...rows,
  '',
  '> Coverage measures which code RUNS, not which code is CHECKED.',
  '> 95 % with no asserts is 0 % of value. See lesson 02-04.',
].join('\n');

console.log(summary);
if (process.env.GITHUB_STEP_SUMMARY) {
  await appendFile(process.env.GITHUB_STEP_SUMMARY, `${summary}\n`);
}

if (lines < LINES_THRESHOLD || branches < BRANCHES_THRESHOLD) {
  console.error(`\nInsufficient coverage: lines ${lines.toFixed(1)}% (min ${LINES_THRESHOLD}%), branches ${branches.toFixed(1)}% (min ${BRANCHES_THRESHOLD}%)`);
  process.exit(1);
}
console.log('\nCoverage above the thresholds.');

Add reports/ to .gitignore. And try it:

mkdir -p reports
npm run test:coverage && npm run coverage:check

What you should see:

## Coverage

**Lines: 93.4 %** ✅ (threshold 85 %)
**Branches: 84.1 %** ✅ (threshold 75 %)

| File | Lines | Branches |
|---|---|---|
| `src/availability.js` | 100.0 % | 96.2 % |
| `src/repository-sqlite.js` | 88.9 % | 66.7 % |
| `src/repository.js` | 96.0 % | 90.0 % |
| `src/server.js` | 91.2 % | 80.6 % |

Coverage above the thresholds.

Checking that it fails when it should fail. Raise the threshold temporarily and verify that it breaks:

LINES_THRESHOLD=99 npm run coverage:check
# Insufficient coverage: lines 93.4% (min 99%), branches 84.1% (min 75%)
echo $?   # 1

The mandatory warning. Coverage measures which code runs, not which code is checked. You can reach 100 % with this test:

test('false coverage', () => {
  calculateSlots(MORNING, [{ start: '10:00', end: '11:00' }], 30);
  // ...and no assert at all. Runs everything. Verifies nothing.
});

That is why the threshold is used as a regression detector —"let us not drop below where we are"— and not as a target. A team given a 90 % coverage target produces, without exception, tests with no asserts. Set the threshold 2-3 points below the current value and raise it when it rises naturally.

  1. A matrix of Node versions

Mini-Reservalia declares "node": ">=20.6.0". Nothing verifies that claim. The matrix does.

  test:
    name: Tests (Node ${{ matrix.node }})
    runs-on: ubuntu-latest
    strategy:
      # Without fail-fast, if Node 22 fails we ALSO want to know whether Node 20 fails.
      # With fail-fast (the default), GitHub cancels the other entries
      # as soon as one fails and you lose half the information.
      fail-fast: false
      matrix:
        node: ['20', '22']
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
          cache: 'npm'
      - run: npm ci
      - run: npm test

What you should see: two checks, Tests (Node 20) and Tests (Node 22), running at the same time.

A warning that is going to bite you: using a matrix changes the check names. Your ruleset from 07-01 requires a check called Tests that no longer exists, so the PR will be stuck waiting for it forever. There are two solutions to this and the difference is worth understanding:

Solution How Trade-off
Update the ruleset with the new names Add Tests (Node 20) and Tests (Node 22) The configuration has to be touched every time the matrix changes
Aggregator job A ci-ok job with needs: [...] and if: always() that fails if anything failed; it is the only required check The ruleset is never touched again

The second one is what Reservalia uses and what we are going to implement in section 10.

  1. Sharding: splitting the suite in two

With 34 tests and 1.8 seconds, sharding adds nothing. We do it now precisely to have the mechanism in place before it is needed, which is when you have 700 tests and 11 minutes and everybody is waiting.

Node 20.6+ ships --test-shard=<index>/<total>, which distributes files deterministically:

node --test --test-shard=1/2 test/   # first half
node --test --test-shard=2/2 test/   # second half

In the matrix, two crossed dimensions:

    strategy:
      fail-fast: false
      matrix:
        node: ['20', '22']
        shard: [1, 2]
    steps:
      # ...
      - name: Tests (shard ${{ matrix.shard }}/2)
        run: node --test --test-shard=${{ matrix.shard }}/2 test/

Result: four jobs in parallel. Typical measurement on this project:

Configuration Wall-clock time of the test step Machine minutes consumed
1 job, whole suite ~2.0 s 1×
2 shards ~1.2 s ~2×
4 jobs (2 Node × 2 shards) ~1.2 s ~4×

The gain here is ridiculous because the split is by file and we have four very uneven files: the shard containing e2e.test.js (which starts a process) dominates the time. That is the real lesson of sharding, and 06-03 already hinted at it: splitting by file count gives poor results; splitting by historical timings gives good ones. CircleCI ships it out of the box; on GitHub Actions you have to build it or accept the naive split.

Rule of thumb: do not shard until the suite goes past 3-4 minutes, and when you do, balance the files manually or implement a timing-based split. Badly balanced sharding multiplies the cost without reducing the time.

  1. The complete ci.yml

# .github/workflows/ci.yml - 07-02 VERSION
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

permissions:
  contents: read

jobs:
  quality:
    name: Quality
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - name: ESLint
        run: npm run lint

  test:
    name: Tests
    runs-on: ubuntu-latest
    timeout-minutes: 15
    strategy:
      fail-fast: false
      matrix:
        node: ['20', '22']
        shard: [1, 2]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
          cache: 'npm'
      - run: npm ci

      - name: Run shard ${{ matrix.shard }}/2
        run: |
          mkdir -p reports
          node --test \
            --test-shard=${{ matrix.shard }}/2 \
            --test-reporter=tap --test-reporter-destination=reports/tests-${{ matrix.node }}-${{ matrix.shard }}.tap \
            --test-reporter=spec --test-reporter-destination=stdout \
            test/

      # `if: always()` to upload the report ALSO when the tests fail,
      # which is exactly when the report is worth something.
      - name: Upload test report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: tests-node${{ matrix.node }}-shard${{ matrix.shard }}
          path: reports/
          retention-days: 7

      - name: Annotate failures in the PR
        if: failure()
        run: node scripts/annotate-failures.js reports/tests-${{ matrix.node }}-${{ matrix.shard }}.tap

  coverage:
    name: Coverage
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - name: Full suite with coverage
        run: |
          mkdir -p reports
          npm run test:coverage
      - name: Coverage thresholds
        run: npm run coverage:check
        env:
          LINES_THRESHOLD: '85'
          BRANCHES_THRESHOLD: '75'
      - name: Upload LCOV
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: coverage-lcov
          path: reports/lcov.info

  build:
    name: Build
    runs-on: ubuntu-latest
    needs: [quality, test, coverage]
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm run build
      - uses: actions/upload-artifact@v4
        with:
          name: dist-${{ github.sha }}
          path: dist/
          retention-days: 7

  # Aggregator job: the ONLY required check in branch protection.
  # That way the matrix can grow or shrink without touching the ruleset.
  ci-ok:
    name: CI OK
    runs-on: ubuntu-latest
    needs: [quality, test, coverage, build]
    if: always()   # runs even if one of the `needs` failed or was skipped
    steps:
      - name: Evaluate the pipeline result
        run: |
          echo "quality:  ${{ needs.quality.result }}"
          echo "test:     ${{ needs.test.result }}"
          echo "coverage: ${{ needs.coverage.result }}"
          echo "build:    ${{ needs.build.result }}"
          if [ "${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}" = "true" ]; then
            echo "::error::At least one pipeline job did not pass."
            exit 1
          fi
          echo "Full pipeline green." >> "$GITHUB_STEP_SUMMARY"

Update the ruleset so that the only required check is CI OK:

# Find the id of the ruleset created in 07-01
gh api "repos/{owner}/{repo}/rulesets" --jq '.[] | "\(.id) \(.name)"'

And in Settings → Rules → protect-main, replace the three checks with a single one: CI OK.

Why if: always() and not if: success(). Without always(), if one of the needs fails, the aggregator job is skipped instead of failing. A skipped check does not report a status, and GitHub interprets that as "pending" forever. The PR is left blocked with a grey check and nobody understands why. With always(), the job always runs and always reports: green or red, but it reports.

  1. Manufacturing a flaky test and putting it in quarantine

This is the part of the lesson that will serve you best in a real job. We are going to create a test that fails sometimes, watch it fail, and apply the complete procedure.

11.1 Manufacturing it

Create test/bookings-today.test.js:

// test/bookings-today.test.js
// WARNING: this test is deliberately WRONG. It is the lesson's flaky example.
import test from 'node:test';
import assert from 'node:assert/strict';
import { calculateSlots } from '../src/availability.js';

const OPENING_HOURS = [{ start: '09:00', end: '14:00' }, { start: '16:00', end: '20:00' }];

/** Returns the slots that have not gone by yet, according to the system clock. */
function remainingSlotsToday(appointments = []) {
  const now = new Date();
  const hhmm = `${String(now.getHours()).padStart(2, '0')}:${String(now.getMinutes()).padStart(2, '0')}`;
  return calculateSlots(OPENING_HOURS, appointments, 60).filter((s) => s.start >= hhmm);
}

test('there are slots available today', () => {
  // FLAKY: it depends on the time of day the pipeline runs.
  // Green by day, red at night, red on a runner with TZ=UTC if you are in UTC+2.
  assert.ok(remainingSlotsToday().length > 0, 'no slots left today');
});

test('the first slot today starts at 09:00', () => {
  // FLAKY, and even worse: it only passes if it is before 09:00.
  assert.equal(remainingSlotsToday()[0]?.start, '09:00');
});

Run it several times with simulated times to see the pattern without waiting for nightfall:

# On Linux/macOS with the faketime library installed:
faketime '10:00' npm test    # the first one passes, the second fails
faketime '21:00' npm test    # both fail

# Without faketime, change the process time zone:
TZ=Pacific/Auckland npm test   # in Europe, this usually lands on night-time over there
TZ=UTC npm test

What you should see: the same commit, the same code, different results. Push it to the pipeline and re-run the workflow several times with Re-run all jobs; you will see green and red runs over identical code.

11.2 The damage it does

Consequence Measurable effect
People re-run the job instead of investigating "Re-run" becomes the default reflex
Red stops meaning "it is broken" People start merging with red checks "because it is that flaky one"
The signal from real failures is lost A genuine bug gets mistaken for the flaky test and reaches production
Merge time goes through the roof Every PR needs 2-3 passes

A single flaky test in a suite of 300 is enough to degrade trust in the other 299. That is why 02-04 insisted: a flaky test is not a minor failure, it is a failure of the signal.

11.3 The quarantine policy, step by step

Step 1 — Detect and label. Mark the test, do not delete it:

// test/bookings-today.test.js
import test from 'node:test';

// QUARANTINE #12 - flaky through dependency on the system clock.
// Isolated on 2026-04-10 by @your-username. Deadline: 2026-04-24.
// If it is not fixed by the 24th, it gets DELETED. See the policy in CONTRIBUTING.md.
test('there are slots available today', { skip: 'flaky: depends on the clock (#12)' }, () => {
  /* ... */
});

Step 2 — Isolate. Get it out of the critical path but keep it running, so it does not drop off the radar. Add an informational job:

  flaky-watch:
    name: Quarantined tests (informational)
    runs-on: ubuntu-latest
    # NOT in the `needs` of ci-ok: it blocks nothing.
    continue-on-error: true
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20', cache: 'npm' }
      - run: npm ci
      - name: Run the quarantine 5 times
        run: |
          FAILURES=0
          for i in 1 2 3 4 5; do
            node --test --test-skip-pattern='^$' test/bookings-today.test.js || FAILURES=$((FAILURES+1))
          done
          echo "### Quarantine: $FAILURES/5 failures" >> "$GITHUB_STEP_SUMMARY"
          [ "$FAILURES" -eq 0 ] && echo "Candidate to leave quarantine." >> "$GITHUB_STEP_SUMMARY"

Running the test N times is the right way to measure flakiness: one run does not distinguish "broken" from "unstable"; five do. And the job reports without blocking, which is what "quarantine" means.

Step 3 — Open the ticket with a deadline.

gh issue create \
  --title "Flaky: 'there are slots available today' depends on the system clock" \
  --body "$(cat <<'EOF'
**Symptom:** fails intermittently depending on the runner's time of execution.
**Frequency:** ~40 % of runs (100 % after 19:00 UTC).
**Root cause:** `remainingSlotsToday()` calls `new Date()` directly.
**Status:** in quarantine since 2026-04-10 (`skip`).
**Deadline:** 2026-04-24. If it is not fixed, the test gets deleted.
**Proposed fix:** inject the clock as a parameter.
EOF
)" --label flaky

Step 4 — Fix the root cause. A clock-driven flaky test is fixed by injecting the clock, never with a sleep or with retries:

// src/schedule.js
import { calculateSlots } from './availability.js';

/**
 * Slots that have not started yet at a given time.
 * @param {object} options
 * @param {() => Date} options.clock Injectable: in production `() => new Date()`,
 *        in the tests a function returning a fixed date.
 */
export function remainingSlots(openingHours, appointments, durationMin, { clock = () => new Date() } = {}) {
  const now = clock();
  const hhmm = `${String(now.getHours()).padStart(2, '0')}:${String(now.getMinutes()).padStart(2, '0')}`;
  return calculateSlots(openingHours, appointments, durationMin).filter((s) => s.start >= hhmm);
}
// test/schedule.test.js  (replaces bookings-today.test.js)
import test, { describe } from 'node:test';
import assert from 'node:assert/strict';
import { remainingSlots } from '../src/schedule.js';

const OPENING_HOURS = [{ start: '09:00', end: '14:00' }, { start: '16:00', end: '20:00' }];
const at = (hhmm) => () => new Date(`2026-04-15T${hhmm}:00`);

describe('remainingSlots (with an injected clock: deterministic)', () => {
  test('at 08:00 every slot of the day is still available', () => {
    assert.equal(remainingSlots(OPENING_HOURS, [], 60, { clock: at('08:00') }).length, 9);
  });

  test('at 12:30 only the ones from 13:00 onwards are left', () => {
    const slots = remainingSlots(OPENING_HOURS, [], 60, { clock: at('12:30') });
    assert.deepEqual(slots.map((s) => s.start), ['13:00', '16:00', '17:00', '18:00', '19:00']);
  });

  test('at 21:00 there are none left', () => {
    assert.deepEqual(remainingSlots(OPENING_HOURS, [], 60, { clock: at('21:00') }), []);
  });

  test('the edge case of exactly 20:00: the last slot has already started', () => {
    assert.deepEqual(remainingSlots(OPENING_HOURS, [], 60, { clock: at('20:00') }), []);
  });
});
rm test/bookings-today.test.js
npm test    # green, and green ALWAYS, at any hour, in any time zone

Run the suite ten times in a row to demonstrate determinism:

for i in $(seq 1 10); do npm test > /dev/null 2>&1 && echo "run $i: OK" || echo "run $i: FAILED"; done
# 10 times OK

What to take away. A flaky test almost always hides a hidden dependency on the environment: the clock, the execution order, a pinned port, a shared file, a genuine race condition, the network. Fixing it improves the code, not just the test. In this case, injecting the clock makes the function testable and makes it possible to implement "view a business's diary in another time zone" tomorrow without touching anything. Automatic retries, by contrast, hide the problem and often hide a real concurrency bug.

The most frequent sources and their fixes:

Source Symptom Correct fix Fake fix (do not do this)
Clock / date Fails at night, at month end, in another TZ Inject the clock TZ=Europe/Madrid in CI
Test order Fails when sharding or in parallel A beforeEach that cleans the state Force serial execution
Pinned port Intermittent EADDRINUSE listen(0), ephemeral port A sleep beforehand
Fixed wait Fails when the runner is slow Polling with retries and a limit Increase the sleep
External service Fails when the network fails A test double in integration; the real one only in E2E Retries

  1. Reports as artifacts and annotations in the PR

The workflow already uploads the .tap files. What is missing is turning the failures into annotations on the code of the PR. GitHub Actions reads special commands from standard output:

// scripts/annotate-failures.js
// Turns a node:test TAP report into GitHub Actions annotations.
// Usage: node scripts/annotate-failures.js reports/tests-20-1.tap
import { readFile } from 'node:fs/promises';

const path = process.argv[2];
if (!path) {
  console.error('Usage: node scripts/annotate-failures.js <file.tap>');
  process.exit(2);
}

const lines = (await readFile(path, 'utf8')).split('\n');
let failures = 0;

for (let i = 0; i < lines.length; i++) {
  const failure = lines[i].match(/^not ok \d+ - (.+)$/);
  if (!failure) continue;
  failures++;
  const name = failure[1].trim();

  // The YAML block that follows carries file/line/failureType/error.
  let file = '';
  let line = '1';
  let message = '';
  for (let j = i + 1; j < Math.min(i + 30, lines.length); j++) {
    const m = lines[j].match(/^\s*(file|line|error):\s*(.*)$/);
    if (!m) continue;
    const value = m[2].replace(/^['"]|['"]$/g, '').trim();
    if (m[1] === 'file') file = value.replace(`${process.cwd()}/`, '');
    if (m[1] === 'line') line = value;
    if (m[1] === 'error') message = value;
    if (lines[j].startsWith('  ...')) break;
  }

  // Escaping is mandatory: line breaks and ':' break the command.
  const clean = `${name}: ${message}`.replace(/%/g, '%25').replace(/\r?\n/g, '%0A').replace(/\r/g, '%0D');
  console.log(`::error file=${file || 'test'},line=${line},title=Test failed::${clean}`);
}

console.log(`::notice::${failures} failed test(s) in ${path}`);

What you should see when a test fails in a PR: in the Files changed tab, a red box over the exact line of the test file, with the assert's message. No need to open the log any more.

Try the whole chain: break an assert in test/api.test.js (change assert.equal(body.total, 9) to 10), push and watch:

  1. Two of the four Tests jobs red (the ones containing that file according to the shard).
  2. The tests-node20-shard1 artifact downloadable despite the failure, thanks to if: always().
  3. The red annotation over the assert's line.
  4. Build skipped.
  5. CI OK red, not grey.
  6. The merge button disabled.

Revert the change before continuing.

  1. Final verification

# Check How Expected
1 Three layers present ls test/ availability, repository, api, e2e, schedule
2 Suite green locally npm test ~38 tests, 0 failures
3 Repository contract npm test log The block repeats for memory and sqlite
4 E2E starts the real process Log Lines [server] Mini-Reservalia e2e-test listening...
5 Coverage published Run front page A "Coverage" table with percentages
6 The threshold breaks the build LINES_THRESHOLD=99 npm run coverage:check Exit code 1
7 A matrix of 4 jobs Run graph Tests (20,1), (20,2), (22,1), (22,2)
8 fail-fast: false works Break a test and look All 4 jobs run; none are cancelled
9 Report as an artifact on failure Artifacts section of a red run .tap downloadable
10 Annotation in the PR Files changed Red box over the assert
11 The flaky test is gone 10 consecutive runs 10 greens
12 CI OK is the only required check Ruleset A single check

Common Mistakes and Tips

Symptom: Error: Cannot find module 'better-sqlite3' or was compiled against a different Node.js version. Cause: a native module compiled for another Node version (typical when switching versions with nvm without reinstalling), or an npm cache restored from a different version. Fix: locally, rm -rf node_modules && npm ci. In CI, make sure setup-node comes before npm ci. If it persists, include the Node version in the cache key.

Symptom: the test job finishes, but the process takes an extra 30 s to exit. Cause: a server or a database left open and not closed. after() did not run, or repository.close() is missing. Fix: close everything in after(). To diagnose: node --test --test-force-exit test/ finishes anyway; if that makes it fast, you have a resource left open.

Symptom: EADDRINUSE: address already in use 127.0.0.1:3100 in the E2E test, only in CI. Cause: two shards on the same runner starting the process on the same port. Fix: either use port 0 and read the port from the child's log, or offset the port per shard with the PORT_OFFSET variable already provided for in the code: env: { PORT_OFFSET: ${{ matrix.shard }} }.

Symptom: coverage comes out at 0 %, or reports/lcov.info is empty. Cause: the reports/ directory did not exist when the reporter tried to write to it. Fix: mkdir -p reports before running the tests (that is why it is in the workflow).

Symptom: the PR is blocked with a grey Expected — Tests check. Cause: the ruleset requires a check name that is no longer generated, because the matrix changed the names. Fix: the CI OK aggregator job from section 10, and making it the only required check.

Symptom: npm test passes locally and fails in CI with differences in the order of appointments. Common cause: SQLite returns rows in insertion order when there is no ORDER BY, and that order can differ. If your test depends on the order, the order must be in the query. Fix: an explicit ORDER BY start (it is already in the repository) and assert.deepEqual over sorted lists, or an order-insensitive comparison.

Tip — how to distribute the effort. The healthy proportion in a project like this: ~70 % unit, ~25 % integration, ~5 % E2E. Not out of dogma, but out of economics: a unit test costs microseconds and points at the exact line; an E2E test costs seconds and only tells you "something in the flow is broken". If your suite takes too long, first check whether you have written as an E2E test something that was really a unit test.

Tip — the order of the stages. Cheap first. Lint (5 s) before unit tests (2 s) before integration (10 s) before E2E (30 s). A PR with a syntax error should die in the first job, not after four minutes.

Exercises

Exercise 1: a contract test for a third implementation

Write JsonRepository, which persists to a JSON file, and make it pass the same contract battery without modifying test/repository.test.js beyond adding it to the list. If your contract battery is well written, it should catch at least one bug in your implementation on the first attempt.

Exercise 2: minimum coverage per file, not just globally

The global threshold has a hole in it: you can have 90 % globally with a critical file at 40 %. Modify scripts/coverage.js so that it also fails if any file in src/ drops below 70 % of lines, with a configurable exclusion list.

Exercise 3: detecting flaky tests before they reach main

Add a job that, only on PRs that touch test/, runs the modified tests 5 times and fails if they are not deterministic. That is how a new flaky test never gets in.

Solutions

Solution 1.

// src/repository-json.js
import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'node:fs';
import { dirname } from 'node:path';
import { validateAppointment } from './repository.js';

export class JsonRepository {
  #path;
  #data;

  constructor(path) {
    this.#path = path;
    mkdirSync(dirname(path), { recursive: true });
    this.#data = existsSync(path)
      ? JSON.parse(readFileSync(path, 'utf8'))
      : { nextId: 1, appointments: [] };
  }

  #save() {
    // Atomic write: write to a temporary file and rename. Without this, a failure
    // mid-write leaves the file corrupt and the app never starts again.
    const temp = `${this.#path}.tmp`;
    writeFileSync(temp, JSON.stringify(this.#data, null, 2));
    require('node:fs').renameSync(temp, this.#path);
  }

  listAppointments(date) {
    return this.#data.appointments
      .filter((a) => a.date === date)
      .sort((a, b) => a.start.localeCompare(b.start));
  }

  createAppointment(data) {
    const appointment = { id: this.#data.nextId++, ...validateAppointment(data) };
    this.#data.appointments.push(appointment);
    this.#save();
    return appointment;
  }

  deleteAll() {
    this.#data = { nextId: 1, appointments: [] };
    this.#save();
  }

  close() {
    this.#save();
  }
}

(With ESM, replace the require with an import { renameSync } from 'node:fs' at the top: that is precisely one of the bugs the battery catches on the first attempt, because require does not exist in an ESM module.)

In the test, a single line:

import { mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { JsonRepository } from '../src/repository-json.js';

const IMPLEMENTATIONS = [
  ['memory', () => new MemoryRepository()],
  ['sqlite', () => new SqliteRepository(':memory:')],
  ['json', () => new JsonRepository(join(mkdtempSync(join(tmpdir(), 'repo-')), 'appointments.json'))],
];

Bugs the battery usually catches on the first attempt: the id returned as a string, the default customer not applied (if you forget to call validateAppointment), and dates getting mixed up if the filter is wrong. That is the value of a contract test: one battery, N implementations, zero duplication, and the guarantee that they really are interchangeable.

Solution 2.

// At the end of scripts/coverage.js, before the final exit:

const PER_FILE_THRESHOLD = Number(process.env.FILE_THRESHOLD ?? 70);
const EXCLUDED = (process.env.COVERAGE_EXCLUDE ?? 'src/server.js')
  .split(',')
  .map((s) => s.trim())
  .filter(Boolean);

const belowThreshold = files
  .filter((f) => f.file.includes('/src/'))
  .map((f) => ({ path: f.file.replace(`${process.cwd()}/`, ''), pct: pct(f.LH, f.LF) }))
  .filter((f) => f.pct < PER_FILE_THRESHOLD && !EXCLUDED.includes(f.path));

if (belowThreshold.length > 0) {
  const detail = belowThreshold.map((f) => `- \`${f.path}\`: ${f.pct.toFixed(1)} % (min ${PER_FILE_THRESHOLD} %)`);
  const block = ['', '### ❌ Files below the individual threshold', '', ...detail].join('\n');
  console.error(block);
  if (process.env.GITHUB_STEP_SUMMARY) {
    await appendFile(process.env.GITHUB_STEP_SUMMARY, `${block}\n`);
  }
  for (const f of belowThreshold) {
    console.log(`::error file=${f.path}::Coverage ${f.pct.toFixed(1)} %, below the minimum of ${PER_FILE_THRESHOLD} %`);
  }
  process.exit(1);
}

The exclusion list must be explicit and live in the repository, not hidden in a secret or in a tool's interface. And every exclusion should carry a comment and a date, exactly like the vulnerability exceptions in 07-05: an exception with no expiry date is a permanent exception.

Solution 3.

  flaky-detection:
    name: Flaky detection
    runs-on: ubuntu-latest
    if: github.event_name == 'pull_request'
    timeout-minutes: 20
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0   # needed to diff against the PR base

      - uses: actions/setup-node@v4
        with: { node-version: '20', cache: 'npm' }
      - run: npm ci

      - name: Locate the tests modified in this PR
        id: changes
        run: |
          FILES=$(git diff --name-only \
            "${{ github.event.pull_request.base.sha }}" HEAD \
            -- 'test/**/*.test.js' | tr '\n' ' ')
          echo "files=$FILES" >> "$GITHUB_OUTPUT"
          echo "Modified tests: ${FILES:-(none)}"

      - name: Run 5 times and demand determinism
        if: steps.changes.outputs.files != ''
        run: |
          set -u
          FILES="${{ steps.changes.outputs.files }}"
          FAILURES=0
          for i in 1 2 3 4 5; do
            echo "--- Pass $i of 5 ---"
            if node --test $FILES; then
              echo "pass $i: OK"
            else
              echo "pass $i: FAILED"
              FAILURES=$((FAILURES + 1))
            fi
          done

          {
            echo "## Flaky detection"
            echo ""
            echo "Files analysed: \`$FILES\`"
            echo ""
            echo "Result: **$((5 - FAILURES))/5 passes green**"
          } >> "$GITHUB_STEP_SUMMARY"

          if [ "$FAILURES" -gt 0 ] && [ "$FAILURES" -lt 5 ]; then
            echo "::error::NON-DETERMINISTIC behaviour: $FAILURES of 5 passes failed over the same code."
            echo "The tests in this PR are not deterministic. Check the clock, ordering, ports and fixed waits." >> "$GITHUB_STEP_SUMMARY"
            exit 1
          fi
          if [ "$FAILURES" -eq 5 ]; then
            echo "::error::The tests fail EVERY time: it is not flaky, it is broken."
            exit 1
          fi
          echo "Deterministic over 5 passes."

The distinction at the end is the key to the exercise and the one almost nobody implements: 5/5 failures = broken (the signal is correct, fix the code); 1-4/5 failures = flaky (the signal is corrupt, fix the test). They are two different problems with two different answers, and confusing them is exactly what leads a team to normalise red.

To go further: run it on a loaded runner too (stress-ng in the background) to catch the timing flaky tests that only appear when the machine is slow. That is what separates "green on my laptop" from "green on the runner on a Friday afternoon".

Optional Challenge

Rewrite the shard split so that it works by historical timings instead of by file count: store the duration of each test file in an artifact, retrieve it on the next run, and distribute the files with a greedy "the next file goes to the least loaded shard" algorithm. With four files of durations 0.2 s / 0.3 s / 0.4 s / 1.5 s, the naive split gives 0.5 s and 1.9 s; the greedy one gives 1.5 s and 0.9 s. It is exactly what CircleCI does with --split-by=timings (06-03), and doing it by hand teaches you why it is a feature and not a line of configuration.

What You Have Built

  • A persistence layer with three interchangeable pieces and a contract test that guarantees they are.
  • The three layers of the pyramid: 17 unit tests with real edge cases, integration against SQLite and against the HTTP routes, and an end-to-end test against the running process with its environment-based configuration.
  • Coverage measured, published in the run summary and gated by a threshold that breaks the build, verified from the failure side.
  • A matrix of 4 jobs (2 Node versions × 2 shards) with fail-fast: false, and an understanding of why naive sharding pays off so little.
  • An aggregator job CI OK that decouples branch protection from the shape of the pipeline.
  • Reports as artifacts even when it fails and annotations on the code in the PR.
  • A flaky test manufactured, diagnosed, quarantined with a deadline and fixed at its root cause, with the lesson that the fix improved the production code.

Conclusion

The pipeline has gone from "the code compiles and seven tests pass" to "the code is verified at three levels, on two Node versions, with a known coverage and with a mechanism that stops an unstable test corrupting the signal". That difference is what separates a decorative CI from one you can trust enough to deploy without looking.

And that last sentence is the hinge into the next lesson. Everything you have built has a single purpose: to let a change reach production without anybody having to review it by hand. For now, the pipeline produces a dist/ directory that is kept for seven days and goes nowhere.

In 07-03 we close the loop. You will package Mini-Reservalia into a multi-stage Dockerfile with a non-root user and a HEALTHCHECK, build it with Buildx and cache, publish it to ghcr.io identified by its digest (free, no AWS, using the GITHUB_TOKEN), separate CI from CD with a cd.yml triggered by workflow_run, create Environments with automatic staging and production with a required reviewer —and watch the run waiting for your approval—, deploy with an idempotent script, verify with a smoke test with retries, promote by digest from staging to production while checking it is the same image byte for byte, and write a rollback.yml that you will run with a stopwatch in hand. And, as always, you will break something on purpose: you will deploy a version that fails the /health check to watch the gate close and the rollback work.

◄ Previous
◄
Next ►
►

CI/CD Course: Continuous Integration and Deployment

Module 1: Introduction to CI/CD

Module 2: Continuous Integration (CI)

Module 3: Continuous Deployment (CD)

Module 4: Advanced CI/CD Practices

Module 5: Implementing CI/CD in Real Projects

Module 6: Tools and Technologies

Module 7: Practical Exercises

Module 8: Additional Resources

The Project | About Us | Contribute | Donations | License
© Copyright 2026. All rights reserved