Jenkins solved the "I want to automate anything" problem and left everything else — repository, registry, environments, security — in the hands of plugins or other products. GitLab starts from the opposite position: a single product that contains the repository, code review, CI/CD, the container registry, the environments, security analysis, issues and the wiki, with everything connected out of the box. That architectural bet — what their marketing calls a DevOps platform and what we will simply call an integrated platform — is what you have to understand in order to evaluate it properly, because it explains both its greatest virtues and its costs. GitLab CI is also historically important: its .gitlab-ci.yml (2015) was one of the first to normalise pipeline as code embedded in the repository, and its model of stages with jobs inside them is the one many engineers carry in their heads when they think "pipeline". In this lesson we will look at that model, its evolution from fixed phases to a graph with needs, the Reservalia pipeline translated a second time, the distinction between cache and artifacts that causes the most confusion, runners and their executors, environments and review apps, the reuse mechanisms, and what it costs to operate your own instance.
Contents
- The integrated platform: what it is and what it implies
- The execution model: stages, jobs and the move to a graph
- The Reservalia pipeline in
.gitlab-ci.yml cacheversusartifacts: the distinction that confuses most- Parallelisation:
parallelandparallel: matrix - Runners and executors
- Variables, secrets and protection
- Environments, manual deployment and review apps
- Reuse:
extends,include, components and parent-child pipelines - The integrated container registry
- Auto DevOps, with judgement
- SaaS versus self-hosted, and the real cost
- When GitLab CI/CD is the right choice
- Common Mistakes and Tips
- Exercises
- Conclusion
- The integrated platform: what it is and what it implies
In a typical organisation using GitHub, the toolchain is assembled: GitHub for the code, GitHub Actions for CI, ECR or Docker Hub for the registry, Snyk or Dependabot for dependencies, Jira for issues, something separate for environments. In GitLab, all of that consists of tabs in the same project.
What you gain, and this is not rhetoric:
- Zero integrations to maintain between pieces. There are no crossed tokens between CI and the registry, no webhooks that break, no "the Jira bot lost its permissions". The CI job pushes to the registry of the same project with a variable that already exists.
- End-to-end traceability. From the issue to the merge request, to the pipeline, to the artifact, to the environment where it is deployed. The question "which version is in production and which issues does it include?" has an answer in the interface without building anything.
- A single permissions model. Whoever can merge can deploy to staging but not to production, and that is configured once.
- Features that only exist because the pieces are together: the security reports that appear inside the merge request comparing the branch against
main, the environments that show which commit is deployed, the review apps with their link in the MR.
What you pay, with the same frankness:
- Tight coupling. Leaving GitLab means leaving all of it at once: repository, CI, registry and environment history. It is the highest switching cost in the module, and 06-07 quantifies it.
- The interesting features are tiered by subscription level. A good deal of what gets covered in articles and conference talks — multiple approvals, some security analyses, certain compliance controls — is not in the free tier. It is unwise to design an architecture on top of a feature and only afterwards find out which tier it lives in.
- If you self-host it, you operate a whole platform, not a CI server: database, object storage, Redis, Gitaly, the registry, the runners. Section 12.
A useful note on vocabulary: in GitLab, a merge request (MR) is what GitHub calls a pull request. The concept is the one 02-07 described.
- The execution model: stages, jobs and the move to a graph
GitLab CI's original model is sequential phases with parallel jobs inside them:
flowchart LR
subgraph S1["stage: prepare"]
A["prepare"]
end
subgraph S2["stage: verify"]
B["quality"]
C["test 1/4"]
D["test 2/4"]
E["test 3/4"]
F["test 4/4"]
end
subgraph S3["stage: build"]
G["build"]
H["security"]
end
subgraph S4["stage: publish"]
I["publish"]
end
S1 --> S2 --> S3 --> S4
The classic rule: all the jobs in a stage run in parallel, and the next stage does not start until the previous one has finished entirely. It is an easy model to reason about with one clear flaw: if test 4/4 takes eight minutes and the rest take two, build waits eight minutes even though it only depended on prepare. It is exactly the stage-barrier problem 04-01 described.
The correction came with needs, which turns the pipeline into a DAG: a job with needs starts as soon as its specific dependencies finish, without waiting for its stage. With needs, stages stop being barriers and become mostly visual grouping.
stages model |
needs model (DAG) |
|
|---|---|---|
| When a job starts | When the whole previous stage finishes | When its dependencies finish |
| Ease of reading | Very high | Medium: you have to follow the graph |
| Total time | Sum of the maxima per stage | The real critical path |
| Risk | Unnecessary waiting | Tangled graphs if nobody reviews them |
GitLab vocabulary translated:
| GitLab | Equivalent in the course |
|---|---|
| Pipeline | Workflow run |
| Stage | Stage (grouping) |
| Job | Job |
Script (a line of script:) |
Step |
| Runner | Runner / agent |
| Executor | How the runner materialises the environment (shell, docker, kubernetes) |
| Artifacts | Artifacts between jobs and downloadable |
| Environment | Deployment environment with history |
- The Reservalia pipeline in
.gitlab-ci.yml
.gitlab-ci.ymlSecond translation of the same pipeline: install with cache → lint and test in parallel → build the image → publish by digest.
# .gitlab-ci.yml — Reservalia · CI
stages: [prepare, verify, build, publish, deploy] # 1
default: # 2
image: node:22-bookworm
interruptible: true # cancels if a new push arrives
retry:
max: 2
when: [runner_system_failure, stuck_or_timeout_failure] # 3 · retry infrastructure failures only
variables:
NPM_CONFIG_CACHE: "$CI_PROJECT_DIR/.npm" # 4 · cache inside the workspace
IMAGE: "$CI_REGISTRY_IMAGE/api"
FF_USE_FASTZIP: "true"
workflow: # 5
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- when: never # nothing else triggers a pipeline
# ---------------------------------------------------------------- templates
.node: # 6 · reusable template
cache:
key:
files: [package-lock.json] # 7 · key derived from the lockfile
paths: [.npm/]
policy: pull # read only; the "prepare" job writes it
before_script:
- npm ci --prefer-offline --no-audit
# ---------------------------------------------------------------- jobs
prepare:
stage: prepare
extends: .node
cache:
key:
files: [package-lock.json]
paths: [.npm/]
policy: pull-push # 8 · this one does update the cache
script:
- echo "Dependencies installed and cache populated"
quality:
stage: verify
extends: .node
needs: [prepare] # 9 · DAG
script:
- npx prettier --check .
- npm run lint
- npm run typecheck
test:
stage: verify
extends: .node
needs: [prepare]
parallel: 4 # 10 · sharding
services: # 11 · PostgreSQL as a service
- name: postgres:16-alpine
alias: db
variables:
POSTGRES_DB: reservalia_test
POSTGRES_PASSWORD: test
DATABASE_URL: "postgres://postgres:test@db:5432/reservalia_test"
script:
- npm run migrate
- npm test -- --shard=$((CI_NODE_INDEX))/$CI_NODE_TOTAL
artifacts: # 12
when: always
expire_in: 1 week
reports:
junit: reports/junit-*.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura.xml
build-web:
stage: build
extends: .node
needs: [prepare]
script:
- npm run build --workspace apps/web
artifacts:
paths: [apps/web/dist/] # 13 · this really is an artifact
expire_in: 1 day
build-api:
stage: build
needs: [prepare]
image: docker:27
services: [docker:27-dind] # 14 · Docker-in-Docker
variables:
DOCKER_TLS_CERTDIR: "/certs"
script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
- |
docker buildx build \
--file apps/api/Dockerfile \
--cache-from type=registry,ref=$IMAGE:cache \
--cache-to type=registry,ref=$IMAGE:cache,mode=max \
--tag $IMAGE:$CI_COMMIT_SHA \
--push .
- docker buildx imagetools inspect $IMAGE:$CI_COMMIT_SHA
--format '{{.Manifest.Digest}}' > digest.env.tmp
- echo "DIGEST=$(cat digest.env.tmp)" > build.env
artifacts:
reports:
dotenv: build.env # 15 · outputs between jobs
security:
stage: build
extends: .node
needs: [prepare]
script:
- npm audit --audit-level=high
- gitleaks detect --no-git --exit-code 1
allow_failure: false
publish:
stage: publish
needs: [quality, test, build-api, build-web, security] # 16 · fan-in
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # only on the default branch
image: docker:27
services: [docker:27-dind]
script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
- docker buildx imagetools create --tag $IMAGE:stable $IMAGE@$DIGEST # promotion by digest
deploy-staging:
stage: deploy
needs: [publish]
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
environment: # 17
name: staging
url: https://staging.reservalia.example
script:
- ./scripts/deploy.sh staging "$IMAGE@$DIGEST"
deploy-production:
stage: deploy
needs: [deploy-staging]
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: manual # 18 · approval
allow_failure: false
environment:
name: production
url: https://app.reservalia.example
script:
- ./scripts/deploy.sh production "$IMAGE@$DIGEST"stagesdeclares the order. A job with nostagefalls intotestby default, which is surprising; always declare it.defaultapplies to every job what used to be repeated.interruptible: truetogether with the project's auto-cancel option is thecancel-in-progressof 04-04.- Selective
retry. Retrying any failure masks the flaky tests of 02-04 and turns the pipeline into a false-green generator; retrying only runner infrastructure failures is legitimate. That distinction is the difference between resilience and self-deception. - The cache has to live inside
$CI_PROJECT_DIR: GitLab can only cache paths within the workspace. That is why the npm cache is redirected there instead of being left in~/.npm. It is the number one caching mistake in GitLab. workflow:rulesdecides whether a pipeline is created at all. Without it, a push to a branch with an open MR generates two pipelines (one for the branch and one for the MR): it doubles the spend and confuses the signal.- A template is a job whose name starts with a dot: GitLab does not run it and it exists for
extends. cache:key:filesderives the key from the hash of the lockfile: change the lockfile, change the cache. It is thehashFilesof 04-02 under another name.policyis the piece almost nobody uses and that saves a lot of time:pullonly downloads,pull-pushdownloads and uploads at the end. With six jobs uploading the same identical cache you waste minutes; here onlypreparewrites it.needsturns the pipeline into a graph. Withneeds: []a job starts immediately, ignoring its stage.parallel: 4creates four instances of the job withCI_NODE_INDEX(1..N) andCI_NODE_TOTAL. It is the sharding of 02-04; the split by historical timings you will see in 06-03 does not come as standard.servicesbrings up auxiliary containers reachable by theiralias, just like the PostgreSQL of 02-02.artifacts:reportsare artifacts with semantics: GitLab interprets them and shows them in the MR (test results, coverage, security findings).when: alwaysis essential: test reports matter above all when the job fails.- A fundamental distinction, developed in section 4:
dist/is an artifact — pipeline output, passed to other jobs;.npm/is cache — it speeds things up, and losing it breaks nothing. - Docker-in-Docker is the usual pattern for building images in GitLab, and it has security implications that 06-05 develops: the
dindservice requires privileged runners. artifacts:reports:dotenvis the mechanism for outputs between jobs: the file's variables become available in the jobs that depend on this one. It is the equivalent of GitHub Actions'needs.<job>.outputs.- The fan-in of 04-01:
publishwaits for the five signals. environmentrecords the deployment: GitLab stores which commit is onstaging, since when, and offers a button to redeploy an earlier version.when: manualwithallow_failure: falseis the gate of 03-01: the pipeline stops and waits for a person to press the button, and who may press it is determined by the environment's protection.
Compared with the original ci.yml, the translation is almost one to one. The real differences: sharding is a number rather than a matrix, outputs between jobs travel through a dotenv file instead of through outputs, the cache requires configuring paths and policy by hand, and the container registry and the environments are part of the same product rather than external services with their own credentials.
cache versus artifacts: the distinction that confuses most
cache versus artifacts: the distinction that confuses mostBoth store files and both restore them in another job. They are not the same, and confusing them produces slow or incorrect pipelines.
cache |
artifacts |
|
|---|---|---|
| Purpose | Speed things up (dependencies, intermediate builds) | Carry results between jobs and make them downloadable |
| If it disappears | The job is slower, but it works | The pipeline fails or the result is lost |
| Where it is stored | On the runner (or in shared storage, if configured) | Always on the GitLab server |
| Scope | Shared between pipelines and branches, according to key |
Belongs to the specific pipeline run |
| How it is retrieved | By key, best-effort, no guarantees |
Automatically from the jobs that need it |
| Expiry | Runner policy | expire_in, explicit |
| Typical content | .npm/, ~/.gradle, vendor/ |
dist/, .war, JUnit reports, SBOM |
A mental rule that resolves 100% of cases: if deleting it breaks the pipeline, it is an artifact; if it only makes it slower, it is cache.
Two related mechanisms:
# Download ONLY the artifacts this job needs
publish:
needs:
- job: build-web
artifacts: true # downloads dist/
- job: security
artifacts: false # ordering dependency only, no file transferWithout this, a job downloads by default the artifacts of every job in previous stages, and in a large pipeline that is hundreds of megabytes moved for no reason: one of the most frequent and least diagnosed causes of a slow pipeline. dependencies: [] is the old way of saying "do not download anything for me".
And expire_in is not optional: artifacts consume billable storage in SaaS and disk when self-hosted. The retention policy of 02-06 is one line per job here.
- Parallelisation:
parallel and parallel: matrix
parallel and parallel: matrix# Simple sharding: N identical copies that split the work by index
test:
parallel: 4
script: [ "npm test -- --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL" ]
# Matrix: combinations of variables, like the matrix of 02-04
test-compatibility:
parallel:
matrix:
- NODE: ["20", "22"]
POSTGRES: ["15", "16"] # → 4 jobs
- NODE: ["22"] # additional blocks are added on top
POSTGRES: ["17"]
EXPERIMENTAL: "true"
image: node:$NODE
services: [ "postgres:$POSTGRES-alpine" ]
script: [ "npm test" ]Differences from GitHub Actions' matrix worth being clear about: there is no exclude — you model it by adding blocks instead of subtracting — and there is no global fail-fast: control is per job with allow_failure. In exchange, parallel: matrix does let you vary image and services, which gives you plenty of room.
- Runners and executors
A runner is the process that executes jobs. It registers against an instance and is assigned jobs according to its tags.
| Runner type | Scope | When |
|---|---|---|
| Shared | The whole instance | The normal case in SaaS; consumable minutes |
| Group | All the projects in a group | Your own runners shared by a department |
| Project-specific | One project | Special hardware, access to particular networks |
And the executor determines how it materialises the environment:
| Executor | How it runs | Isolation | When to use it |
|---|---|---|---|
shell |
Commands on the runner's machine | None | Almost never; it inherits the contaminated agent problem of 06-01 |
docker |
One container per job, from image: |
Good | The normal case |
docker+machine |
Creates a VM per job and destroys it | Very good | Autoscaling in the cloud |
kubernetes |
One Pod per job | Very good | If you already have a cluster |
ssh, custom |
Remote machines, custom integrations | Variable | Unusual hardware |
# config.toml of a self-hosted runner with the docker executor
concurrent = 8 # simultaneous jobs on this machine
check_interval = 3
[[runners]]
name = "runner-reservalia-1"
url = "https://gitlab.example.com/"
token = "glrt-..." # registration token
executor = "docker"
[runners.docker]
image = "node:22-bookworm" # default image if the job does not specify one
privileged = false # true ONLY if dind is needed (see 06-05)
volumes = ["/cache", "/certs/client"]
memory = "4g"
cpus = "2"
[runners.cache] # cache shared between runners
Type = "s3"
Shared = true
[runners.cache.s3]
ServerAddress = "s3.eu-west-1.amazonaws.com"
BucketName = "reservalia-ci-cache"
AuthenticationType = "iam" # instance role, no long-lived keysThree operational points with consequences:
privileged = truefor Docker-in-Docker is a real hole: a job with access to the privileged daemon can escape the container and compromise the machine, including the secrets of other jobs running there. Unprivileged alternatives in 06-05 (Kaniko, Buildah, rootless BuildKit).- Cache with shared storage changes performance completely. With a local cache on the runner and several runners, each one has its own copy and the hit rate collapses. With shared S3, everybody shares.
concurrentis the cost lever: sizing it badly either leaves jobs queued (which is perceived as "CI is slow", even though execution time is the same) or wastes machines.
- Variables, secrets and protection
GitLab has a single mechanism — CI/CD variables — with attributes that change its behaviour:
| Attribute | Effect | When to enable it |
|---|---|---|
| Protected | Only exposed in pipelines of protected branches and tags | Always, for any staging or production credential |
| Masked | Its value is replaced by [MASKED] in the logs |
Always, for secrets |
| File | Materialised as a file, with the variable holding the path | Kubeconfig, keys, certificates |
| Environment scope | A different value per environment (production, staging, *) |
Per-environment configuration from 03-02 |
| Expanded | Other variables are interpolated inside it | Turn off for secrets containing $ |
Two important warnings. Masked has formatting restrictions — minimum length, no spaces or certain characters; a secret that does not comply simply is not masked, and GitLab says so in a notice that is easy to miss. And masking has the same limitation as in Jenkins: it covers the literal match, not the transformed one. The conclusion of 04-03 does not change with the tool.
Protected is the real protection, and the one most often forgotten. Without it, anyone who opens an MR from an arbitrary branch can write a .gitlab-ci.yml that prints the production credentials. With the variable marked as protected, that variable does not exist in pipelines of unprotected branches. Rule: every deployment credential protected; no exceptions.
For federated identity, GitLab issues OIDC tokens per job with id_tokens, exactly the no-long-lived-keys pattern of 03-02:
deploy-production:
id_tokens:
AWS_TOKEN:
aud: https://gitlab.example.com # audience the identity provider expects
script:
- >
export $(aws sts assume-role-with-web-identity
--role-arn "$AWS_ROLE_ARN"
--role-session-name "gitlab-$CI_JOB_ID"
--web-identity-token "$AWS_TOKEN"
--query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]'
--output text | awk '{print "AWS_ACCESS_KEY_ID="$1"\nAWS_SECRET_ACCESS_KEY="$2"\nAWS_SESSION_TOKEN="$3}')
- ./scripts/deploy.sh productionThe trust condition on AWS is defined over attributes of the token (project, branch, environment), so that only a main pipeline deploying to production can assume the role. It is the same design as 03-02 with a different issuer.
- Environments, manual deployment and review apps
environment: is one of GitLab's finest pieces and it has no exact out-of-the-box equivalent in every tool: it turns a job into a recorded deployment, with a history of which commit is deployed, a link to the URL, and buttons to redeploy or roll back to an earlier deployment.
review:
stage: deploy
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
environment:
name: review/$CI_COMMIT_REF_SLUG # 1 · one environment per branch
url: https://$CI_COMMIT_REF_SLUG.review.reservalia.example
on_stop: stop-review # 2
auto_stop_in: 3 days # 3
script:
- ./scripts/deploy-review.sh "$CI_COMMIT_REF_SLUG" "$IMAGE@$DIGEST"
stop-review:
stage: deploy
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
when: manual
environment:
name: review/$CI_COMMIT_REF_SLUG
action: stop # 4
variables:
GIT_STRATEGY: none # the code is not needed to tear it down
script:
- ./scripts/destroy-review.sh "$CI_COMMIT_REF_SLUG"- One dynamic environment per branch: these are the review apps, the direct equivalent of the per-PR previews of 02-07. GitLab publishes the link inside the merge request, so Nuria can try the change without checking out the branch.
on_stoplinks the job that tears the environment down; it runs automatically when the MR is closed or merged.auto_stop_inis the line that prevents the surprise bill: without it, every abandoned branch leaves an environment running indefinitely. It is the problem 05-01 pointed out with orphaned previews, solved here with a single line.action: stopmarks the job as the one that shuts the environment down.
On approvals: when: manual holds the job until somebody launches it, and who may do so is controlled by protecting the environment (Deployments → Protected environments), which is the equivalent of the Environment reviewers of 03-02. The paid tiers offer multiple approvals and separation of duties — whoever approves cannot be whoever wrote the change — which is what audits usually demand.
- Reuse:
extends, include, components and parent-child pipelines
extends, include, components and parent-child pipelinesGitLab offers more reuse mechanisms than any other tool in the module, and it pays to know which one applies.
# a) YAML anchors: textual substitution, with no knowledge of GitLab
.base: &base
image: node:22
retry: 1
quality:
<<: *base
script: [ "npm run lint" ]
# b) extends: the same thing but structure-aware (deep-merges maps)
.node:
image: node:22
cache: { key: { files: [package-lock.json] }, paths: [.npm/] }
quality:
extends: .node # preferable to anchors: merges properly and allows multiple inheritance
script: [ "npm run lint" ]
# c) include: bring YAML in from outside
include:
- local: '/ci/templates/node.yml' # same repository
- project: 'reservalia/ci-templates' # another project on the instance
ref: 'v3.2.0' # pinned to a tag, not to main!
file: '/templates/build-publish.yml'
- remote: 'https://example.com/template.yml' # URL: no version control → avoid
- template: 'Security/SAST.gitlab-ci.yml' # official GitLab template
- component: gitlab.com/reservalia/components/build@1.2.0 # component with typed inputs
inputs:
dockerfile: apps/api/Dockerfile
publish: trueextends versus anchors: anchors are pure YAML, they are resolved before GitLab understands anything and they do not work across included files; extends deep-merges maps, supports chains and does cross include. Use extends.
Components (CI/CD components) are the most recent evolution and the closest to what 04-05 argued for: units with typed inputs and their own version, published in a catalogue, instead of YAML included blind. An include: template validates nothing; a component fails while parsing the file if you pass it a boolean where it expects a string.
And the rule from 04-03 applies just the same here: an include from another project pinned to main means somebody else's commit changes your pipeline without you having merged anything. Pin to a tag or a SHA.
Parent-child pipelines for monorepos, which is the selective execution problem of 04-04:
# The parent's .gitlab-ci.yml
generate-children:
stage: prepare
script:
- node scripts/generate-pipeline.js > child.yml # decides what to build based on what changed
artifacts: { paths: [child.yml] }
run-api:
stage: verify
rules:
- changes: [ "apps/api/**/*", "packages/shared/**/*" ] # only if this changed
trigger:
include:
- artifact: child.yml # dynamically generated pipeline
job: generate-children
strategy: depend # the parent waits and reflects the resultAnd trigger:project to launch another project's pipeline — the multi-project case of 05-03 — with strategy: depend if the parent has to wait for the child.
| Mechanism | What it reuses | When |
|---|---|---|
| YAML anchors | Fragments, same file | Almost never: use extends |
extends |
Job configuration | Repetition within one project |
include: local |
Files from the repository | Splitting up a large .gitlab-ci.yml |
include: project |
Templates across projects | Standardising across the organisation |
| Component | Versioned unit with inputs | The recommended option today for shared templates |
| Child pipeline | A whole pipeline | Monorepo, dynamic pipelines |
trigger: project |
Another project's pipeline | Multi-project, microservices |
- The integrated container registry
Every GitLab project comes with its own container registry, and that removes the credential paperwork that takes up a whole job in Reservalia's ci.yml:
publish:
image: docker:27
services: [docker:27-dind]
script:
# These three variables exist without configuring them: GitLab injects them per job
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
- docker build -t "$CI_REGISTRY_IMAGE/api:$CI_COMMIT_SHA" -f apps/api/Dockerfile .
- docker push "$CI_REGISTRY_IMAGE/api:$CI_COMMIT_SHA"CI_REGISTRY_PASSWORD is an ephemeral token with permission only over that project's registry and with the lifetime of the job. It is least privilege (04-03) without having configured anything, and it is a good example of what the integration gives you for free.
The registry includes cleanup policies by age and by tag pattern, which is the retention of 02-06 as a form. Without them, storage grows unchecked: a pipeline that publishes per commit produces dozens of images a day. Configure them on day one and always keep whatever an active environment references, because deleting the image that is deployed makes the rollback by digest of 03-05 impossible.
There is also a package registry (npm, Maven, PyPI…) and a dependency proxy registry, which solves the private registry case of 04-02 without setting up Nexus or Artifactory separately.
- Auto DevOps, with judgement
Auto DevOps is a complete preconfigured pipeline: it detects the language, builds with buildpacks, runs tests, SAST analysis, dependencies, licences, container scanning, DAST, and deploys to Kubernetes with review, staging and production with canary. You enable it with a checkbox.
An honest assessment: it is an excellent demonstration and a debatable production baseline. In favour: for a new, standard project, in ten minutes you have a pipeline with more quality gates than many companies have after a year. Against: the pipeline is a large black box that you have to understand in order to modify, it assumes Kubernetes with a specific topology, building with buildpacks produces images you do not control — and 06-05 will explain why you would want to control them — and as soon as you need something particular you end up overriding so many variables that writing the pipeline yourself would have been cheaper.
Recommended use: as a catalogue of ideas and as a starting point to switch off early. Enable it, look at what jobs it generates, copy the ones that suit you into your own .gitlab-ci.yml and disable it. What is worth adopting separately are the security templates (include: template: Security/...), which are discrete, understandable, adjustable pieces.
- SaaS versus self-hosted, and the real cost
| GitLab.com (SaaS) | Self-hosted | |
|---|---|---|
| Who operates it | GitLab | You |
| Cost model | Per user per month, with compute minutes included and additional consumables | Licence per user (depending on tier) plus infrastructure plus people |
| Upgrades | Continuous | Yours, with a window and a rehearsal |
| Data | Outside your network | Wherever you decide |
| Runners | Shared, or your own | Your own |
| Point of failure | The provider | Your instance |
Self-hosting GitLab is not self-hosting a CI server: it is operating a platform with PostgreSQL, Redis, Gitaly (the Git service), object storage, the registry and the runners. The real recurring tasks: upgrades — GitLab releases at a fast pace and skipping intermediate versions is not always supported — backups with a rehearsed restore, storage growth (artifacts and the registry are the ones that explode), and runner sizing. In practice it is at least half a part-time person in a mid-sized organisation, and it is not a cost that goes away over time.
On the figures: per-user prices, included minutes and which feature sits in which tier change frequently and by region. What matters here is the model: you pay per user, not per project; compute is billed per minute with multipliers depending on the machine type; artifact and registry storage counts too. Check the current pricing before deciding and, above all, verify which tier holds the specific feature you want to build on before designing anything.
- When GitLab CI/CD is the right choice
Yes, quite clearly, when:
- Your code is already in GitLab. It is the factor that decides most cases and 06-07 puts it first for a reason: using another CI against a GitLab repository means synchronising identities, webhooks and permissions in order to get less.
- You want a platform rather than a toolchain, and you value not maintaining integrations or reconciling permissions across five products.
- You need to self-host everything for regulatory reasons, and you want an integrated experience Jenkins does not give you. It is probably the best point in the self-hosted quadrant.
- Environments and review apps matter to you: the
environmentmodel with history,on_stopandauto_stop_inis mature and saves you writing your own code. - You work in a monorepo and need dynamic pipelines: parent-child with YAML generation is a first-class solution to the problem of 04-04.
No, or think twice, when:
- Your code is on GitHub. Mirroring repositories in order to use GitLab CI is a permanent source of friction.
- Your team is small and does not want to operate anything: SaaS solves that, but the per-user cost with many occasional collaborators weighs.
- You depend on features from higher tiers: check the tier before designing, not after.
- You need a lot of macOS or exotic hardware: it is feasible with your own runners, but the mobile ecosystem is better trodden in other tools (06-03, 05-02).
Common Mistakes and Tips
Caching paths outside $CI_PROJECT_DIR. The cache is not stored and there is no visible error, only slowness. Redirect NPM_CONFIG_CACHE, GRADLE_USER_HOME or equivalents to the workspace.
Every job with policy: pull-push. They upload the same cache over and over. Only the job that generates it needs to write it.
Duplicated pipelines in MRs. A push to a branch with an open MR creates two pipelines. workflow:rules fixes it and saves half the compute.
Confusing cache with artifacts. Putting dist/ in the cache produces deployments with a build from another branch; putting node_modules in artifacts uploads hundreds of megabytes per job to the server. The rule: if losing it breaks the pipeline, it is an artifact.
Not setting expire_in. Storage grows until somebody gets the bill or the disk fills up.
Downloading artifacts you do not need. By default you get those of every previous stage. Use needs with artifacts: false where you only need ordering.
Deployment variables not marked as protected. Any branch can read them from a .gitlab-ci.yml modified in an MR. It is the most direct route to credential exfiltration in GitLab.
Review apps without on_stop or auto_stop_in. Orphaned environments piling up and billing.
retry without when. Retrying everything hides flaky tests and produces false green (02-04).
An include from another project pointing at main. Somebody else's change modifies your pipeline without review. Pin to a tag.
Privileged runners out of habit. privileged = true only where dind is genuinely needed, and evaluate daemonless alternatives (06-05).
Exercises
Exercise 1. The Reservalia pipeline in GitLab takes 19 minutes. You observe: the verify stage finishes in 6 min but build does not start until minute 9; each of the six jobs uploads 400 MB of cache; the publish job downloads 1.2 GB of artifacts and only uses a digest file; and a push to a branch with an open MR launches two pipelines. Write the concrete fixes and estimate the effect of each one.
Exercise 2. Nuria wants review apps for apps/web: one environment per MR with its own URL, an automatic comment in the MR, teardown when it is closed and automatic expiry after three days. Write the complete jobs and explain what each line protects. Add what to do about secrets, knowing that an MR may come from any team member's branch.
Exercise 3. The company that bought Gestor Citas 4 (05-04) has its code in a self-hosted GitLab 14.x, with pipelines that use only/except, without needs and with a single shell runner that executes everything on the server machine. Write the phased modernisation plan, with what is gained in each phase and what the risks are.
Solutions
Solution 1.
| Problem | Cause | Fix | Estimated effect |
|---|---|---|---|
build waits 3 min |
Stage barrier: it waits for the slowest job in verify |
needs: [prepare] in build-web and build-api |
−3 min off the critical path |
| 6 × 400 MB of cache upload | All of them with policy: pull-push |
Only prepare with pull-push; the rest pull |
−2 GB of traffic; ~1 min per job |
publish downloads 1.2 GB |
Implicit download of artifacts from previous stages | Explicit needs with artifacts: false except the build's dotenv |
−1 to 2 min |
| Duplicated pipelines | Missing workflow:rules |
Rules for MR and default branch, when: never for the rest |
−50% of the compute spend |
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- when: never
.node:
cache:
key: { files: [package-lock.json] }
paths: [.npm/]
policy: pull # read only
prepare:
extends: .node
cache:
key: { files: [package-lock.json] }
paths: [.npm/]
policy: pull-push # the only one that writes
script: [ "npm ci --prefer-offline" ]
build-api:
needs: [prepare] # does not wait for the verify stage
# ...
publish:
needs:
- job: build-api
artifacts: true # only the dotenv with the digest
- job: build-web
artifacts: true # dist/ really is needed
- job: quality
artifacts: false # ordering only
- job: test
artifacts: false
- job: security
artifacts: falseTotal estimate: from 19 min to around 12-13 on the critical path, and roughly half the billed compute thanks to the duplicated-pipeline fix. That last point is the most profitable of the four and the most overlooked, because it does not show up as slowness but as a bill: it is exactly the lesson from 04-04 that optimising starts by measuring where the time and the money go, not by guessing.
Solution 2.
review:
stage: deploy
needs: [build-web]
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes: [ "apps/web/**/*", "packages/shared/**/*" ] # only if it affects the web app
environment:
name: review/$CI_COMMIT_REF_SLUG
url: https://$CI_COMMIT_REF_SLUG.review.reservalia.example
on_stop: stop-review
auto_stop_in: 3 days
script:
- aws s3 sync apps/web/dist/ "s3://reservalia-review/$CI_COMMIT_REF_SLUG/" --delete
- |
curl -sS --request POST \
--header "PRIVATE-TOKEN: $MR_BOT_TOKEN" \
--data-urlencode "body=Preview ready: $CI_ENVIRONMENT_URL (commit $CI_COMMIT_SHORT_SHA)" \
"$CI_API_V4_URL/projects/$CI_PROJECT_ID/merge_requests/$CI_MERGE_REQUEST_IID/notes"
stop-review:
stage: deploy
needs: []
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
when: manual
environment: { name: review/$CI_COMMIT_REF_SLUG, action: stop }
variables: { GIT_STRATEGY: none }
script:
- aws s3 rm "s3://reservalia-review/$CI_COMMIT_REF_SLUG/" --recursiveWhat each line protects: changes avoids spinning up environments for changes that do not touch the web app; auto_stop_in caps the cost of abandoned branches; on_stop guarantees cleanup when the MR is closed; GIT_STRATEGY: none speeds up the teardown and reduces surface; needs: [] means the stop job depends on nothing and works even if the original pipeline failed.
On secrets: the review/* environment must use its own, limited credentials — a role that can only write to the reservalia-review/ prefix of a bucket with no real data — never those of staging or production. The production ones are marked as protected and their environment scope is production, so they do not even exist in this job. MR_BOT_TOKEN is a project token with the minimum permission to comment. And the general rule that applies just as in 04-03: a preview environment is built on the assumption that its content is public, because its URL is guessable and it carries no real authentication.
Solution 3. Four phases, each with value of its own — the approach of 05-04:
Phase 0, inventory and remove immediate risk (1-2 weeks). The shell runner on the server machine is the urgent problem: any job runs commands as the runner's user on the same host as the instance, so a .gitlab-ci.yml in an MR can read GitLab's database. That gets changed first, before any performance improvement: a new runner with the docker executor on a different machine, and the shell one switched off. In parallel, an inventory of variables not marked as protected — rotating any that could have been exposed — and of projects with a .gitlab-ci.yml.
Phase 1, upgrade GitLab (2-4 weeks of calendar time). 14.x is a long way behind: you have to climb through the required upgrade stops, not in one leap, and rehearse the backup restore on a test instance before touching the real one. What you gain: security, CI/CD components, improvements to rules and to environments. Main risk: the downtime window and long database migrations; mitigated by rehearsing on a copy and announcing the window.
Phase 2, only/except → rules (1 week, project by project). only/except still works but it cannot be combined with rules in the same job and it cannot express compound conditions. The translation is mechanical:
| Old | Modern |
|---|---|
only: [main] |
rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH |
only: [merge_requests] |
rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event" |
only: changes: [src/**] |
rules: - changes: [src/**] |
except: [tags] |
rules: - if: $CI_COMMIT_TAG when: never + a positive rule |
You do a trial project first and check that the set of pipelines triggered is the same before and after, which is the only verification that genuinely proves equivalence.
Phase 3, needs and shared cache (1-2 weeks). Add needs to move from stages to a graph, and configure an S3 cache shared between runners. What you gain is pipeline time. It comes last on purpose: it is the most visible phase and the least important, and putting it before phase 0 would mean optimising the speed of a system with an open security hole. That ordering — risk, then maintainability, then performance — is the same one that governed the increments of 05-04.
Conclusion
GitLab CI/CD is what happens when CI is designed inside the platform instead of alongside it. The .gitlab-ci.yml was born as pipeline as code, without the UI heritage Jenkins drags along; the model evolved from sequential phases to a graph with needs without breaking what came before; and the integration gives you for free things that cost effort in other chains: per-job registry credentials, test and security reports inside the merge request, environments with history and rollback, and review apps with expiry in a single line. In exchange you pay coupling — leaving means leaving everything — features tiered by subscription level, and considerable operational work if you self-host it.
Of everything covered, three things carry over to any other tool: the distinction cache versus artifact — if losing it breaks the pipeline, it is an artifact — is universal even when the name changes; marking deployment credentials as protected is the concrete defence against the malicious MR, and its absence is a real hole in many installations; and pinning includes to a version is the same rule as Jenkins' Shared Libraries and the SHA-pinned actions of 04-03.
The next tool attacks a different dimension. Where GitLab competes on breadth, CircleCI competes on depth in a single thing: pipeline speed. It brings the most explicit and controllable cache in the module, a packaged reuse system — orbs — and, above all, test splitting by historical timings, which is the most mature answer to the sharding problem we have been carrying since 02-04. We will see the Reservalia pipeline for the third time and what that speed costs.
CI/CD Course: Continuous Integration and Deployment
Module 1: Introduction to CI/CD
- Basic CI/CD Concepts
- Benefits of CI/CD
- Popular CI/CD Tools
- The Course Project: the Application We Are Going to Automate
- DORA Metrics: How Software Delivery Is Measured
Module 2: Continuous Integration (CI)
- Introduction to Continuous Integration
- Setting Up a CI Environment
- Build Automation
- Automated Testing
- Code Quality and Static Analysis
- Artifacts, Versioning and Promotion
- Integration with Version Control
Module 3: Continuous Deployment (CD)
- Introduction to Continuous Deployment
- Deployment Automation
- Infrastructure as Code and Reproducible Environments
- Deployment Strategies
- Feature Flags, Rollback and Failure Recovery
- Monitoring and Feedback
Module 4: Advanced CI/CD Practices
- CI/CD Pipelines
- Dependency Management
- Security in CI/CD
- Scalability and Performance
- Pipeline as Code: Templates, Reuse and Testing the Pipeline
- Databases in the Pipeline: Safe Migrations
Module 5: Implementing CI/CD in Real Projects
- Case Study: Web Project
- Case Study: Mobile Application
- Case Study: Microservices
- Case Study: Modernising a Legacy Project
Module 6: Tools and Technologies
- Jenkins
- GitLab CI/CD
- CircleCI
- Travis CI
- Docker and Kubernetes
- GitHub Actions in Depth
- Comparison and Criteria for Choosing a Tool
Module 7: Practical Exercises
- Exercise 1: Setting Up a Basic Pipeline
- Exercise 2: Integrating Automated Tests
- Exercise 3: Deploying to a Production Environment
- Exercise 4: Monitoring and Feedback
- Exercise 5: Hardening the Pipeline with Security and Secrets
- Final Project: A Complete End-to-End Pipeline
