Module 5 closed by saying that principles are not negotiable and decisions are made by looking at the context. This module flips the focus: until now the tool was a means to an end — almost always GitHub Actions — and from here on it is the subject. We start with Jenkins for one practical reason and one didactic one. The practical one: it is, by a wide margin, what you are most likely to find already installed when you join a company more than ten years old; nobody picks Jenkins today for a greenfield project anywhere near as often as they inherit it. The didactic one: Jenkins predates almost everything this course takes for granted — containers, pipeline as code, ephemeral runners, federated identity — and watching it solve those problems after the fact, with grafts onto a 2005 design, explains why GitLab CI, CircleCI and GitHub Actions were designed exactly the way they were. We are going to look at its architecture, its original sin, the Reservalia pipeline translated into a complete declarative Jenkinsfile, how it materialises the cross-cutting themes you already know, and what it really costs to operate.

1. Where Jenkins comes from and what that explains about its design
2. Architecture: controller, agents, executors and labels
3. The original sin: freestyle jobs configured through the UI
4. The Reservalia pipeline in a declarative Jenkinsfile
5. Scripted pipeline: when you really need Groovy
6. Ephemeral agents: Docker and Kubernetes
7. Credentials and secrets
8. Shared Libraries: reuse in Jenkins
9. Multibranch pipelines and organisation
10. The plugin ecosystem: the greatest strength and the greatest risk
11. Real operation: upgrades, backups and JCasC
12. When Jenkins is the right choice and when it is a millstone
13. Common Mistakes and Tips
14. Exercises
15. Conclusion

Jenkins was born in 2005 as Hudson, inside Sun Microsystems, written by Kohsuke Kawaguchi. In 2011, after the trademark dispute that followed Oracle's acquisition of Sun, the community forked the project and named it Jenkins. That lineage is not trivia: it explains almost every one of its design decisions.

In 2005 the world of software builds was different in four specific ways:

Hence the most useful characterisation of Jenkins: it is not a CI/CD tool, it is a general-purpose automation engine with a plugin community on top of it. It can do literally anything — build firmware, orchestrate tests against hardware, move files over SFTP — and that is simultaneously its superpower and the reason two Jenkins installations at two companies look nothing alike. In GitHub Actions or GitLab CI, if you know how to use it at one company you know how to use it at another; in Jenkins, not necessarily.

It is free software, written in Java, that you install yourself. There is no official "Jenkins SaaS": there are commercial offerings built on top (CloudBees), but the baseline model is you operate the server. That single point governs everything else and will come back in section 11 and in 06-07.

The Jenkins vocabulary translated into the course's:

Three consequences worth understanding properly:

The controller must not run builds. By default it ships with executors on the controller itself, and that is the first thing to set to zero. A build running on the controller has access to the filesystem where the encrypted credentials, the configuration of every job and the full history live; a malicious — or merely careless — Jenkinsfile can read JENKINS_HOME. On top of that, a heavy build leaves the interface unresponsive for everyone. Rule: zero executors on the controller, always.

Labels are the assignment mechanism. Each agent declares labels (linux, docker, macos, arm64, gpu) and the pipeline asks for one: agent { label 'linux && docker' }. It is the conceptual equivalent of runs-on with self-hosted runner labels in GitHub Actions (which you will see in 06-06), and it accepts boolean expressions.

The persistent workspace is the biggest mental difference compared with modern tools. In GitHub Actions each job starts on a clean machine; in Jenkins with static agents, the previous job's workspace is still there. That makes builds faster (no need to re-clone everything) and less reproducible: generated files that survive, node_modules from another branch, an old dist/ that makes a test pass when it should fail. It is the "false green" of 04-04 with a different cause. That is why cleanWs() exists and why section 6 on ephemeral agents matters so much.

For its first ten years, the normal way to use Jenkins was the freestyle job: you go to the web UI, click "New item", fill in a form with the repository, tick some checkboxes, type a shell script into a <textarea> and save. The configuration is serialised into a config.xml inside JENKINS_HOME, on the server, outside the project's repository.

The problems, all the ones 04-05 listed and then some:

• It is not versioned with the code. There is no diff, no review in a PR, no git blame when the pipeline starts failing. The answer to "who changed this?" is the Jenkins audit log, if the plugin happens to be installed.
• It cannot branch. If one branch needs an extra step, there is no way to express it: the job is one for all branches, or the job gets duplicated.
• It cannot be reviewed. A change to the pipeline is applied in production the moment you press "Save".
• It cannot be reproduced. Migrating the installation to another machine means copying the whole of JENKINS_HOME and praying.
• It gets copied and pasted between jobs. Forty jobs with the same slightly different shell block is the natural state of an installation with a few years on it.

Pipeline as Code (04-05) was the correction, and it arrived with the Pipeline plugin in 2016: the pipeline is written in a Jenkinsfile at the root of the repository, travels with the code, branches with it and is reviewed in the PR. It is exactly the idea GitLab CI and Travis were born with, and that Jenkins had to retrofit. Even today you will see installations with hundreds of freestyle jobs; migrating to Jenkinsfile is one of the most common modernisation jobs, and it looks a lot like the incremental approach of 05-04.

This is the module's central device: the same pipeline, six times. Reservalia's ci.yml has the stages prepare → quality / test (matrix with sharding) → build → security → publish (image to ECR by digest). Translated into declarative Jenkins:

Block by block, with an explicit comparison against what you already know:

1. agent none at pipeline level forces every stage to declare where it runs. It is the correct practice: if you set a global agent, the pipeline occupies an executor for its entire life, even while it waits at the approval input — which can be hours. It is a classic resource-consumption mistake with no equivalent in GitHub Actions, where each job requests its own runner.
2. options gathers what in GitHub Actions is spread across timeout-minutes, concurrency and the repository's retention settings. disableConcurrentBuilds(abortPrevious: true) is the cancel-in-progress of 04-04. buildDiscarder is the retention policy of 02-06, and here it is essential: without it, the controller's disk fills up, and a Jenkins with a full disk does not start.
3. environment defines variables for the whole pipeline; it can also be declared inside a stage. It accepts credentials('id') as a source, which is a convenient shortcut with a risk we will see in section 7.
4. checkout scm clones the revision that triggered the build; with skipDefaultCheckout() in options, you control where that happens. Without that option, every stage with an agent does its own checkout automatically, which is sometimes what you want and sometimes multiplies your clones by six.
5. stash/unstash is the passing of data between stages from 04-01: it compresses files on the controller and retrieves them on another agent. And here is the first real limit: the stash travels through the controller, so an 800 MB stash of node_modules saturates the controller's network and disk. Practical rule: stash for code and small artifacts; for anything large, external storage (S3, Nexus, Artifactory) or rebuild.
6. parallel with nested stages is the fan-out of 04-01. Each branch can have its own agent, its own post and its own when. By default, if one branch fails the rest are aborted; parallel accepts failFast true to make that explicit.
7. matrix has existed in the declarative syntax since 2019 and works like the matrix of 02-04, with axes, and accepts excludes. Here it implements sharding across four partitions. Honest downside: it is more verbose than its YAML equivalent and does not support dynamic matrices generated by an earlier stage without dropping down to script.
8. junit publishes results in JUnit XML format and is what feeds the test view, the failure history and the flaky detection of 02-04. It goes in post { always } because results must be published even when the build fails, which is precisely when they matter.
9. archiveArtifacts with fingerprint: true stores the artifact against the build and records its hash, so Jenkins can answer "in which builds and in which jobs was this exact file used?". It is the traceability of 02-06 implemented in the server itself. Careful: archived artifacts live on the controller's disk, so artifactNumToKeepStr is not optional.
10. when { branch 'main' } is the stage conditional. It supports changeRequest(), changeset (equivalent to the paths filters of 02-07), expression { } with Groovy, allOf/anyOf/not, and beforeAgent true to evaluate the condition before reserving an agent, which saves executors.
11. withCredentials injects the secret only inside the block, which is the correct pattern (section 7).
12. script { } is the escape hatch into scripted pipeline: inside it you can write arbitrary Groovy. Use it sparingly; every script block is a piece of pipeline that is no longer declarative and that nobody can read at a glance.
13. input is the manual approval, the conceptual equivalent of Environments with reviewers from 03-02. And here there is an important design difference: in GitHub Actions or GitLab, a job waiting for approval consumes no runner; in Jenkins, if the input sits inside a stage with an agent, it holds the executor. That is why this pipeline's input lives in a stage with no agent, and why it carries a timeout: an input without a timeout can sit waiting for months. submitter restricts who may approve.
14. post with always, success, failure, unstable, changed, fixed, regression, aborted, cleanup. fixed and regression are especially useful for notifications: they fire only on transitions, not on every build, which is what stops the Slack channel becoming noise nobody reads (03-06).

What this example reveals by comparison: Jenkins expresses the same thing as ci.yml, but the dependency graph is implicit. In GitHub Actions jobs relate to each other through needs and the engine works out the order; in Jenkins declarative, stages run in sequence and parallelism is explicit and nested. It is a tree model, not a graph model. For 90% of pipelines it makes no difference; for a complex fan-in of the 04-01 kind, you feel it.

Before declarative there was the scripted pipeline: plain Groovy inside node { }, with no imposed structure.

When you need it: matrices computed at run time, loops over a list that comes from an API, complex conditional logic, or building the whole pipeline programmatically. When you do not: almost always. Declarative covers the normal case, it can be validated (jenkins-cli declarative-linter), it reads without knowing Groovy and it is what the visualisation tools understand.

Two concrete warnings if you do end up writing scripted. First, the code runs on the controller, not on the agent: only what goes inside sh runs on the agent. A heavy Groovy loop burns controller CPU and affects everybody. Second, Jenkins executes that Groovy with CPS (Continuation Passing Style) so it can serialise the state and survive restarts, and that breaks things that work fine in ordinary Groovy: .each {} with awkward closures, non-serialisable variables such as Matcher or InputStream crossing an sh, and error messages of the java.io.NotSerializableException kind that tell you nothing useful. It is one of the most frustrating experiences in Jenkins and a genuine reason to stay in declarative.

The contaminated agent problem is the one that produces the most false builds in older installations: a static machine where hand-installed Node versions, caches from three projects, a full /tmp and an environment variable somebody set in 2019 have all piled up. The build passes on one agent and fails on another, and nobody knows why.

The modern answer: make the agent not exist before the build or after it.

The Kubernetes plugin is today the recommended way to operate Jenkins at scale: the controller lives in the cluster, each build creates a Pod with the containers it needs, and the Pod dies when it finishes. Direct benefits: real isolation, autoscaling for free (the cluster does it), zero configuration drift, and cost proportional to usage. It is the same idea as the ephemeral runners you will see in 06-05 and in 06-06 with ARC, and it is what brings Jenkins closest to how modern tools behave.

Honest downside: you now have to operate Jenkins and a Kubernetes cluster. If you did not already have Kubernetes, this simplifies nothing. And the fine detail remains: the podTemplate has to be sized (a badly set requests either fills the cluster or leaves builds pending), the cache no longer persists between builds by definition, and starting a Pod adds tens of seconds to the queue time that 04-04 taught you to measure separately from execution time.

Jenkins keeps credentials in its own store, encrypted in JENKINS_HOME with a key that is also in JENKINS_HOME — which means anyone who can read that directory can decrypt them, and that is why the controller must not run builds. Types: secret text, username/password, file, SSH key, certificate, and whatever plugins add (AWS credentials, GitHub tokens…).

1. withCredentials narrows the scope: the variable exists only inside the block. The alternative — environment { REG_PASS = credentials('ecr-bot') } — exposes the secret to the whole pipeline, including stages that do not need it, and contradicts the least privilege of 04-03.
2. set +x turns off the shell trace. Jenkins automatically masks credential values that appear literally in the log (it replaces them with ****), but that masking is fragile by design: it does not cover transformed values. If you do echo $TOKEN | base64, if the secret gets split across two lines, or if a script prints it JSON-encoded, the masking does not recognise it and it appears in the clear. This is exactly the lesson 06-04 will tell with the Travis incident: a secret that reaches a log is a burned secret, and masking is a safety net, not a guarantee.

On federated identity: Jenkins has no native OIDC towards cloud providers in the way GitHub Actions does (03-02). The options are plugins that issue OIDC tokens, the instance role of the machine hosting the agent (if it runs on AWS), or IRSA if it runs on EKS. It is achievable, but it is a setup you build yourself, not a box you tick; and that specific contrast — long-lived credentials by default versus federated identity by default — is one of the genuinely weighty arguments in favour of platform-native tools.

This is the equivalent of the reusable workflows and composite actions of 04-05. A Shared Library is a Git repository with a fixed structure:

1. @Library('reservalia@v3') can point at a branch, a tag or a SHA. Pointing at main means that one commit in the library changes the behaviour of every pipeline in the company at once, without anyone having merged anything in their own repositories. It is exactly the central-repository risk of 04-05 and the reason for pinning by SHA from 04-03, with a sharper edge here because the blast radius is the entire organisation. Pin to a tag at minimum; to a SHA if the environment is sensitive.

Two more warnings. A library marked as a trusted global library in the Jenkins configuration runs Groovy without the security sandbox: whoever can merge into that repository runs arbitrary code on the controller. Treat it as production code with CODEOWNERS (02-07). And libraries are hard to test: JenkinsPipelineUnit exists for unit-testing pipelines, but it is a world of its own and few people maintain it; in practice most people test on a branch with a toy repository, which is the "test the pipeline" approach of 04-05 in its artisanal version.

The Multibranch Pipeline is the job type that makes Jenkins look like modern tools: you point it at a repository, Jenkins discovers every branch that contains a Jenkinsfile, creates one job per branch automatically, runs it on each push and deletes the job when the branch disappears. With the right plugin, it does the same with pull requests (change requests), which enables the per-PR previews of 02-07.

One level up is the Organization Folder (GitHub Organization, GitLab Group, Bitbucket Team): you point it at the whole organisation and it discovers repositories with a Jenkinsfile. It is what saves you creating jobs by hand when you have eighty repositories.

An operational detail that bites: branch discovery happens by polling or by webhook. With polling, Jenkins asks the Git server every X minutes, which adds latency and, with hundreds of repositories, punishes the Git server. With a webhook, the reaction is immediate but the controller has to be reachable from the Git server, which in an on-premise installation behind the firewall is not always trivial. It is a problem that integrated SaaS tools simply do not have.

Jenkins has more than eighteen hundred plugins. Practically anything you might want to integrate already has one: a serial console for firmware, an ancient ticketing system, a hardware signing device, a mainframe. That is the strongest argument in favour of Jenkins in 2026, and it is no small thing: when your odd integration does not exist in any other tool, in Jenkins it does.

The price, with the same honesty:

• Uneven quality. Alongside plugins maintained by companies there are plugins with a single maintainer who has not touched the code in five years, and no visible signal in the interface to tell you so before you install it.
• Dependencies between plugins and with the core. Updating one plugin may require updating Jenkins, which may break three others. The upgrade turns into a dependency-resolution problem solved by hand.
• CVE surface. Jenkins security advisories are frequent and many affect plugins, not the core. An unmaintained plugin with a known vulnerability is a decision: you remove it or you live with it. This is the dependency management of 04-02 applied to your own CI system.
• Shared global state. Plugins share the controller's JVM: one with a memory leak degrades the whole of Jenkins.

Minimum hygiene, which is real and recurring work: an inventory of installed plugins with a justification for why each one is there, uninstalling what is not used (most installations have dozens), subscribing to the Jenkins project's security advisories, and a monthly upgrade window with a test environment where you rehearse it first.

The question almost never asked when choosing Jenkins is who operates it. It is not rhetorical: it is a fraction of a person with a first name and a surname.

On backups: JENKINS_HOME is everything — configuration, credentials, history, plugins, archived artifacts. And the rule from 03-05 applies here just the same: a backup that has never been restored is not a backup, it is a hope. Rehearse the restore on a clean machine at least once a quarter.

The modern answer to "the configuration lives in config.xml on a server" is JCasC (Jenkins Configuration as Code): describing Jenkins' own configuration in versioned YAML.

1. The policy from section 2, now written down and reviewable in a PR.
2. The whole agent cloud described in the same file: rebuilding the controller from scratch stops being archaeology.
3. Secrets are not written into the YAML: they are interpolated from the environment, normally from an external secrets manager, exactly as established in 04-03.

With JCasC plus a Jenkins deployed by Helm on Kubernetes, the controller becomes replaceable, which is precisely what 03-03 asked of any infrastructure. A realistic warning: migrating an existing installation to JCasC is neither automatic nor quick, and there are plugins that do not expose their configuration to JCasC. It is done in increments, exactly with the approach of 05-04.

With no zealotry in either direction. It is still the right choice when:

• You have to run on hardware you control: a device lab, firmware test benches, machines with licences tied to one specific box, your own GPUs, unusual architectures.
• The code cannot leave the corporate network, by regulation or contract, and a SaaS self-hosted runner is not acceptable because the control plane is still outside.
• You need an integration that exists only as a Jenkins plugin, and writing it from scratch in another tool costs more than operating Jenkins.
• You have several version control systems at once — Git, and still Subversion or Perforce somewhere: Jenkins is agnostic and platform-native tools are not.
• You already have a healthy installation, with Jenkinsfile, JCasC and ephemeral agents, and the team knows how to operate it. Migrating takes months and the benefit may not be worth it (06-07 puts numbers on this).

It is a millstone when:

• Nobody really operates it. An ownerless Jenkins degrades on its own: unpatched plugins, full disk, dead agents, freestyle jobs nobody understands.
• All your code is on GitHub or GitLab and you have no special constraint: you are paying the cost of operating a server and synchronising identities to obtain what the platform already gives you built in.
• The team is small. The fraction of a person that operation consumes is an enormous percentage of a team of five, and that time comes straight out of the product. It is the argument Diego would make at Reservalia, and he would be right.
• The installation is from the freestyle era and nobody has started the migration: every change is frightening and the pipeline is the exact opposite of the executable documentation 04-05 argues for.

Running builds on the controller. It is mistake number one and it has security and availability consequences at the same time. Set numExecutors: 0 and never revisit it.

Not setting buildDiscarder. The controller's disk fills up with artifacts and history, and a Jenkins with a full disk neither starts nor lets you restore anything. Retention policy in every pipeline, from the first one.

Trusting secret masking. It masks literal matches. Transformed, chopped up or encoded, the secret comes out in the clear. Treat any secret that could have touched a log as compromised and rotate it.

An input inside a stage with an agent. It holds an executor for hours or days. Put it in a stage with no agent and always with a timeout.

Overusing script { }. Every block turns a piece of declarative pipeline into Groovy that only its author understands, and exposes you to CPS serialisation errors. If a pipeline has five script blocks, the logic belongs in a script in the repository (scripts/*.sh, npm run) invoked with sh — the portability lesson 06-07 will develop.

Stashing enormous directories. The stash passes through the controller. node_modules in a stash is an effective way to take Jenkins down for everybody.

Pointing @Library at main. A merge in the library changes the pipeline for the whole organisation without anyone asking for it. Pin to a tag or a SHA.

Dirty workspaces. Without ephemeral agents, use cleanWs() in post and be suspicious of any failure that "only happens on agent 3".

A tip for migrating from freestyle: do not convert the forty jobs at once. Pick the most frequently run one, write its Jenkinsfile, have it run in parallel with the freestyle job for two weeks comparing results, and only then switch the old one off. It is the parallel pipeline of 05-04 applied here.

Exercise 1. The Reservalia pipeline in Jenkins takes 22 minutes, but actual execution adds up to 9. Investigating, you find: agent { label 'linux' } at pipeline level, the approval stage with input inside that scope, a 640 MB stash (it includes node_modules), and four stages doing their own checkout scm because skipDefaultCheckout() is not set. Diagnose each cause and write the corrected Jenkinsfile in its relevant parts.

Exercise 2. Marta wants the team to stop repeating the preparation and publishing blocks across Reservalia's six repositories. Design the Shared Library: file structure, the contents of vars/publishImage.groovy with parameters, how it is versioned, how it is referenced from the Jenkinsfiles, who may modify it and how a change is tested without breaking all six repositories at once.

Exercise 3. Diego asks: "We already have GitHub Actions working. Why would we set up Jenkins?". At the same time, the company has just signed a healthcare customer that requires the source code not to leave their network and the integration tests to run against a physical device connected over USB in their data centre. Write the technical answer: what the options are, what each one costs and what you recommend.

Solution 1.

The expected result is not "9 minutes" — there is always queueing and container start-up — but it does remove the ~13 minutes of waiting and transferring that were not work. And a methodological observation worth more than the specific fix: the diagnosis came from separating queue time, transfer time and execution time, exactly the instrumentation of 04-04. Without that separation, the conclusion would have been "Jenkins is slow".

Solution 2. Structure:

Versioning: semantic tags (v1, v1.3.0) and the Jenkinsfiles reference @Library('reservalia@v1'), never @main. A breaking change goes up to v2 and repositories migrate when they can, which is the same contract as 04-05.

Governance: CODEOWNERS with the platform team as mandatory reviewer; the library marked as trusted runs code without a sandbox on the controller, so the review bar is that of production code with privileged access.

How a change is tested without breaking all six repositories: three layers. (a) Unit tests with JenkinsPipelineUnit for the pure logic — which command is generated with which parameters — run in the library's own CI. (b) A toy repository, reservalia-lib-sandbox, whose Jenkinsfile points at @main and exercises every vars on each library commit: it is the canary. (c) Progressive rollout of the new tag: v2 is published, one real repository adopts it for a week, and only then is it propagated. It is the progressive migration of 04-05 and it also explains why pinning to main is so dangerous: it removes layers (b) and (c) in one go.

Solution 3. Diego's question is the right one and the answer is not "set up Jenkins".

Options on the table:

Recommendation: start with A, because most "the code does not leave our network" requirements are satisfied by a self-hosted runner, and you must read the contract before designing anything — that is the first action, not an afterthought. If reading it reveals that the restriction covers the control plane, then D: a minimal Jenkins, deployed with JCasC, zero executors on the controller, an agent holding the device, scope limited to the hardware test, and the rest of the pipeline untouched.

The direct answer to Diego, which is the one that matters: you do not set up Jenkins because it is better, you set it up because there is a specific constraint — the physical device and the customer's network — that the current tool cannot satisfy, and you set up the minimum that resolves that constraint. Adding Jenkins "just in case" or "to have everything in one place" would add half a day of operation a month for the platform team without solving any problem that exists today. And the contract needs numbers on it: if the healthcare customer bills less than the annual cost of operating the installation, the conversation due is the 05-04 one with the business, not a technical decision.

Jenkins is a general-purpose automation engine from 2005 to which, one by one, every concept this course takes for granted was added: pipeline as code arrived in 2016 with the Jenkinsfile, ephemeral agents with the Docker and Kubernetes plugins, reproducible configuration with JCasC. Every graft works, and none of them feels native; that is the difference between a design that had it from the start and one that took it on later. In exchange it offers two things no other tool in the module matches: total control over where and how execution happens, and a plugin ecosystem where your odd integration probably already exists.

What to take away: the controller never runs builds; declarative covers 90% of cases and scripted is the exception, not the starting point; ephemeral agents solve the contaminated environment problem and are today the sensible way to operate it; withCredentials narrows the secret's scope and log masking is not a guarantee; Shared Libraries are the reusable workflows of 04-05 with the same blast-radius risk if pinned to a moving branch; and operating Jenkins is a real fraction of a person that has to be named before choosing it, not discovered afterwards.

The next tool solves the opposite problem. If Jenkins is an engine that integrates everything but brings nothing, GitLab CI/CD is a platform that brings everything integrated: repository, CI, container registry, environments, security analysis and issues in a single product, with the pipeline defined in a .gitlab-ci.yml that was born as pipeline as code and with an execution model that evolved from fixed phases to a graph. We will see what that integration buys, what it costs, and the Reservalia pipeline translated a second time.