The previous lesson ended with an uncomfortable idea: the frontend is an old client you cannot force to update. There the consequence was minor — the user only has to reload the page — and the rollback cost under a minute. Now we take that same property to its extreme. Reservalia Pro is the React Native mobile application the professional uses to manage their schedule from their phone: they check the day's slots between one customer and the next, confirm or cancel appointments and take payment. It is used by 190 of the 340 paying businesses, and its pipeline breaks four rules we have been taking for granted for seven modules. The user decides when they update, and many never do. A store reviews every version for hours or days before publishing it, so between the merge and the user there is a third party you do not control. You cannot roll back a version already published: the only thing that exists is publishing another one. And dozens of client versions coexist at once against the same API, not two for five minutes. In this lesson we will see which parts of the pipeline survive intact, what has to be reinvented — code signing, staged distribution, over-the-air updates, API compatibility — and how the DORA metrics are read when "deploying" stops meaning "reaching the user".

Contents

  1. What this context has that the web did not
  2. Reservalia Pro: product, repository and what gets reused
  3. Code signing and credential management
  4. The mobile CI pipeline
  5. Versioning and automatic numbering
  6. Distribution channels and staged rollout
  7. The release Fastfile, annotated
  8. Over-the-air updates and their limits
  9. Compatibility with old clients
  10. Crash reporting as a feedback loop
  11. The DORA metrics when deploying is not arriving
  12. Case summary
  13. Common Mistakes and Tips
  14. Exercises
  15. Conclusion

  1. What this context has that the web did not

apps/web (05-01) Reservalia Pro (mobile)
Who decides the update The server: reload and it is done The user, or their auto-update setting
Intermediaries None App Store and Google Play, with human review
Merge → user time 12 minutes From 4 hours to 5 days, then weeks of adoption
Rollback Repoint index.html: <1 min Does not exist: only publishing a new version
Live client versions 1 or 2 Dozens, some a year old
Artifact A folder of files Signed APK/AAB and IPA with cryptographic material
Runner cost Linux, cheap macOS for iOS, ~10× the price per minute
Failure signal 5xx errors on your server Crash reports from other people's devices

The signing row introduces the most operational problem and the one that blocks the most teams, so it comes first. The rollback row is the one that changes the design most: if you cannot undo, you have to be able to stop, and from that comes the staged rollout of section 6. And the live versions row is the one that forces the discipline of 04-06 onto the API contract.

What does not change deserves to be stated with the same clarity, because it is the bulk of it: version control and trunk-based development (02-07), npm ci with a lockfile (04-02), static analysis (02-05), an immutable artifact built once and promoted (02-06), secrets by reference and least privilege (04-03), pipeline as code with reusable actions (04-05). A mobile pipeline is not another world: it is the same one with three new pieces and one hard constraint.

  1. Reservalia Pro: product, repository and what gets reused

Reservalia Pro lives in its own repository, reservalia/pro-mobile, not in the monorepo. The reason is purely practical and worth understanding because it recurs in the next lesson: the app consumes packages/shared — the types and the availability logic — but its build cycle has nothing to do with the API's (native tooling, macOS runners, 18-minute builds). Keeping it in the monorepo would force every API PR to drag the mobile configuration along, and the paths from 02-07 would become fragile. The connection is made by publishing @reservalia/shared as a versioned package in the private registry from 04-02.

pro-mobile/
  src/                      # React Native (TypeScript)
  android/                  # Gradle project
  ios/                      # Xcode project
  fastlane/
    Fastfile                # the "lanes": beta, release, screenshots
    Appfile                 # app identifiers and accounts
    Matchfile               # match configuration (certificates)
  .github/workflows/
    ci-mobile.yml           # on every PR
    release-mobile.yml      # on every v* tag

Two flows, just as in the monorepo: a fast one on every PR and a publishing one triggered by a tag. The difference with ci.yml/cd.yml is that here the second one does not end in production, it ends at a store's front door.

  1. Code signing and credential management

Every mobile binary is signed, and the signature is not an optional security detail: without it the operating system installs nothing. The two platforms solve the same thing in different ways:

Android iOS
Material A keystore (.jks) with a key and its password A distribution certificate + provisioning profile
Who issues it You do Apple, tied to your developer account
Expires No (but losing it is fatal) Yes: certificate at 1 year, profile at 1 year
If you lose it You can never publish updates to that app again (unless Play-managed signing is in place) It is regenerated; annoying but recoverable
Rotation Practically never Annual, and it always catches you by surprise

The first row of "if you lose it" is the one to internalise: the Android keystore is the project's most critical asset. Reservalia keeps one copy in the company's secrets manager, another encrypted offline, and also uses Google Play-managed signing as a safety net. That is a business decision, not a technical one.

Fastlane match solves the iOS problem elegantly: it stores the certificates and profiles encrypted in a private git repository, and each machine — Nuria's laptop or the CI runner — downloads and decrypts them with a single password. Instead of N machines with different material, there is one versioned source of truth.

# fastlane/Matchfile
git_url("git@github.com:reservalia/ios-certificates.git")   # 1 · private, separate repo
storage_mode("git")
type("appstore")                                            # 2
readonly(true)                                              # 3
  1. A repository separate from the code: whoever contributes to the app does not need access to the signing material.
  2. In CI only the appstore type is used; development profiles stay on the laptops.
  3. readonly(true) in CI is essential: the runner downloads what is there, it never regenerates certificates. A CI with write permission can revoke the whole team's certificates in one badly written job, and that incident leaves everybody unable to publish.

In the workflow, the material arrives via secrets and never touches the repository in the clear:

  build-ios:
    runs-on: macos-14                                   # 1
    timeout-minutes: 45
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/prepare-node            # reused from 04-05

      - name: Prepare temporary keychain                # 2
        run: |
          security create-keychain -p "$KEYCHAIN_PASS" build.keychain
          security default-keychain -s build.keychain
          security unlock-keychain -p "$KEYCHAIN_PASS" build.keychain
        env:
          KEYCHAIN_PASS: ${{ secrets.KEYCHAIN_PASS }}

      - name: Download certificates with match
        run: bundle exec fastlane match appstore --readonly
        env:
          MATCH_PASSWORD: ${{ secrets.MATCH_PASSWORD }}          # 3
          MATCH_GIT_BASIC_AUTHORIZATION: ${{ secrets.MATCH_GIT_TOKEN }}

      - name: Build and upload to TestFlight
        run: bundle exec fastlane beta
        env:
          APP_STORE_CONNECT_API_KEY: ${{ secrets.ASC_API_KEY }}  # 4
  1. macos-14 is mandatory for building iOS: Xcode only exists on macOS. On GitHub Actions, a minute of macOS costs ten times a minute of Linux, so this job is what dominates the app's CI bill. Hence the policy in section 4: iOS builds do not run on every PR.
  2. A temporary keychain per job, created and destroyed within the same run. On a hosted runner this is almost ceremonial because the machine is destroyed; on a self-hosted runner (02-02) it is essential, because otherwise the certificates stay on the machine for the next job, which may belong to another repository.
  3. MATCH_PASSWORD is the master key to all the signing material. It lives in the secrets manager of the release environment (03-02), with reviewers, and not in the repository secrets accessible to any workflow.
  4. Authentication via an App Store Connect API key, not with a human's username and password. It is the mobile equivalent of the OIDC from 03-02: a service credential, with scoped permissions and revocable without affecting any person.

  1. The mobile CI pipeline

On every pull request, and with the cost of the macOS runner very much in mind:

name: ci-mobile
on: { pull_request: { branches: [main] } }
concurrency: { group: ci-mobile-${{ github.ref }}, cancel-in-progress: true }

jobs:
  static:                                # 1 · Linux, 2 min
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/prepare-node
      - run: npm run lint && npx tsc --noEmit
      - run: npm test -- --coverage

  android-debug:                         # 2 · Linux, 7 min
    runs-on: ubuntu-22.04
    needs: [static]
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/prepare-node
      - uses: actions/setup-java@v4
        with: { distribution: temurin, java-version: '17', cache: gradle }
      - run: cd android && ./gradlew assembleDebug           # 3 · no release signing
      - uses: actions/upload-artifact@v4
        with: { name: apk-debug-${{ github.sha }}, path: android/app/build/outputs/apk/debug/*.apk }

  instrumented:                          # 4 · emulator, 11 min
    runs-on: ubuntu-22.04
    needs: [android-debug]
    steps:
      - uses: actions/checkout@v4
      - uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: 33
          script: cd android && ./gradlew connectedDebugAndroidTest

  ios-debug:                             # 5 · macOS, only under a condition
    if: contains(github.event.pull_request.labels.*.name, 'ios') || github.event.pull_request.base.ref == 'release'
    runs-on: macos-14
    needs: [static]
    steps:
      - uses: actions/checkout@v4
      - run: xcodebuild -workspace ios/Pro.xcworkspace -scheme Pro -sdk iphonesimulator build
  1. The cheap job goes first and blocks the rest. It is the fail fast from 04-01, and here it has a direct financial benefit: a type error caught in two minutes of Linux avoids spending forty of macOS.
  2. The debug APK is built on every PR and uploaded as an artifact. Nuria installs it on a real device to review the change: it is the mobile equivalent of the per-PR preview environment from 02-07. An automatic comment with a download link and a QR code completes the circuit.
  3. assembleDebug uses the debug signature, which does not need the release keystore. That lets this job run on any PR, even one from an external contributor's branch, without exposing sensitive material. It is the same least-privilege logic as 04-03.
  4. The instrumented tests run on an emulator inside the runner. They are slow and more fragile than unit tests, so the pyramid from 02-04 is applied strictly: about eight of them, on the flows that make money. Flaky ones go into quarantine under the same policy as always.
  5. iOS is only built on demand, via an ios label on the PR or if the PR targets the release branch. The justification is cost: building iOS on Reservalia's 60 monthly PRs would cost around 25 hours of macOS; with this condition it comes down to five. The risk accepted is real — a change that breaks only iOS is detected later — and it is bounded by setting up a scheduled nightly build.

Diego: "How much is this costing us a month?" The answer, with numbers: 60 PRs × 20 min of Linux, plus 12 iOS builds × 40 min of macOS. Around 20 h of Linux and 8 h of macOS, and half the bill comes from those 8 hours. Without the condition in point 5, it would be three times as much.

  1. Versioning and automatic numbering

Three numbers coexist and confusing them causes rejected releases:

Concept Android iOS What it is for
Visible version versionName (1.14.0) CFBundleShortVersionString What the user sees; SemVer from 02-06
Internal number versionCode (integer) CFBundleVersion (build) Orders the uploads; must always increase
Identity applicationId bundleIdentifier Identifies the app; never changes

The hard rule: the internal number has to be strictly increasing and cannot be reused. If you upload build 412 to TestFlight and then want to upload that same version again with a fix, it has to be 413 even if the versionName is still 1.14.0. Deriving it by hand guarantees that one day somebody will repeat it and the upload will be rejected after forty minutes of building.

# fastlane/Fastfile (excerpt)
lane :bump_version do
  version = sh("node -p \"require('./package.json').version\"").strip   # 1
  build   = latest_testflight_build_number(version: version) + 1        # 2
  increment_version_number(version_number: version)
  increment_build_number(build_number: build)
  UI.message("Publishing #{version} (build #{build})")
end
  1. The visible version comes from package.json, which in turn is set by semantic-release from the conventional commits of 02-06. The app thus inherits the versioning of the rest of the system, with no parallel mechanism.
  2. The build number is queried from the store and incremented. It is the only source that cannot get out of sync, better than github.run_number: if you migrate CI, the counter resets and uploads start being rejected. Asking whoever holds the truth is more robust than keeping count yourself.

  1. Distribution channels and staged rollout

Here is the most important conceptual substitution of the lesson. The canary from 03-04 directed 10% of traffic to the new version through the ALB weighting, and if the metrics got worse it was withdrawn in seconds. On mobile there is no load balancer and there is no withdrawal: the version is already installed on the user's device. What does exist is control over how many people are offered it.

flowchart LR
    B["Signed build"] --> I["Internal<br/>the team · minutes"]
    I --> A["Alpha / TestFlight<br/>~30 volunteer businesses"]
    A --> C["Open beta<br/>~200 users"]
    C --> R1["Production 5%"]
    R1 --> R2["20%"]
    R2 --> R3["50%"]
    R3 --> R4["100%"]
    R1 -.->|"crash-free < 99.5%"| H["Halt the rollout"]
    R2 -.->|"crash-free < 99.5%"| H
    H --> F["Publish a fix version"]

The channels, with what they contribute and what they cost:

Channel Who gets in Store review What it is for
Internal Up to 100 email addresses; the team None or minimal Verify the signed binary starts up
Alpha / internal TestFlight Volunteer businesses (~30) Light Real usage, qualitative feedback
Open beta Anyone who signs up (~200) Yes Coverage of devices and OS versions
Staged production An increasing percentage Already passed Limit the damage of a failure

The staged rollout works like this: you publish to 5%, and the store offers the update only to that randomly chosen fraction of users. You watch the crash-free session rate for 24 hours, raise it to 20%, and so on to 100% over four or five days. Its two differences from the canary are the ones to be clear about:

  • Halting is not reverting. If at 20% you spot a failure, you can pause the rollout — nobody else will receive the version — but those who already installed it keep it. Google Play also allows a "halt", which withdraws the offer; iOS allows the phased release to be paused. In no case does the previous binary come back to the devices.
  • The only fix is forward. It is the roll-forward from 03-05, but here it is not one option out of two: it is the only one. And it brings a store review with it, which in an urgent case can be sped up by requesting an expedited review, with no guarantees.

Hence the operational conclusion: since you cannot revert, you have to detect earlier and with fewer users. 5% of 190 businesses is 9 or 10; enough for a start-up crash to show up in crash reporting within hours, and small enough that only 10 customers suffer it.

  1. The release Fastfile, annotated

# fastlane/Fastfile
default_platform(:android)

platform :android do
  desc "Upload to Play on the given track with a partial rollout"
  lane :publish do |options|
    bump_version                                               # 1

    gradle(task: "clean bundleRelease",                        # 2
           properties: {
             "android.injected.signing.store.file"     => ENV["KEYSTORE_PATH"],
             "android.injected.signing.store.password" => ENV["KEYSTORE_PASS"],
             "android.injected.signing.key.alias"      => ENV["KEY_ALIAS"],
             "android.injected.signing.key.password"   => ENV["KEY_PASS"],
           })

    upload_to_play_store(
      track: options[:track] || "internal",                    # 3
      rollout: options[:rollout] || "0.05",                    # 4
      release_status: "inProgress",
      skip_upload_screenshots: true,
      skip_upload_images: true,
      mapping_paths: ["android/app/build/outputs/mapping/release/mapping.txt"]  # 5
    )
  end
end
  1. The numbering is resolved first, so that a rejection for a repeated number happens before building rather than after.
  2. bundleRelease produces an AAB, not an APK. Google Play uses it to generate device-optimised APKs, which brings the user's download size down considerably. The signing credentials come in through environment variables that the workflow fills from secrets: they never appear in build.gradle, which is versioned.
  3. The track is a parameter, not a fixed value. The same lane serves internal, beta and production; what changes is how it is invoked. It is the same idea as the reusable workflow from 04-05.
  4. rollout: 0.05 starts at 5%. Subsequent increments do not upload the binary again: they are done from the console or with a lane that only changes the percentage. The artifact is the same — build once, deploy many times (02-06); what moves is the exposure.
  5. The mapping file is uploaded with the release. Without it, the crash reports from section 10 arrive obfuscated and are useless. It is the step most often forgotten and the one that costs most dearly at three in the morning.

In the workflow, this is triggered by a tag and with human approval:

  publish-android:
    if: startsWith(github.ref, 'refs/tags/v')
    environment: stores                        # mandatory reviewers, as in 03-02
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - run: bundle exec fastlane android publish track:production rollout:0.05
        env:
          KEYSTORE_PATH: ${{ runner.temp }}/release.jks
          KEYSTORE_PASS: ${{ secrets.KEYSTORE_PASS }}
          SUPPLY_JSON_KEY_DATA: ${{ secrets.PLAY_SERVICE_ACCOUNT }}

  1. Over-the-air updates and their limits

Since React Native runs a JavaScript bundle on top of a native shell, there is the possibility of replacing that bundle without going through the store: the app downloads the new version at start-up and applies it on the next launch. That is what CodePush or EAS Update do, and it gives back something resembling the continuous deployment we had on the web.

OTA update Store release
What can change Only JavaScript and assets Anything, including native code
Time to the user Minutes or hours Hours or days + adoption
Review None Yes
Reversion Yes: go back to the previous bundle No
Risk A broken bundle leaves the app unusable Lower, there is a review in between

The limits have to be stated plainly. Technical: if the change touches a native dependency, changes permissions or upgrades the React Native version, OTA will not do and you have to go through the store; and the OTA bundle has to be compatible with the native shell already installed, which forces it to be tied to a specific app version. Store policy: Apple's rules allow interpreted code to be updated as long as it does not change the app's main purpose or introduce functionality substantially different from what was reviewed; using OTA to skip the review is grounds for removal. Reservalia's practical reading:

  • OTA yes for urgent JavaScript fixes, copy changes, styling tweaks and turning a feature flag (03-05) that already shipped in the binary on or off.
  • OTA no for visible new functionality, permission changes, native dependencies or anything that alters what the store reviewed.
  • Always staged in OTA too, with the same percentage discipline, because a broken bundle distributed to 100% of users in ten minutes is worse than any store failure.

Marta: "OTA is the fire extinguisher, not the lift." It is used to put out a fire, not as the usual delivery route. A team that delivers everything by OTA ends up with an out-of-date store version and a growing gap between what was reviewed and what the user is running.

  1. Compatibility with old clients

This is the mobile equivalent of expand and contract (04-06), and it is the part that affects the API team even though they never touch the app. Reservalia's figures, taken thirty days after a release:

Pro version Active users
1.14.x (latest) 61%
1.13.x 22%
1.12.x 9%
1.9.x – 1.11.x 6%
Earlier than 1.9 2%

39% of users are running code that is months old. That turns any API change into a contract change with dozens of clients you cannot update. Four tactics, in order of preference:

(1) Additive, backward-compatible changes, whenever possible. A new field in the response breaks nobody; renaming one does. It is literally the pattern from 04-06 applied to JSON instead of to the schema: the new field is added, both coexist for months, and the old one is retired when usage drops to zero.

(2) Explicit API versioning when the change cannot be additive: /v1/appointments and /v2/appointments coexisting, with a written policy on how long each version is maintained — at Reservalia, twelve months from the publication of the next one.

(3) Contract tests in the API pipeline. The app publishes its expectation and the API verifies in CI that it does not break it. It is the technique the next lesson develops in depth for microservices; here the mechanism is enough: a set of contracts versioned by live app version, run as API tests.

(4) Minimum supported version, as a last resort. The API responds with a specific code and the app shows a blocking screen inviting the user to update:

// apps/api/src/plugins/client-version.ts
const MINIMUM = '1.9.0';

app.addHook('onRequest', async (req, reply) => {
  const v = req.headers['x-app-version'] as string | undefined;   // 1
  if (!v) return;                                                 // 2
  if (compareSemver(v, MINIMUM) < 0) {
    return reply.code(426).send({                                 // 3
      error: 'unsupported_version',
      minimum: MINIMUM,
      message: 'Update Reservalia Pro to carry on using the application.',
    });
  }
});
  1. The app sends its version on every request. Without that header there is no way of knowing who is calling you, and it is the data that feeds the table above.
  2. If the header is absent, nothing is blocked: it could be the web app, an internal script or an ancient version. Blocking on absence breaks legitimate integrations.
  3. 426 Upgrade Required with a body the app knows how to interpret. The blocking screen has to exist before it is needed: if version 1.8 does not know how to handle a 426, forcing it only produces incomprehensible errors. That is the classic trap of this technique, and the reason the 426 handling is implemented early even if it goes unused for years.

Forcing an update has a business cost: a professional mid-shift whose schedule you block is not a happy one. It is reserved for serious reasons — a vulnerability, an unavoidable incompatible change — and warned about beforehand with a non-blocking screen for weeks.

  1. Crash reporting as a feedback loop

In 03-06 the feedback loop was server observability: metrics, logs and traces from our own machines. Here the code runs on other people's devices, and the equivalent signal is crash reports. The governing metric is the crash-free session rate:

Metric Target for Reservalia Pro What it decides
Crash-free sessions ≥ 99.5% Whether the rollout continues or halts
Crash-free users ≥ 99.8% Whether a failure is widespread or hits an unusual device
ANR (Android) < 0.3% Interface freezes the user experiences as "it hung"
4xx/5xx errors per version Correlates a failure with an API change

Two requirements for this to work. First, upload the symbols with the release — Android's mapping.txt and iOS's dSYMs — or the reports arrive without function names. Second, tag the version and the stage: every report carries versionName, build and the active rollout percentage, so you can answer "did 1.14 bring this or was it already happening?". It is the equivalent of the deployment markers on the dashboard from 03-06.

The automation Reservalia does have: a scheduled job queries the crash-free session rate of the version being rolled out every hour and, if it drops below the threshold, pauses the rollout automatically and posts to Slack. It is the direct descendant of the metric-driven automatic rollback from 03-05, with the difference that here it reverts nothing: it stops the expansion.

  1. The DORA metrics when deploying is not arriving

Applying the definitions from 01-05 without translating them gives absurd results: "deployment" would come to mean "upload to the store" and the lead time would say five days without distinguishing where the time goes. The translation Reservalia uses:

Metric Definition on the API Definition on mobile Value
Deployment frequency Deployments to prod Releases published to the store 1 every 2 weeks
(auxiliary) builds to internal/beta 8 per week
Lead time Commit → prod Commit → available in the store 3.5 days
(auxiliary) commit → beta 4 hours
Change failure rate Deployments with an incident Releases with a drop in crash-free sessions or an urgent fix 12%
Time to restore Rollback: 9 min Fix published and adopted by 50% 2.5 days

Three readings. The first: a time to restore of 2.5 days is not a team failure, it is a property of the distribution channel, and that is why the effort shifts from "recover fast" to "do not publish the failure" — more beta, slower staging at the start, feature flags to switch things off without publishing. The second: the auxiliary metrics are the ones the team can genuinely improve; if commit→beta is four hours, the pipeline works, even if the store adds three days on top. The third: the 12% change failure rate is three times the API's 3.8%, and it has a structural explanation — each release accumulates two weeks of changes, against the small, frequent deployments of the API. The lesson from 01-02 is confirmed in reverse: large batches, more failures. And since here the batch cannot be reduced at will, it is offset with the beta and the staged rollout.

  1. Case summary

Context React Native, its own repository, 190 businesses, iOS + Android, fortnightly publishing
What still holds as-is Trunk-based (02-07), lockfile (04-02), static analysis (02-05), single artifact (02-06), secrets and least privilege (04-03), reusable actions (04-05)
Decision 1 Signing with match in a separate repository, readonly in CI and a temporary keychain per job
Decision 2 iOS built only with a label or on the release branch, plus a nightly build; saving ~20 h of macOS/month
Decision 3 Build number queried from the store, visible version inherited from semantic-release
Decision 4 Staging 5 → 20 → 50 → 100% with an automatic pause if crash-free sessions drop below 99.5%
Decision 5 OTA only for JS fixes and flags; never for new functionality
Decision 6 x-app-version header, contracts per live version and 426 implemented years before it was used
Effect on DORA Real lead time 3.5 days (auxiliary to beta: 4 h); CFR 12%; restore 2.5 days
What you take to any project When you cannot revert, the investment moves to detecting earlier and exposing fewer people

Common Mistakes and Tips

Mistake 1: storing the keystore in the repository, even "temporarily" or on a private branch. Anyone who clones the repo can sign as you. Mistake 2: giving match write permission in CI, which lets a job regenerate and revoke the whole team's certificates. Mistake 3: deriving the build number from a CI counter; when you change tooling it resets and uploads start being rejected.

Mistake 4: building iOS on every PR without needing to: it is half the CI bill for a marginal benefit. Mistake 5: forgetting to upload the mapping or the dSYMs, which leaves the crash reports unreadable precisely when they are needed. Mistake 6: publishing to 100% in one go because "this version is small"; the staged rollout costs nothing and it is the only net there is.

Mistake 7: using OTA for new functionality and skipping the review, risking the app's removal. Mistake 8: implementing the minimum-version block without the old versions knowing how to interpret it, so the user sees an incomprehensible error instead of an invitation to update. Mistake 9: measuring lead time only up to the store and concluding the pipeline is slow when the time is taken by the review.

Tip 1: treat the keystore as the critical asset it is, with copies in two places and Play-managed signing as a net. Tip 2: publish the debug APK as a PR artifact with a link and a QR code: that is your preview environment. Tip 3: send x-app-version from day one even if you do not use it; without that data you will never know who you would break. Tip 4: write down the release procedure — who approves, what is checked at each stage, when it is paused — before the first urgent publication.

Exercises

Exercise 1

Reservalia Pro 1.14.0 is published on Tuesday at 5%. On Wednesday at 09:40, the crash-free session rate for that version is 97.1% against 99.7% for 1.13.4; the reports point to a crash when opening the schedule on Android 11 devices. Describe the complete response procedure, indicating what can and cannot be done, and compare each step with what the team would have done if the same failure had occurred in apps/api.

Exercise 2

The API team wants to rename the field startTime to startUtc in the GET /appointments response, consistent with the migration from 04-06. The live versions table for the app shows that 39% of users are running versions earlier than 1.14. Design the complete plan, with phases and criteria for moving on, and say at exactly what moment the old field can be retired.

Exercise 3

Diego proposes scrapping the open beta: "nobody reports anything through it, it is 200 users who never say a word, and it delays every release by two days". Evaluate the proposal with arguments for and against, and propose a reasoned decision along with what you would measure to validate it.

Solutions

Solution 1. The first thing is to accept the constraint: there is no rollback. The users who already installed 1.14.0 have it, and no action of yours replaces it with 1.13.4. What can be done, in order:

(1) Halt the rollout, within minutes. In Google Play the rollout is paused — and, if the failure is serious, a "halt" is used, which withdraws the offer from anyone who has not yet updated. Cost: zero. Effect: the damage is bounded to the ~10 businesses that already have it. (2) Contain with what is already deployed. If the new schedule view was behind a feature flag (03-05), it is switched off from the server and the failure disappears without publishing anything; this is why mobile features are wrapped in flags even though the app is not a service. And if the problem were on the server side — a new field the app does not tolerate — it can be fixed in the API in minutes. (3) Notify those affected. Ten businesses are identifiable: support calls them and offers an alternative (using the web app) for the duration. That is not a technical step, but in a paid product it is the one that decides whether the incident turns into a cancellation. (4) Fix forward. It is fixed, 1.14.1 is published and, if the impact justifies it, an expedited review is requested. If the failure is in JavaScript and does not touch native code, an OTA update arrives in hours rather than days: this is exactly the legitimate use case for the fire extinguisher. (5) Resume the rollout from 5%, not from where it left off, and with specific monitoring of Android 11. (6) A blameless post-mortem (03-05), with a concrete question: why did the 200-user beta not cover Android 11?

Comparison with the same failure in apps/api: step 1 would be rollback.yml returning the previous digest in four minutes, with all users recovered, not just future ones; step 3 would not exist because there would be no residual casualties; step 4 would stop being urgent, because the system is already healthy and can be fixed calmly. The underlying difference is a single one: on the API the recovery time depends on you; on mobile it depends on the distribution channel and on the user's willingness. That is why all the effort shifts left — beta, staged rollout, flags — and why the 2.5-day time to restore in the table in section 11 is a property of the medium and not a defect of the team.

Solution 2. The plan is expand and contract, with the peculiarity that the coexistence phase does not last a week but more than a year, because you do not control when the client updates.

Phase 1 — Expand (API). GET /appointments returns both fields, startTime and startUtc, with the same value. It is additive: no version of the app breaks, not even 1.8. It is deployed with the normal pipeline, with no ceremony. Phase 2 — Instrument. Add a metric to the API that counts, per app version, how many requests come from clients still consuming the old field. Since the server does not know which field the client reads, it is approximated with the x-app-version header: you record which versions are live and cross-reference that with the known fact of which app version started reading startUtc. Without this step, phase 4 is decided on intuition. Phase 3 — App. Version 1.15.0 of Reservalia Pro reads startUtc and stops using startTime; it is published with the usual staged rollout. Here the criterion for moving on is not time but adoption: you wait for 1.15 or higher to exceed 95% of active users, which at Reservalia is around four months. Phase 4 — Contract. Only when three conditions hold at once: the versions that read startTime are below 1% of active users; the published compatibility period has elapsed (twelve months from 1.15); and it has been verified that no other consumer — the web app, an internal script, a large customer's integration — uses the field.

The exact moment of retirement is therefore the maximum of those three conditions, and in practice it is determined by the third or the second, not the first. Two reinforcements that make the wait cheaper: (a) for the residual 1%, the 426 technique from section 9 lets you force an update for those specific users instead of keeping the field forever; and (b) while the coexistence lasts, a contract in the API pipeline for each live app version stops a future refactor removing startTime by accident. And one underlying observation: the cost of maintaining a duplicated field for a year is minuscule compared with breaking 39% of your users. The asymmetry of costs is what decides, not the elegance of the contract.

Solution 3. Diego's arguments are true but they measure the wrong thing. In favour of scrapping it: the beta adds two days to the lead time; spontaneous qualitative feedback is almost nil — people do not report, they uninstall; and maintaining release notes and a communication channel for 200 users costs somebody's time. Against: the value of the beta is not the feedback people write, it is the telemetry they generate without writing anything. Two hundred real users over two days produce sessions on dozens of device models and operating system versions that neither the team nor the emulator covers, and that is where the failures you cannot see in development show up: manufacturers with aggressive memory management, old Android versions, unusual screen sizes, accessibility settings turned on. The Android 11 failure from exercise 1 is literally the case a well-populated beta catches.

The reasoned decision: do not scrap it, change its purpose and measure it. Specifically: (1) stop expecting written reports and treat it as a telemetry phase, with an automatic criterion for moving on — X hours of accumulated usage, N distinct models, crash-free sessions ≥ 99.5% — so that if the coverage is reached in twelve hours, the release moves on in twelve hours rather than two days; (2) recruit the beta deliberately, seeking device diversity rather than numbers, because 60 users across 40 models are worth more than 200 across 6; (3) measure its real performance over six months with a concrete metric: how many failures were caught in beta that would not have got past the beta into production. If over six months the beta has caught three failures that would have reached paying users, two days of delay is more than justified — the cost of a production failure includes support, an urgent fix, an expedited review and the risk of losing a customer. If it has caught none, Diego is right and it goes. What is not defensible is deciding it on gut feeling in either direction: it is exactly the kind of question 01-05 taught you to answer with data.

Conclusion

Reservalia Pro has put the pipeline built in the previous modules to the test and confirmed two things. The first is how much survives: the branching model, the lockfile, static analysis, the single artifact built once and promoted, secrets management by reference with least privilege and reusable actions all work here just as they do on the API. The second is what breaks and why. Cryptographic material appears — keystore, certificates, profiles — that has to be safeguarded, rotated and lent to CI in read-only mode and in a temporary keychain. An intermediary appears with human review that turns a lead time of hours into one of days, and a real macOS runner cost that forces decisions about what gets built and when. Rollback disappears, and with it the mechanism that underpinned the peace of mind of module 3; in its place are the percentage-based staged rollout with automatic pausing, feature flags that switch things off without publishing, OTA as a fire extinguisher with technical and policy limits, and roll-forward as the only way out. And dozens of live client versions appear at once, which turns every API change into a backward-compatibility exercise lasting months: additive fields, coexisting versions, contracts per live version and a 426 implemented long before it was needed. In the end the DORA metrics had to be translated, separating what the team controls — commit to beta, four hours — from what the channel imposes — commit to user, three and a half days — and the operational conclusion was that when you cannot revert, all the investment shifts to detecting earlier and exposing fewer people.

Notice the pattern repeating: on the web there was one old client; here there are dozens. What holds the system up in both cases is an explicit contract and a long coexistence between versions. The next lesson, Case Study: Microservices, takes exactly that problem and multiplies it in another direction. Reservalia grows and splits into five services — appointments, businesses, notifications, payments, availability — with different teams deploying whenever they like. There is no longer one client app and one API, but a graph in which each service is at once an old client and a provider to another. There the contract stops being a version table watched by hand and becomes executable contract tests with an automatic deployment gate, the pipeline is multiplied by five, and decisions have to be made about what gets standardised and what is left to each team.

CI/CD Course: Continuous Integration and Deployment

Module 1: Introduction to CI/CD

Module 2: Continuous Integration (CI)

Module 3: Continuous Deployment (CD)

Module 4: Advanced CI/CD Practices

Module 5: Implementing CI/CD in Real Projects

Module 6: Tools and Technologies

Module 7: Practical Exercises

Module 8: Additional Resources

© Copyright 2026. All rights reserved