We close Part VII with Chapter 31: Platform Engineering and Internal Developer Platform, a very current discipline that represents the peak of infrastructure maturity. The idea: instead of each team of developers having to learn and configure all the infrastructure on their own, a specialized team builds an internal platform that provides it ready-made, easily and securely. We start with the fundamental concept of this discipline: golden paths, built on Terraform.

The problem: each team reinventing the wheel

Imagine a company with many development teams. Each one needs infrastructure: networks, databases, servers, pipelines... Without a common platform, each team has to:

Each team, on their own:
   - learn Terraform, AWS, networks, security in depth
   - configure their infrastructure from scratch
   - make security decisions (Will I do it right?)
   - keep everything up to date
   → lots of duplicated work, inconsistent decisions, security errors

This has serious problems: a lot of work is duplicated, each team does things differently (inconsistency), and developers—experts in their application, not necessarily in infrastructure—can make security or design mistakes. Plus, they get distracted from what really adds value: their product.

Analogy: it's as if in a company each employee had to build their own computer, install the operating system, and configure the network before being able to work. It would be a huge waste of time, everyone would do it differently, and many would do it wrong. The logical thing is for an IT team to prepare standard, ready-to-use, and secure computers, and for each employee to focus on their work. Platform Engineering applies that same idea to cloud infrastructure.

What is Platform Engineering

Platform Engineering is the discipline of building an internal platform that makes it easy for development teams to create and manage their infrastructure easily, quickly, and securely, without having to be experts in all the details. A platform team builds "tools and paths" that other teams use.

The goal: for developers to be able to self-serve the infrastructure they need (remember cloud self-service, subchapter 1.2) in a simple and secure way, focusing on their product instead of infrastructure details.

What is a golden path

The central piece of Platform Engineering is the golden path: a recommended, easy, and well-designed way to do something, which the platform team prepares for developers to follow. It's the "prepared path" that leads to a good result effortlessly and without risks.

   Golden path = the prepared and recommended path:
   "Need a database? Follow THIS golden path:
    fill in these few details and you'll get a database
    properly configured, secure, and following the standards."

A golden path does not force (teams could go off it if they have a special need), but it's so easy and good that most are happy to follow it. It makes the right thing also the most convenient.

Analogy: a golden path is like a well-marked and paved trail on a mountain. You could climb cross-country (do it all yourself), but it's hard, risky, and easy to get lost. The golden trail is prepared, safe, and takes you straight to the top with minimal effort. Most take it because it's simply the best way to get there. The platform team "paves" those trails for common needs.

How golden paths are built: on Terraform

Here it connects with everything you know. Infrastructure golden paths are typically built on Terraform and modules (Chapter 18). The platform team creates well-designed, secure, and compliant modules that encapsulate best practices, and offers them to developers as golden paths:

Platform team creates expert Terraform modules:
   module "standard-database"  (secure, with backups, well configured)
   module "standard-web-app" (with load balancing, autoscaling, logs...)
        │
        ▼
Developers use them with just a few parameters
   → they get expert infrastructure without being experts

Remember modules (Chapter 18) and the idea of modules as an internal product (which we'll see in subchapter 31.4): the platform team acts as a product team whose "customers" are the other developers, and whose "product" is these golden paths. All the discipline of modules, versioning (subchapter 18.4), and testing (Chapter 21) you learned applies here to create quality golden paths.

Why it matters: speed, security, and consistency

Golden paths on Terraform provide three major benefits at once:

  • Speed: developers get their infrastructure in minutes, without having to learn everything or configure it from scratch.
  • Security and best practices "out of the box": since the path was designed by the expert team, the infrastructure comes out secure and well done by default, without the developer having to be a security expert.
  • Consistency: all teams using the same golden path get coherent infrastructure, making it easier to maintain and govern everything.
Without golden paths: each team slow, inconsistent, at risk of errors
With golden paths: everyone fast, secure, and consistent "out of the box"

Real-world example: a company with 15 development teams sets up a Platform Engineering team. This team creates golden paths on Terraform: for example, a "standard web service" module that, by just specifying the name and a few parameters, deploys an application with load balancing, autoscaling, logs, security, and backups already configured according to the company's best practices. Before, setting all this up took each team days (and sometimes they did it wrong). Now, a developer has it in minutes, without being an infrastructure expert, and it comes out secure and consistent with the rest. The 15 teams move faster, make fewer mistakes, and the company maintains control. Developers focus on their product, which is what adds value.

What you should remember

  • Without a common platform, each team reinvents the wheel with infrastructure: duplicated work, inconsistency, and risk of security errors, distracting from their product. Like if each employee had to build their own computer.
  • Platform Engineering is the discipline of building an internal platform that makes it easy for teams to create and manage their infrastructure easily, quickly, and securely, without being experts in everything (self-service).
  • A golden path is the recommended, easy, and well-designed way to do something, prepared by the platform team. It doesn't force, but it's so good and convenient that most follow it: it makes the right thing also the easiest. Like a marked and paved trail to the top.
  • Infrastructure golden paths are built on Terraform and modules (Ch. 18): the expert team creates secure and compliant modules that developers use with a few parameters.
  • They provide speed (infrastructure in minutes), security and best practices out of the box, and consistency across teams.

In the next subchapter, we'll look at an AWS tool to offer this type of pre-approved resources in a self-service way: the Service Catalog.

Cloud, AWS & Terraform — From Zero to Expert

Chapter 1 · What is cloud computing

Chapter 2 · The cloud market and major providers

Chapter 3 · Regions, availability zones and edge

Chapter 4 · Compute: EC2

Chapter 5 · Storage: S3

Chapter 6 · Networking: VPC

Chapter 7 · Identity and access: IAM

Chapter 8 · Managed databases

Chapter 9 · Why Infrastructure as Code

Chapter 10 · HCL: the Terraform language

Chapter 11 · Providers and state

Chapter 12 · Your first real infrastructure in Terraform

Chapter 13 · Load balancing and auto scaling

Chapter 14 · Serverless with Lambda

Chapter 15 · Messaging and events

Chapter 16 · Content delivery and DNS

Chapter 17 · Containers on AWS

Chapter 18 · Modules: reuse and composition

Chapter 19 · Workspaces and environment management

Chapter 20 · Remote backends and locking

Chapter 21 · Infrastructure testing

Chapter 22 · Terraform in CI/CD

Chapter 23 · Defense in depth

Chapter 24 · Observability: logs, metrics and traces

Chapter 25 · Cost optimization

Chapter 26 · High availability and disaster recovery

Chapter 27 · AWS Well-Architected Framework

Chapter 28 · Serverless architectures at scale

Chapter 29 · Data platforms on AWS

Chapter 30 · Multi-account and landing zones

Chapter 31 · Platform Engineering and Internal Developer Platform

Chapter 32 · Relevant AWS certifications

Chapter 33 · Projects to consolidate what you've learned

Chapter 34 · Resources and community

© Copyright 2024. All rights reserved