You already know that the state is the Terraform inventory. The next question is: where is that inventory stored? By default, on your own computer. But that causes problems as soon as you work in a team. In this subchapter, we’ll see the difference between local and remote state, and how to configure the classic S3 + DynamoDB backend, one of the most important practices in professional Terraform.

Local State: The Starting Point

By default, Terraform saves the state in a terraform.tfstate file in the project folder on your computer. This is called local state.

To learn and experiment on your own, it’s fine. But it has serious problems when things get serious:

Problem 1: No Collaboration

If the state is on your laptop, your teammates don’t have it. Each person would have their own version of the state, which contradict each other. Impossible to work as a team.

Problem 2: Risk of Loss

If your laptop breaks, gets lost, or you accidentally delete the folder, you lose the state. And you already know (subchapter 11.2) how serious it is to lose the state: Terraform “forgets” everything it manages.

Problem 3: No Locking

If two people run apply at the same time on the same infrastructure, they could corrupt the state or overwrite each other’s changes. Local state has no way to prevent this.

Problem 4: Security

A file with sensitive data (subchapter 11.2) on your laptop, unencrypted, is a risk.

Remote State: The Professional Solution

Remote state saves the tfstate in a central and shared location (in the cloud), instead of on your computer. This solves all the previous problems:

  • Collaboration: the whole team reads and writes the same central state.
  • Security: it is stored encrypted and protected.
  • Durability: it’s not lost if your laptop fails.
  • Locking: prevents two people from modifying at the same time (we’ll see this soon).

The configuration of where the state is stored is called the backend. There are several types of remote backend; the most classic in AWS is S3 + DynamoDB.

The Classic Backend: S3 + DynamoDB

This combination uses two services you already know from Part II:

Service Role in the backend
S3 (Chapter 5) Stores the state file (encrypted, versioned, durable)
DynamoDB (Chapter 8) Manages locking so there aren’t two apply at the same time

Analogy:

  • S3 is the shared safe where the inventory (the state) is stored, accessible to the whole team, secure and backed up.
  • DynamoDB is the “occupied/free” system of a bathroom: when someone is using the state (running apply), they put up the “occupied” sign (a lock) so no one else enters until they’re done.

Why S3 is Ideal for State

  • Durable (remember the “eleven nines” from Chapter 5): almost impossible to lose it.
  • Encrypted at rest: protects sensitive data.
  • Versioned (subchapter 5.3): keeps the history of the state, so you can recover a previous version if something goes wrong.
  • Shared: the whole team accesses the same file.

Why DynamoDB for Locking

DynamoDB manages a “lock”: before modifying the state, Terraform puts a lock in a DynamoDB table. While that lock exists, no one else can run apply. When it finishes, it releases it. This prevents corruption from simultaneous access. We’ll see locking in depth in Chapter 20.

Note: Recent versions of Terraform/S3 allow managing locking directly in S3 without DynamoDB. But the S3 + DynamoDB pattern is still the most well-known and the one you’ll see in most projects and documentation.

How It’s Configured (Overview)

The backend configuration is declared in the terraform block:

terraform {
  backend "s3" {
    bucket         = "my-company-terraform-state"
    key            = "production/network/terraform.tfstate"
    region         = "eu-west-1"
    dynamodb_table = "terraform-locks"
    encrypt        = true
  }
}
  • bucket: the S3 bucket where the state is stored.
  • key: the “path” inside the bucket (organizes states for different projects/environments).
  • dynamodb_table: the table for locking.
  • encrypt = true: encrypts the state.

After writing this, you run terraform init and Terraform configures the backend. We’ll see the details step by step in Chapter 20.

The Chicken and Egg Problem: to store the state in S3, you need an S3 bucket… but that bucket is also infrastructure. The usual solution: create the bucket and table once (by hand or with a small Terraform using local state) and then use them as the backend for everything else. We’ll see this in Chapter 20.

Local vs Remote: Comparison Table

Local State Remote State (S3 + DynamoDB)
Where it lives Your computer In the cloud, shared
Team collaboration No Yes
Risk of loss High Low (durable, versioned)
Locking (prevents conflicts) No Yes (DynamoDB)
Encryption / security Manual Yes
When to use Learning, solo testing Any serious or team project

What You Should Remember

  • By default, the state is local (on your computer): fine for solo learning, but not for teams or production.
  • Remote state stores it in a central and shared place (a backend), solving collaboration, security, durability, and locking.
  • The classic backend in AWS is S3 + DynamoDB: S3 stores the state (encrypted, versioned, durable) and DynamoDB manages locking (prevents two simultaneous apply).
  • It’s configured in the terraform { backend "s3" { ... } } block and activated with terraform init.
  • For any serious or team project, use remote state. We’ll see it in detail in Chapter 20.

In the last subchapter of this chapter, we’ll review the essential Terraform commands you’ll use daily: init, plan, apply, destroy, fmt, and validate.

Cloud, AWS & Terraform — From Zero to Expert

Chapter 1 · What is cloud computing

Chapter 2 · The cloud market and major providers

Chapter 3 · Regions, availability zones and edge

Chapter 4 · Compute: EC2

Chapter 5 · Storage: S3

Chapter 6 · Networking: VPC

Chapter 7 · Identity and access: IAM

Chapter 8 · Managed databases

Chapter 9 · Why Infrastructure as Code

Chapter 10 · HCL: the Terraform language

Chapter 11 · Providers and state

Chapter 12 · Your first real infrastructure in Terraform

Chapter 13 · Load balancing and auto scaling

Chapter 14 · Serverless with Lambda

Chapter 15 · Messaging and events

Chapter 16 · Content delivery and DNS

Chapter 17 · Containers on AWS

Chapter 18 · Modules: reuse and composition

Chapter 19 · Workspaces and environment management

Chapter 20 · Remote backends and locking

Chapter 21 · Infrastructure testing

Chapter 22 · Terraform in CI/CD

Chapter 23 · Defense in depth

Chapter 24 · Observability: logs, metrics and traces

Chapter 25 · Cost optimization

Chapter 26 · High availability and disaster recovery

Chapter 27 · AWS Well-Architected Framework

Chapter 28 · Serverless architectures at scale

Chapter 29 · Data platforms on AWS

Chapter 30 · Multi-account and landing zones

Chapter 31 · Platform Engineering and Internal Developer Platform

Chapter 32 · Relevant AWS certifications

Chapter 33 · Projects to consolidate what you've learned

Chapter 34 · Resources and community

© Copyright 2024. All rights reserved