In the previous subchapter, when talking about the Saga pattern, we mentioned that we needed a way to orchestrate multi-step processes: direct the order, decide what to do if a step fails, wait, retry... When you have many Lambdas (Chapter 14) that must collaborate in a complex flow, coordinating them "by hand" becomes chaos. That's what AWS Step Functions is for: a service that allows you to orchestrate multi-step workflows in a visual, orderly, and reliable way. It's like having a conductor for your serverless functions.
The problem: coordinating many functions is a mess
Imagine a business process with several steps: validate an order, charge, reserve stock, schedule shipping, notify... Each step could be a Lambda. If you try to coordinate them by having one call the next directly, problems arise:
❌ "By hand" coordination (Lambda calls Lambda): - What if a step fails? How do I retry? - How do I know which step the process is at right now? - How do I handle steps that take time, waits, decisions (if this, do that)? - The flow is "hidden" inside the code, hard to see and change
This "coordination hidden in the code" is fragile, hard to follow, and hard to modify. You need to separate the flow logic (the order of the steps, what to do if they fail) from the logic of each step (what each Lambda does).
What is Step Functions
AWS Step Functions is a service to orchestrate workflows: you define a sequence of steps —with their order, decisions, retries, and error handling— and Step Functions executes and coordinates it for you. The flow is defined in a visual and declarative way, separate from the code of each step.
Step Functions executes a flow like this:
[Validate order] → Valid?
├─ yes → [Charge payment] → [Reserve stock] → [Ship] → [End]
└─ no → [Reject] → [End]
(with retries and error handling at each step)Analogy: Step Functions is like the conductor of an orchestra. Each musician (Lambda) knows how to play their instrument (do their task), but it's the conductor who sets the order, when each one comes in, what to do if someone makes a mistake, and keeps everyone coordinated so it sounds like a symphony and not chaos. Without a conductor, 50 musicians playing at once without coordination would be a disaster. Step Functions conducts your functions so they collaborate in an orderly flow.
Another way to see it: it's like a flowchart that actually runs. You draw the process (this step, then this, if this happens do that) and Step Functions carries it out.
What Step Functions gives you
- Visual and clear flow
The workflow looks like a diagram: at a glance you understand the entire process (what steps there are, in what order, what decisions are made). This makes the process easy to understand and modify, instead of being buried in the code.
- Integrated error handling and retries
Step Functions automatically manages what to do when a step fails: it can retry (with increasing waits), jump to an error handling step, or execute compensations (just what the Saga pattern from subchapter 28.2 needs!). You don't have to code all that logic by hand.
- State tracking
Step Functions remembers which step each execution is at and keeps the history. You can see exactly where a process is, what steps were completed, and where it failed if something went wrong. This gives huge visibility (complements the observability from Chapter 24).
- Steps that wait or take time
It naturally handles flows that take time (minutes, hours, or even days) or that must wait for something (a human approval, an external response). Something very hard to do with just Lambdas (which have a maximum execution time, remember subchapter 14.5).
Step Functions and the Saga pattern
Step Functions is the ideal tool to implement an orchestration Saga (subchapter 28.2): you define the process steps and, for each one, what compensation to execute if something fails later. Step Functions takes care of executing the compensations in order if a step fails, maintaining consistency, all in a visual and controlled way.
Saga with Step Functions:
[Charge] → [Reserve] → [Ship] ✗ fails
└─ Step Functions automatically executes compensations:
[Release reservation] → [Refund] → consistent stateReal world example: a company processes loan applications, a process with many steps: validate data, check credit history (call to an external service that takes time), calculate risk, wait for human approval if the amount is high, and finally disburse or reject. They implement it with Step Functions: the complete flow looks like a clear diagram, steps that take time or wait (like human approval, which can take days) are handled without issue, and if any step fails, there are defined retries and error handling. The team can see at what point each application is in real time. What would have been fragile and unreadable to coordinate by hand with Lambdas, with Step Functions is a clear, robust, and easy-to-modify process.
When to use Step Functions
- When you have a multi-step process to coordinate (especially with Lambdas).
- When you need robust error handling, retries, or compensations (like in a Saga).
- When the flow has decisions ("if this, do that"), waits, or steps that take time.
- When you want to see and understand the process clearly, not hide it in the code.
💡 For a single simple task, a standalone Lambda is enough. Step Functions shines when there is a multi-step flow to coordinate.
What you should remember
- Coordinating many functions by having one call the next is fragile and hard to follow; it's better to separate the flow logic from the logic of each step.
- AWS Step Functions orchestrates multi-step workflows: you define the order, decisions, retries, and error handling in a visual and declarative way, and it executes and coordinates them. Like the conductor of an orchestra (or a flowchart that runs).
- It gives you: clear visual flow, integrated error handling and retries, state tracking (which step each execution is at), and support for steps that wait or take time (even days).
- It's the ideal tool to implement the Saga pattern by orchestration (executes compensations automatically if a step fails).
- Use it for multi-step processes with decisions, waits, or robust error handling. 💡 For a simple task, a standalone Lambda is enough.
In the last subchapter of the chapter, we'll see how to run serverless logic very close to users, at the edge of the network, with Lambda@Edge and CloudFront Functions.
Cloud, AWS & Terraform — From Zero to Expert
Chapter 1 · What is cloud computing
- 1.1 The traditional client-server model
- 1.2 Problems the cloud came to solve
- 1.3 On-premise vs cloud vs hybrid
- 1.4 The three service models: IaaS, PaaS, SaaS
- 1.5 The five pillars of cloud (according to NIST)
- 1.6 Real advantages: elasticity, pay-as-you-go, global availability
Chapter 2 · The cloud market and major providers
- 2.1 AWS, Azure and GCP: differences and market share
- 2.2 Why learn AWS first
- 2.3 Concepts that are universal among providers
Chapter 3 · Regions, availability zones and edge
- 3.1 What is an AWS region and how to choose it
- 3.2 Availability Zones: high availability by design
- 3.3 Edge locations and CloudFront
- 3.4 Latency, resilience and data sovereignty
Chapter 4 · Compute: EC2
- 4.1 Instances: types, families and when to choose each
- 4.2 AMIs, key pairs and Security Groups
- 4.3 Instance lifecycle
- 4.4 Elastic IPs and Placement Groups
- 4.5 Savings Plans vs Reserved vs On-Demand vs Spot
Chapter 5 · Storage: S3
- 5.1 Buckets, objects and keys
- 5.2 Storage classes (Standard, IA, Glacier…)
- 5.3 Versioning and object lifecycle
- 5.4 Bucket policies and ACLs
- 5.5 Static website hosting
Chapter 6 · Networking: VPC
- 6.1 What is a VPC and why you need it
- 6.2 Public and private subnets
- 6.3 Internet Gateway and NAT Gateway
- 6.4 Route Tables and Network ACLs
- 6.5 VPC Peering and endpoints
Chapter 7 · Identity and access: IAM
- 7.1 Users, groups, roles and policies
- 7.2 The principle of least privilege
- 7.3 Identity-based vs resource-based policies
- 7.4 MFA and temporary credentials (STS)
- 7.5 IAM security best practices
Chapter 8 · Managed databases
- 8.1 RDS: engines, Multi-AZ and read replicas
- 8.2 Aurora and its advantages over vanilla RDS
- 8.3 DynamoDB: key-value / document model
- 8.4 ElastiCache for in-memory cache
- 8.5 When to use each type of database
Chapter 9 · Why Infrastructure as Code
- 9.1 Problems with manual provisioning
- 9.2 Declarative vs imperative IaC
- 9.3 Terraform vs CloudFormation vs Pulumi vs CDK
- 9.4 The plan → apply → destroy cycle
Chapter 10 · HCL: the Terraform language
- 10.1 Resource, variable, output, locals blocks
- 10.2 Data types: string, number, bool, list, map, object
- 10.3 Expressions, references and built-in functions
- 10.4 Conditionals and loops (count, for_each, for)
Chapter 11 · Providers and state
- 11.1 How the AWS provider works
- 11.2 The terraform.tfstate file and its importance
- 11.3 Local state vs remote state (S3 + DynamoDB)
- 11.4 Essential commands: init, plan, apply, destroy, fmt, validate
Chapter 12 · Your first real infrastructure in Terraform
- 12.1 Create a VPC with subnets from scratch
- 12.2 Launch a public EC2 instance
- 12.3 Associate a Security Group and an Elastic IP
- 12.4 Outputs and references between resources
- 12.5 Team workflow: PR review of plans
Chapter 13 · Load balancing and auto scaling
- 13.1 Application Load Balancer vs Network Load Balancer
- 13.2 Target Groups, listeners and rules
- 13.3 Auto Scaling Groups: policies and metrics
- 13.4 Warm pools and lifecycle hooks
Chapter 14 · Serverless with Lambda
- 14.1 The Lambda execution model
- 14.2 Triggers: API Gateway, S3, DynamoDB Streams, SQS
- 14.3 Dependency management and layers
- 14.4 Cold starts and strategies to reduce them
- 14.5 Limits and anti-patterns
Chapter 15 · Messaging and events
- 15.1 SQS: standard vs FIFO queues, DLQ
- 15.2 SNS: topics, subscriptions, fan-out
- 15.3 EventBridge: event buses and rules
- 15.4 Patterns: pub/sub, decoupling, saga
Chapter 16 · Content delivery and DNS
- 16.1 Route 53: record types and routing policies
- 16.2 CloudFront: distributions, caches and origins
- 16.3 ACM: free SSL/TLS certificates
- 16.4 WAF integrated with CloudFront
Chapter 17 · Containers on AWS
- 17.1 Docker: quick review of key concepts
- 17.2 ECR: private image registry
- 17.3 ECS: task definitions, services, Fargate vs EC2
- 17.4 EKS: when Kubernetes and when not
Chapter 18 · Modules: reuse and composition
- 18.1 Anatomy of a Terraform module
- 18.2 Input variables, outputs and dependencies
- 18.3 Local modules vs Terraform Registry modules
- 18.4 Module versioning with Git tags
- 18.5 Design of generic vs domain-specific modules
Chapter 19 · Workspaces and environment management
- 19.1 Terraform workspaces: use cases and limitations
- 19.2 Directory strategy per environment (dev/stg/prod)
- 19.3 Terragrunt: DRY for environment configurations
- 19.4 Environment variables and .tfvars files
Chapter 20 · Remote backends and locking
- 20.1 Configure S3 + DynamoDB as backend
- 20.2 State locking: avoiding team corruption
- 20.3 State migration between backends
- 20.4 terraform import: bring existing resources into state
Chapter 21 · Infrastructure testing
- 21.1 Terraform validate and fmt in CI
- 21.2 Checkov and tfsec: static security analysis
- 21.3 Terratest: integration tests in Go
- 21.4 Contract testing between modules
Chapter 22 · Terraform in CI/CD
- 22.1 Basic pipeline: lint → plan → apply in GitHub Actions
- 22.2 Atlantis: GitOps for Terraform
- 22.3 Terraform Cloud / HCP Terraform
- 22.4 Drift detection and automatic reconciliation
Chapter 23 · Defense in depth
- 23.1 AWS Organizations and Service Control Policies
- 23.2 AWS Config: continuous compliance
- 23.3 GuardDuty: threat detection
- 23.4 Security Hub: centralized view
- 23.5 KMS: key management and rotation
- 23.6 Secrets Manager vs Parameter Store
Chapter 24 · Observability: logs, metrics and traces
- 24.1 CloudWatch Logs, metrics and alarms
- 24.2 CloudWatch Dashboards and Contributor Insights
- 24.3 X-Ray: distributed tracing
- 24.4 OpenTelemetry on AWS
- 24.5 Managed Grafana and Managed Prometheus
Chapter 25 · Cost optimization
- 25.1 AWS Cost Explorer and budgets with alerts
- 25.2 Trusted Advisor and Compute Optimizer
- 25.3 Rightsizing: how to detect overprovisioning
- 25.4 Savings Plans vs Reserved Instances: strategic decision
- 25.5 FinOps: culture and processes to control spending
Chapter 26 · High availability and disaster recovery
- 26.1 RTO and RPO: defining objectives
- 26.2 Strategies: backup/restore, pilot light, warm standby, multi-site
- 26.3 Route 53 health checks and automatic failover
- 26.4 AWS Backup: centralized backup policy
Chapter 27 · AWS Well-Architected Framework
- 27.1 The six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, sustainability
- 27.2 Well-Architected Tool: formal reviews
- 27.3 How to apply the framework in design decisions
Chapter 28 · Serverless architectures at scale
- 28.1 Event-driven architecture with Lambda + EventBridge
- 28.2 Saga pattern for distributed transactions
- 28.3 Step Functions: orchestration of complex workflows
- 28.4 Lambda@Edge and CloudFront Functions
Chapter 29 · Data platforms on AWS
- 29.1 Data Lake with S3, Glue and Athena
- 29.2 Kinesis Data Streams and Firehose for streaming
- 29.3 Redshift: data warehousing at scale
- 29.4 Lake Formation: data governance
Chapter 30 · Multi-account and landing zones
- 30.1 Why separate workloads into different accounts
- 30.2 AWS Control Tower and Account Factory
- 30.3 Centralized log and security management
- 30.4 Terraform at multi-account scale with shared modules
Chapter 31 · Platform Engineering and Internal Developer Platform
- 31.1 Golden paths and abstractions over Terraform
- 31.2 AWS Service Catalog
- 31.3 Backstage as a developer portal
- 31.4 Terraform modules as internal product
Chapter 32 · Relevant AWS certifications
- 32.1 Cloud Practitioner: is it worth it?
- 32.2 Solutions Architect Associate → Professional
- 32.3 DevOps Engineer Professional
- 32.4 Specialty: Security, Database, Networking
- 32.5 HashiCorp Terraform Associate
Chapter 33 · Projects to consolidate what you've learned
- 33.1 Project 1: serverless blog (S3 + CloudFront + Lambda + DynamoDB)
- 33.2 Project 2: REST API with ECS Fargate + RDS + ALB
- 33.3 Project 3: data platform with Glue + Athena + Redshift
- 33.4 Project 4: multi-account landing zone with Terraform and Control Tower
