We closed the previous module with MercadoFresco ready to go: a secured account, a chosen region, a console we know our way around and a CLI configured with the mercadofresco-dev profile. All of that was scaffolding. Now the building work starts. In this lesson we bring up MercadoFresco's first real server in the cloud with Amazon EC2 (Elastic Compute Cloud), the service that provides virtual machines on demand and the piece much of what follows rests on.

EC2 is the service almost everyone enters AWS through, and also the one where the most money is thrown away through carelessness. That is why we are not going to take a stroll through the launch screen: we are going to understand what an instance really is, how to choose its size without guessing, what CPU credits are (they will be decisive for the Friday peak), how to automate the server's start-up with user data, and how an Auto Scaling group turns MercadoFresco's problem 1 —the Friday-evening outages— into a problem solved by configuration.

1. What an EC2 instance is and what problem it solves
2. AMI: the template an instance is born from
3. Instance families and types: how to read t3.micro
4. How to choose the size without guessing
5. Burstable instances and CPU credits
6. Purchase models: on demand, spot, reserved and Savings Plans
7. Instance lifecycle: stopping is not terminating
8. Accessing the instance: key pairs, SSH, Instance Connect and Session Manager
9. user data: letting the instance install itself
10. Instance metadata and IMDSv2
11. Creating the instance in the console, step by step
12. Creating the instance from the CLI with the tagging scheme
13. Launch templates and Auto Scaling: the answer to the Friday peak
14. Shutting everything down and deleting it so you do not spend

An EC2 instance is a virtual machine that runs on AWS's physical infrastructure. It has CPU, memory, network, a boot disk and a complete operating system: as far as your software is concerned it is indistinguishable from a real server. You can install whatever you like, open an SSH session, look at the logs and reboot it.

What makes it different from the server in MercadoFresco's office is not what the machine is, but how it is obtained and given back:

That last row is the conceptual key to the whole module: in AWS, creating and destroying are symmetrical, cheap operations. When Marta needs four storefront servers on Friday at 17:00 and only one on Saturday morning, that stops being a fantasy and turns into a single line of configuration.

A note on what EC2 does not cover in this lesson. Every instance lives inside a virtual network (VPC) and is protected by a security group, which is its firewall. Here we will use the default VPC and create a minimal security group without going into detail: networking is the whole of module 3 (03-01 VPC, 03-02 security groups and NACLs). The instance's disk is managed with EBS, which is lesson 02-02.

An AMI (Amazon Machine Image) is a frozen disk image: operating system, installed packages, configuration and files. When you launch an instance, AWS copies the AMI onto a new volume and boots the machine from it. The AMI is the template; the instance is the living copy.

There are four possible sources:

For MercadoFresco we will use Amazon Linux 2023: it is free, it is optimised for EC2, it ships with the AWS CLI v2 pre-installed and the Systems Manager agent already active (important for Session Manager, further down).

One detail that confuses people at first: an AMI's identifier is different in every region. The same Amazon Linux 2023 has one ami-0abc… in eu-west-1 and a different one in eu-central-1. That is why you never write it by hand in a script; you look it up. The robust way is to ask AWS's public parameter store:

Output (an example; yours will be different):

Breakdown of the command:

• aws ssm get-parameter: Systems Manager Parameter Store holds key/value pairs. AWS publishes the IDs of its AMIs there so that you do not have to hunt for them.
• --query 'Parameter.Value': JMESPath, just as we saw in 01-05, to keep only the value.
• --output text: no quotes, ready to drop straight into a shell variable.

Save it in a variable, because we will use it several times:

The instance type defines how much CPU, memory, network and disk the machine has. AWS offers hundreds of them, but the name is a readable code that you decode piece by piece:

Another example: m6g.large = family m (balanced), generation 6, g for Graviton (AWS's ARM processor), size large.

The families you need to know:

And the sizes, which as a rule double resources and price at every step:

This linearity has an important and counter-intuitive consequence: two large instances cost the same as one xlarge, but two instances survive the loss of an Availability Zone and one xlarge does not. It is the first argument for the multi-AZ design we saw in 01-03, and one of the reasons why we will prefer to scale horizontally (more machines) before scaling vertically (bigger machines).

The professional method is not to guess, it is to measure. For MercadoFresco:

1. Start from the figure you already know. One storefront instance handles 600 orders/hour. The Friday peak is 900 orders/hour. With a single machine, 33 % of the peak's orders are lost: those are the outages of problem 1.
2. Start small. In the cloud, changing size costs you a reboot. Starting big "just in case" costs money every hour, forever.
3. Measure with CloudWatch (lesson 05-01): CPU, memory (needs the agent), network and latency for at least one full week, including a Friday.
4. Apply the 40-60 % rule. If average CPU is below 40 %, the machine is oversized. If it stays above 70 % for long periods, it is undersized.
5. Check Compute Optimizer, the free AWS service that analyses your metrics and recommends a type and a size.

For our starting point we choose t3.micro for two reasons: it falls inside the Free Tier (750 hours a month for 12 months) and its burstable behaviour lets us explain CPU credits, which are exactly what trips up so many people during their first traffic spike.

Here is the concept that causes newcomers the most grief, and the one that matters most for MercadoFresco's Friday.

Instances in the T family do not give you the whole CPU all the time. They give you a baseline —a percentage of the core— and accumulate CPU credits while you consume less than that baseline. When you need more, you spend credits to reach 100 %. If you run out, the instance is throttled and drops abruptly to the baseline, even though the physical CPU is sitting idle.

One credit = one minute of one vCPU at 100 %.

Let us do MercadoFresco's real calculation. A t3.micro accrues 12 credits/hour, with a maximum of 288 (the equivalent of 24 hours of accrual). The Friday peak lasts from 17:00 to 21:00, four hours, and during that time the storefront would be at 100 % CPU on its 2 vCPUs:

In other words: the t3.micro lasts under three hours of the peak and then throttles to 10 % CPU. The storefront does not go down for lack of server, it goes down for lack of credits, and on the CloudWatch dashboard the CPU looks flat and low, which is very misleading if you do not know why.

There are two ways out, and it is worth understanding that only one of them is good:

• unlimited mode (enabled by default on T3): when the credits run out, AWS lets you carry on at 100 % and charges you extra for every surplus vCPU-hour. It avoids the outage, but the bill goes through the roof if the peak is a regular event. You can force standard mode so that it never overcharges, in exchange for accepting the throttling.
• Scale horizontally: add instances during the peak. This is the right answer, and we will see it at the end of this lesson with Auto Scaling.

Checking the credit balance (the CPUCreditBalance metric, in CloudWatch):

Rule of thumb. T instances are excellent for irregular workloads with long troughs (development, test environments, internal services). For a sustained, predictable load, an M works out cheaper and more predictable than a T in unlimited mode.

The same instance can cost four different prices depending on the commitment you take on. Here we only compare them so that you know they exist and when each one is used; the full economic analysis, with payback calculations, is lesson 11-05.

Three ideas to take away:

• Spot is not "cheap and worse": it is exactly the same machine, with the condition that AWS can take it back. It only suits workloads that tolerate being interrupted and retried.
• Reserved instances and Savings Plans are not machines, they are billing discounts applied automatically to consumption you already have.
• Do not buy commitments until you have three months of real data. Marta will not reserve anything until module 11.

An instance moves through well-defined states, and confusing two of them —stopped and terminated— is the most expensive and most irreversible mistake in EC2.

What happens exactly at each transition:

The four nuances to memorise:

1. Stopping deletes nothing. The root EBS volume still exists and still costs money (around 0.08 USD per GB and month with gp3). A stopped instance is almost free, but not quite.
2. Terminating is final. There is no recycle bin. The root volume is deleted unless you have disabled DeleteOnTermination. If you were keeping data there, it has gone.
3. The automatic public IP is lost when you stop. When you start again you get a different one. If you need a stable IP, use an Elastic IP (module 3) or, better, a DNS name (03-05).
4. The private IP is kept for as long as the instance exists.

For production, Marta will always enable termination protection:

To allow deletion again you have to run the command with --no-disable-api-termination. It is deliberate friction: two conscious steps instead of one irreversible click.

A key pair is a pair of cryptographic keys. AWS keeps the public one and places it inside the instance at start-up (in ~/.ssh/authorized_keys); you keep the private one in a .pem file. AWS does not keep a copy of the private key: if you lose it, there is no recovery possible through support.

Notes on the command:

• --key-type ed25519 is more modern and shorter than RSA; both work, but ed25519 is today's default choice (Windows with older instances may require rsa).
• The KeyMaterial output is shown only once. If you do not redirect it to a file, it is gone.

Connecting, once the instance is running:

The default user depends on the AMI: ec2-user on Amazon Linux, ubuntu on Ubuntu, admin on Debian. Logging in as root is disabled on purpose.

There are two alternatives that avoid handling .pem files, and they are worth knowing:

Session Manager (part of AWS Systems Manager) is the option MercadoFresco will use in production: the instance needs neither a public IP nor an open SSH port, because it is the one that starts the outbound connection towards AWS. Less attack surface, and a complete audit trail of who got in and what they typed.

The IAM permissions this requires are covered in lesson 04-01; the security groups that control port 22, in 03-02.

The user data field is a script the instance runs as root, exactly once, on its first start-up. It is the difference between "I have created a server" and "I have created a server that is already serving the storefront".

This is MercadoFresco's script. Save it as user-data-tienda.sh:

How it behaves and how to debug it:

• It runs only on the first start-up. If you stop and start the instance, it is not repeated.
• The complete output is left in /var/log/cloud-init-output.log inside the instance. When something "does not work and I do not know why", that file is the first place to look.
• It must be idempotent and non-interactive: no apt install without -y, no waiting for a keypress.
• Never put credentials in user data: anyone with access to the instance can read it with a curl to the metadata. For secrets you use Secrets Manager (lesson 04-03).

Every instance can ask itself who it is, by querying a special address that only answers from the inside: 169.254.169.254. That is where the instance metadata lives: its ID, its type, its AZ, its IPs, its tags (if you enable them) and —very importantly— the temporary credentials of the associated IAM role.

That last part explains why the security of this endpoint matters so much. Version 1 (IMDSv1) answered any GET, which meant an SSRF-type vulnerability in the web application could make the server leak its own credentials. IMDSv2 requires you to obtain a token first with a PUT, something a simple SSRF cannot do.

Paths that are useful day to day:

MercadoFresco policy: always require IMDSv2. It is enforced when launching the instance with --metadata-options "HttpTokens=required", as we will do in the creation command.

We will take the visual route first, because it teaches the vocabulary, and then get the same result from the CLI, which is the one you automate.

1. Console → search for EC2 → check in the top right that the region is Ireland (eu-west-1). This check is not optional: it is the trap we saw in 01-04.
2. Instances → Launch instances.
3. Name and tags: mercadofresco-tienda-01. Click Add additional tags and fill in the project's mandatory scheme:
   • Proyecto = mercadofresco
   • Entorno = desarrollo
   • Componente = tienda
   • Propietario = luis
   • CentroCoste = operaciones
4. Image (AMI): Amazon Linux 2023, x86_64 architecture. Note the Free tier eligible label.
5. Instance type: t3.micro.
6. Key pair: select mercadofresco-tienda (the one you created earlier) or create it here and download the .pem.
7. Network settings: leave the default VPC and create a security group called sg-mercadofresco-tienda with two inbound rules: HTTP (80) from 0.0.0.0/0 and SSH (22) from My IP. Never SSH open to the world. The detail of this is lesson 03-02.
8. Storage: 8 GiB gp3, the default value. Disks are lesson 02-02.
9. Advanced details → scroll to the bottom and paste the contents of user-data-tienda.sh into the User data field. In the same block, check that IMDS version is set to V2 only.
10. Review the summary panel on the right and click Launch instance.

In 30-60 seconds the state will change to running and the status checks (2/2) will turn green. Copy the public IP and open it in your browser: you will see the MercadoFresco page with the instance ID and its Availability Zone.

Cost warning. A t3.micro is inside the Free Tier for the first 12 months (750 h/month). If you have already used yours up, it costs in the order of 0.01 USD/hour in eu-west-1. The final section explains how to delete everything.

The same result, in a single reproducible command. First the minimal security group (it is detailed in 03-02; here it is purely instrumental):

And now the instance:

Parameter by parameter, because each one has its reason:

• --user-data file://…: the file:// prefix is mandatory; without it, the CLI would send the literal string user-data-tienda.sh as the script. The CLI v2 base64-encodes the file for you.
• --metadata-options HttpTokens=required: enforces IMDSv2. HttpPutResponseHopLimit=1 stops a container inside the instance from reaching the host's metadata.
• --credit-specification CpuCredits=standard: while testing we would rather the instance throttled than generated unexpected charges from unlimited.
• --tag-specifications: it is passed twice, once for the instance and once for the volume. This is the most frequent tagging mistake: the instance gets tagged, the disk is forgotten, and in module 11 an EBS charge turns up that nobody can attribute to anyone.

Check the result with a readable query:

We now have a server. But one server does not solve problem 1: on Fridays from 17:00 to 21:00 900 orders/hour arrive and one instance handles 600. And buying a bigger machine "for Friday" means paying for it all 168 hours of the week in order to use it for 4.

AWS's answer is two pieces that work together.

This is the versioned recipe for what a storefront instance must look like: AMI, type, key pair, security group, user data, tags. There is no longer "the instance Luis built by hand"; there is a definition anyone can reproduce identically.

Templates are versioned: when you change the user data you create version 2 and you can go back to version 1 if something goes wrong. That ability to roll back is the first step towards problem 4 (risky deployments), which will be solved in full in module 8.

An ASG keeps a number of healthy instances spread across several Availability Zones, and adjusts it according to demand. It has three numbers:

The sizing comes out of the real figures:

• --vpc-zone-identifier: two subnets in two different AZs (eu-west-1a and eu-west-1b), just as we decided in 01-03. Replace the IDs with those of your default VPC.
• --health-check-grace-period 120: gives the user data 2 minutes before judging whether the instance is healthy. Without this wait, the ASG kills instances that were still installing.
• PropagateAtLaunch=true: the tags are copied onto every new instance. Without this, the machines created by the ASG would show up untagged on the bill.

And the policy that does the work automatically:

Target tracking is the recommended policy: you declare the goal and AWS works out on its own when to add and when to remove machines. Alternatives: step scaling (manual thresholds) and scheduled scaling, which makes a lot of sense for MercadoFresco because the peak happens at a known time:

The complete flow:

One piece is still missing for this to work for real: something that spreads the traffic across the instances. That is Elastic Load Balancing, lesson 03-03. Here we have built the engine; in module 3 we will connect the steering to it.

Golden rule of this course: whatever you create in a lesson, you delete when you finish it.

If you want to keep the instance for the next lesson without paying for compute, stop it instead of terminating it (you will still pay for the 8 GiB volume only, around 0.64 USD a month):

And remember to check the presupuesto-mensual-mercadofresco budget you set up in 01-02: it is your safety net if something is left switched on.

• Terminating when you meant to stop. The irreversible mistake par excellence. Always enable --disable-api-termination on any instance that holds data.
• Losing the .pem file. AWS has no copy. If you lose it, the only way out is to detach the volume and mount it on another instance. Keep it in your password manager and have Session Manager enabled as a plan B.
• Opening port 22 to 0.0.0.0/0. You will get automated access attempts within minutes. Always <your-ip>/32, or better still, no SSH port at all and Session Manager.
• Being surprised that CPU stays flat at 10 %. Those are exhausted burstable credits. Look at the CPUCreditBalance metric before blaming the application.
• Forgetting file:// in --user-data. The instance starts up "fine" but installs nothing. If the web server does not answer, log in and read /var/log/cloud-init-output.log.
• Tagging the instance and not the volume. Pass --tag-specifications for instance and for volume, always.
• Expecting user data to run on every start-up. It only runs the first time. For per-boot tasks, use a systemd unit created by the user data itself.
• Mistaking the public IP for something stable. It changes when you stop and start. Use DNS (03-05).
• Leaving the ASG with a high min-size "just in case". It is the quietest way to multiply the bill: every instance in the minimum is paid for 24×7.
• Golden tip: start small and measure. It is infinitely cheaper to size an undersized instance up than to discover in module 11 that you have been paying double for six months.

MercadoFresco wants to open in two more cities (problem 3). The forecast is that the Friday peak will go from 900 to 2,100 orders/hour. Each instance still handles 600 orders/hour, and Marta's policy requires that the service keeps working even if an entire Availability Zone goes down.

Work out the ASG's minimum, desired and maximum, justifying each number, and write the corresponding aws autoscaling update-auto-scaling-group command.

Luis launches an instance with the lesson's script, the instance shows as running with 2/2 checks green, but when he opens the public IP the browser just waits indefinitely (it does not say "connection refused", it simply does not answer).

List, in increasing order of investigation cost, the steps you would take and what you would check at each one. Say which is the most likely cause given the "it just waits" clue.

For each of these three MercadoFresco workloads, choose a family, an approximate size and a purchase model, and justify it in one sentence:

• A) Storefront server in production, sustained traffic 24 hours a day, 4 GiB of RAM is enough, CPU at 55 % on average.
• B) Overnight process that recompresses the catalogue's 40,000 photos; it takes 3 hours, it saturates the CPU, and if it is interrupted it can resume from where it was.
• C) Luis's development environment, switched on from 9:00 to 18:00 Monday to Friday, almost always idle with bursts when compiling.

Solution 1.

• Minimum = 2. In the trough no more is needed, but never fewer than 2 so that there is one instance in each AZ and no dependency on a single one.
• Desired = 2. It is the starting point; the target tracking policy will raise it on its own. It can be combined with a scheduled action that takes it to 4 on Fridays at 16:45.
• Maximum = 8. It covers the 4 needed plus the margin so that, if an AZ goes down, the remaining 4 can be launched in the one still standing. The maximum also acts as a spending cap: if an attack or a bug drives CPU up, it will never go beyond 8 instances.

Solution 2.

The key nuance: "it just waits" ≠ "connection refused". If nginx were down but the network reached the machine, the operating system would return an immediate refusal. A timeout indicates that the packets are not reaching the instance, that is, a network problem, not a software one.

Steps in increasing order of cost:

1. Security group (30 seconds, without logging into the machine): check that an inbound TCP 80 rule from 0.0.0.0/0 exists. This is the most likely cause.

   aws ec2 describe-security-groups --group-ids "$SG_ID" \\
     --query 'SecurityGroups[].IpPermissions' \\
     --profile mercadofresco-dev --region eu-west-1
   

2. Public IP: verify that the instance has one (PublicIpAddress not null) and that you are using that one and not the private 172.31.x.x.
3. Public subnet: the subnet must have a route to an internet gateway (this is fully understood in 03-01).
4. Log in through Session Manager (needs no port 22, useful precisely when the network fails):

   sudo systemctl status nginx
   sudo tail -50 /var/log/cloud-init-output.log
   curl -s localhost   # if this answers, the server is fine and the problem is the network
   

Solution 3.

MercadoFresco now has its first server on AWS and, above all, it has the vocabulary and the mechanisms to reason about it. You know that an AMI is the template and the instance the living copy, and that an AMI's ID changes with the region, which is why you look it up instead of copying it. You know how to decode t3.micro and to choose between the T, M, C and R families depending on whether the load is irregular, balanced, CPU intensive or memory intensive. You have seen the CPU credit mechanism from the inside and worked out that a t3.micro does not survive the four hours of the Friday peak: a concrete figure, not a hunch.

You know the four purchase models and which case each one fits, and you have mastered the lifecycle with the distinction that costs the most: stopping keeps the disk, terminating destroys it forever. You know how to reach the machine with a key pair and also without one, through EC2 Instance Connect or Session Manager, which is the option MercadoFresco will use in production because it needs no open port. You have automated the storefront's complete installation with user data, you know where to read its log when it fails, and you have protected the metadata by requiring IMDSv2. You have created the instance in the console and from the CLI, applying the project's tagging scheme to both the instance and its volume.

And, above all, you have put the first real piece in place against problem 1: a versioned launch template and an Auto Scaling group of 2 to 4 instances, with target tracking and scheduled scaling, sized with MercadoFresco's real figures (900 orders/h against 600 per instance). What is still missing is the load balancer that spreads the traffic across them, and that is lesson 03-03.

Before that there is a question Auto Scaling leaves exposed: if instances are born and die on their own, where do the data live? The product photos that today sit in /var/www/fotos cannot live inside a disk that is destroyed along with the machine. In lesson 02-02, "Block and file storage: EBS and EFS", we will look at volumes that outlive the instance, how they are extended while running, how they are shared between several machines, and how snapshots automated with Data Lifecycle Manager finally close MercadoFresco's problem 2: the backups that were never reliable.