AWS EC2 — Virtual Machines On-Demand

What EC2 actually gives you, and what it does not

An EC2 instance is a slice of a physical server in one Availability Zone of one Region, handed to you with a running kernel and nothing else. That is the whole product, and its value is precisely its lack of opinion: any operating system, any runtime, any daemon, any kernel module, any licensing model. Every higher-level AWS compute service — ECS, EKS, EMR, RDS, Lambda — is a set of opinions layered over the same machinery, and you reach for EC2 when none of those opinions fit.

The price of that freedom is that you own everything above the hypervisor. AWS keeps the physical host, the network fabric, the storage fleet and the virtualization layer healthy. You own the guest kernel and its patches, the userland packages and their CVEs, the process supervisor, the log shipping, the host-level firewall, the AMI pipeline and the response when an instance goes unhealthy at 3am. Managed services move that line up; EC2 leaves it at the bottom. Choosing EC2 is choosing to be on call for a machine.

Two placement decisions are made at launch and cannot be changed later. The Region fixes your latency floor to users and your legal jurisdiction. The Availability Zone fixes your correlated-failure domain: an AZ is a distinct set of buildings with distinct power and cooling, and an instance lives in exactly one of them. Everything about EC2 resilience is downstream of that single fact — the instance is a mortal, zone-bound thing, and durability has to come from running more than one of them.

The instance lifecycle, and the three different promises about your data

This is the section worth reading twice, because the states look similar and behave nothing alike. An instance goes pending, then running, then either stopping/stopped or shutting-down/terminated. The distinctions that matter are what each transition does to your storage, your address and your bill.

A reboot is the gentlest: the instance stays on the same physical host. RAM is cleared by the guest OS reboot, but instance-store volumes keep their data, the auto-assigned public IPv4 address is kept, and you are billed straight through. A reboot is an OS event, not an EC2 lifecycle event.

A stop is a different animal. The instance is shut down and detached from its host entirely; on the next start it is placed on a different physical machine. Anything on local instance-store disks is gone permanently. The auto-assigned public IPv4 address is released and a new one is issued at start — which is why anything with a stable DNS name wants an Elastic IP or a load balancer in front of it. The private IPv4 address inside the VPC is retained. You stop paying for instance hours, but the EBS volumes are still provisioned and still billed, and this is the line item people forget in a fleet of parked dev boxes.

Hibernation is a stop that first writes the contents of RAM into the root EBS volume, so that a start resumes the same processes with the page cache warm. It has real prerequisites: it must be enabled at launch time, the root volume must be encrypted and large enough to hold all of memory, and only certain families, sizes and operating systems support it. Hibernation does not save instance-store data either — only RAM and the EBS root.

Terminate is final. The instance ID is retired and never reused, and every attached volume whose DeleteOnTermination flag is true is destroyed with it. The root volume defaults to true; volumes you attach afterwards default to false. Set disableApiTermination on anything whose accidental deletion would ruin a week.

AMIs, user data, and the golden-image question

An AMI is not a file. It is a manifest: one or more EBS snapshots, a block device mapping that says which snapshot lands on which device, plus metadata such as the architecture, the virtualization type, the root device name and the kernel/ENA/NVMe support flags. Launching an instance creates fresh volumes from those snapshots. Because snapshots are backed by S3 and hydrate lazily, a volume restored from a large AMI can be slow on its first read of each block — a real effect on first-boot benchmarks that has nothing to do with the instance type.

AMIs are regional. An AMI ID is meaningless in another Region, which is why any Terraform or CloudFormation that hardcodes one breaks the moment you expand. Look AMIs up by name filter or by SSM public parameter instead, and copy your own images explicitly to each Region you deploy in.

At boot, user data is handed to the instance and, on any modern distribution image, executed by cloud-init: a shell script, or a cloud-config document. This is the fork in the road. Configure at boot keeps one generic AMI and installs everything on startup — flexible, but it puts a package repository on the critical path of every scale-out event and makes boot time a function of the internet's mood. Bake a golden image pre-installs everything, so instances come up in seconds and identically, at the cost of an image pipeline and a rebuild every time a patch lands. Most mature fleets bake, keeping user data down to a few lines that inject environment-specific configuration.

User data is readable by anything on the instance that can reach the metadata service, and by any principal holding ec2:DescribeInstanceAttribute in the account. Never put a secret in it. Give the instance an IAM role and let it fetch from Secrets Manager or Parameter Store at boot.

Launch templates: the launch parameters as a versioned object

Everything the previous section describes — AMI ID, instance type, key pair, security groups, subnet, IAM instance profile, block device mapping, user data, metadata options, tags — is a launch parameter, and passing thirty of them by hand on every run-instances call is how fleets drift. A launch template is that parameter set stored as a first-class, immutable, versioned resource.

The versioning is the point. Template versions are numbered and never edited in place; you create a new version and then decide what points at it. A consumer can reference a specific version number, the $Latest version, or the $Default version, which turns promotion into a one-call operation and rollback into the same call pointing backwards. Auto Scaling groups, EC2 Fleet and Spot Fleet all consume launch templates, and a template can carry an override list of acceptable instance types so a fleet request can be satisfied from several capacity pools at once.

Launch templates also gate features that the older launch configurations simply cannot express — metadata options including IMDSv2 enforcement, Elastic Graphics and accelerator settings, instance-requirement based attribute selection, capacity reservation targeting and tag specifications that apply at creation time rather than after. Launch configurations are the legacy path and should not be used for anything new; the mechanics of migrating and of structuring template versions are the subject of the dedicated EC2 launch templates article.

Reading an instance type name

Instance type names are a compact encoding, and once you can read one you no longer need a lookup table for most decisions. The shape is family letter, then generation number, then optional attribute letters, then a dot and a size.

Two consequences fall straight out of this. First, within a family the size ladder is close to linear: a size with twice the vCPUs generally has twice the memory, roughly twice the network and EBS allowance, and twice the on-demand price. That linearity is what makes horizontal and vertical scaling roughly interchangeable on cost, so you can choose between them on operational grounds rather than financial ones. Second, the small sizes of a family often do not get the family's headline bandwidth; they get a burstable allowance instead.

Which family to pick for which workload, and how the families differ in CPU-to-memory ratio, is covered in EC2 instance families; the right-sizing workflow itself lives in EC2 instance types.

The burstable T family and the credit trap

The T family is the one that surprises people, because it is not sold in whole CPUs. A burstable instance is entitled to a documented baseline fraction of each vCPU — a percentage that varies by size, and is low for the small sizes. While you use less than baseline you accrue CPU credits; each credit buys one vCPU-minute at full speed. When you exceed baseline you spend credits. Accrued credits also expire, capped at roughly a day's worth of earning, so a long-idle instance does not bank an unlimited reserve.

What happens when the balance hits zero depends on the mode. In standard mode the instance is throttled hard down to its baseline — a fraction of a core — and it stays there until it earns credits back. This is the failure that looks like nothing: no error, no alarm, no OOM, just an application whose latency has quintupled and a CPU graph pinned at a flat, suspiciously round number. In unlimited mode the instance keeps running at full speed and you are billed for the surplus vCPU-hours, which is usually the behaviour you want but converts a performance problem into a silent bill. Older T2 instances default to standard; the newer T3, T3a and T4g generations default to unlimited.

The operational rule is simple. Alarm on CPUCreditBalance, not just CPUUtilization — on a throttled T instance utilization looks fine, because it is a percentage of the reduced entitlement you have been cut down to. And treat the T family as what it is: superb for genuinely spiky, mostly-idle work such as build agents, bastions, small internal tools and dev environments, and the wrong instrument for anything with a steady load. If your average utilization sits above baseline all day, an M-family instance of the same size is both faster and, once unlimited surcharges are counted, frequently cheaper.

Nitro: the substrate that changed what an instance is

Classic EC2 virtualization ran a Xen hypervisor with a privileged management domain on the host CPU. That domain did the work of emulating the network card and the disk, which meant a meaningful fraction of every server's CPU and memory was spent on virtualization overhead instead of on customers, and the software attack surface between tenants was large.

The Nitro System moved that work off the main board. Dedicated hardware cards handle VPC networking, EBS attachment, local NVMe storage and instance management, each presenting itself to the guest as a standard PCIe device. A Nitro Security Chip mediates access to hardware and validates firmware. What remains on the host CPU is a minimal KVM-based hypervisor whose only job is to start and stop instances and enforce memory isolation — there is no general-purpose management OS with a console you could log into, by design.

Three consequences show up in daily work. Nearly all of a host's CPU and RAM can be sold to guests, which is why Nitro generations offer instance sizes that consume an entire physical server and why .metal sizes exist at all: bare metal is simply an instance with no hypervisor slice taken out, still getting real ENA networking and real EBS because those live on the cards. Storage is presented as NVMe rather than as emulated Xen block devices, so your devices appear as /dev/nvme0n1, /dev/nvme1n1 and so on — and critically, NVMe enumeration order is not guaranteed to match your block device mapping. Any script that assumes the second data volume is a particular NVMe node is a time bomb; mount by filesystem UUID or label. Third, all traffic is encrypted in transit between Nitro instances within a VPC by default on supported types, with no configuration.

Nitro Enclaves, the attested isolated-compute feature carved out of an instance's own vCPUs and memory, and the card-by-card architecture are covered in the AWS Nitro System.

Storage as the instance sees it

From inside the guest, block devices come in exactly two flavours and confusing them is expensive. Instance store is physical NVMe attached to the host your instance is running on right now. It is the fastest storage EC2 offers, with latency an order of magnitude below anything network-attached, and it is completely ephemeral: it survives a reboot and nothing else. Stop, hibernate, terminate or an underlying host failure all take it with them, and there is no snapshot mechanism. Use it for scratch, spill, shuffle, local caches and replicated database data that the cluster can rebuild — and treat any single copy on it as already lost. The details of the local NVMe surfaces live in EC2 instance store.

EBS is a network-attached, replicated volume that outlives the instance and can be detached and reattached elsewhere in the same AZ. Volume types, snapshots, encryption and the gp3 provisioning model are the subject of AWS EBS and EBS volume types; what belongs here is the part that is an instance property rather than a volume property.

That part is the EBS bandwidth ceiling. Every instance type has a published maximum EBS throughput and IOPS, separate from its general network allowance, and your real performance is the minimum of what the volume is provisioned for and what the instance is allowed. Provisioning 1,000 MB/s of gp3 on an instance whose EBS pipe tops out well below that buys nothing but a larger invoice. On smaller sizes this ceiling is itself burstable — a sustained restore or a bulk load can run at the advertised number for half an hour and then step down to a baseline, which reads exactly like a mysterious mid-job slowdown. Size the instance to the storage workload, not only to the CPU workload.

The network edge: ENIs, ENA and EFA

An instance touches the network through one or more elastic network interfaces. The primary ENI is created at launch, cannot be detached, and carries the primary private IPv4 address that the instance keeps for life. Secondary ENIs can be attached and moved between instances in the same AZ, which makes them a useful failover primitive: an ENI carries its addresses, its MAC and its security groups with it, so moving one moves an identity rather than just an address.

How many ENIs and how many IP addresses each can hold is a function of the instance type, and this is not trivia — on EKS with the VPC CNI, where every pod takes a real VPC address from an ENI, that table is what determines maximum pod density per node. Choosing a node size without checking it is how clusters end up unable to schedule despite having free CPU. Subnets, route tables, security groups and NACLs are AWS VPC's territory and are not restated here.

Enhanced networking means the interface is presented via SR-IOV, so the guest driver talks to a virtual function on real hardware instead of an emulated device. On modern instances that driver is the Elastic Network Adapter (ENA), and it is why single instances can sustain tens of gigabits with low jitter. Instance network bandwidth follows the same baseline-plus-burst pattern as EBS on smaller sizes, and per-flow throughput is capped well below aggregate throughput — a single TCP connection will not saturate a large instance, so bulk transfers need parallel streams. That single-flow cap is raised for instances inside a cluster placement group.

For tightly coupled HPC and distributed training there is the Elastic Fabric Adapter, an ENA that additionally exposes an OS-bypass path so MPI and NCCL can talk to the hardware without traversing the kernel TCP stack. That removes the syscall and copy overhead that dominates small-message collective latency. EFA is available only on selected instance types, requires the EFA driver in the AMI, and requires a security group that allows all traffic to and from itself.

IMDSv2, and why version 1 was a genuine security hole

Every instance can reach a link-local address, 169.254.169.254, that serves the Instance Metadata Service: instance ID, AZ, network configuration, user data, and — the important one — automatically rotated temporary credentials for the IAM role attached to the instance. This is the mechanism that lets an application call AWS APIs without a stored access key, and it is the single best reason to attach roles rather than ship credentials.

In the original protocol, now called IMDSv1, fetching those credentials was an unauthenticated HTTP GET with no headers. That is a catastrophic pairing with server-side request forgery. Any bug that let an attacker persuade your application to fetch a URL of their choosing — a webhook tester, an image-resizing endpoint, a misconfigured reverse proxy, an open redirect — could be pointed at the metadata address and would return live role credentials in the response body. This is not theoretical; it is the pattern behind the 2019 Capital One breach, and it turned a single application-layer flaw into full use of whatever the instance role could do.

IMDSv2 closes it with session tokens. A caller first issues an HTTP PUT to /latest/api/token carrying a TTL header, gets a token back, and must then present that token as a header on every subsequent GET. Each requirement blocks a class of attack: most SSRF primitives can only issue GETs, most cannot set arbitrary request headers, and neither can an open redirect. A third control matters as much — the response hop limit, which is written into the IP TTL. Left at 1, a response cannot cross an extra network hop, so a containerized workload on a bridge network cannot reach the host's credentials at all.

The configuration to enforce is HttpTokens=required, with the hop limit set as low as your container networking allows, applied through your launch template so it is inherited by everything an Auto Scaling group creates. Newer AMIs and account-level metadata defaults now make v2-only the norm for fresh launches, but long-lived fleets often still permit v1 — audit for it, and use the MetadataNoToken CloudWatch metric to find which instances are still being called the old way before you flip the switch. What the role can then do is AWS IAM's problem, and a small role is the second half of this defence.

Placement groups: telling AWS where to put the machines

By default EC2 places instances wherever it likes within the AZ you asked for. A placement group overrides that with one of three strategies, and each optimizes for a different thing.

Cluster packs instances close together on the same high-bisection-bandwidth segment of the network, giving the lowest inter-instance latency and the highest per-flow throughput. It is the right choice for HPC, distributed training and tightly coupled analytics. The trade-off is honest: everything is in one AZ, packed together, so a correlated failure hits all of it, and a large cluster launch can fail with an insufficient-capacity error because the request is much harder to satisfy than scattered instances. Launch the whole group in one request where you can.

Spread is the opposite instruction: put each instance on distinct underlying hardware with separate power and network, so no two share a failure. It suits a handful of critical instances — the members of a quorum, a pair of brokers — and is deliberately limited to a small number of running instances per AZ, because the guarantee gets impossible to keep at scale. Partition is the middle path built for rack-aware distributed systems: instances are divided into partitions, partitions do not share racks, and the partition an instance landed in is exposed through the metadata service so HDFS, Cassandra or Kafka can use it as rack topology and place replicas across failure domains. The mechanics and the limits are detailed in EC2 placement groups.

Four ways to buy the same machine

The instance is identical in every purchasing model; only the commitment and the interruption contract differ.

Savings Plans have largely replaced Reserved Instances for compute. A Compute Savings Plan commits to an hourly spend and applies automatically across families, sizes, Regions and even Fargate and Lambda; an EC2 Instance Savings Plan is narrower, locking to a family in a Region in exchange for a deeper discount. Reserved Instances still matter where they buy something a plan does not — a zonal RI carries a capacity reservation, which a Savings Plan never does. If you need guaranteed capacity without a discount commitment, an On-Demand Capacity Reservation is the separate instrument for that, and it composes with a Savings Plan. The comparison in full is in Savings Plans and Reserved Instances.

Spot sells spare capacity at a steep discount under one condition: AWS can take it back. The contract is specific. You get a two-minute interruption notice, delivered through the instance metadata service, and usually an earlier and softer rebalance recommendation signalling that this pool is at elevated risk. Two minutes is enough to drain a load balancer target, checkpoint a job or finish a message — but only if something is polling for the notice, and building that handler is the actual work of adopting Spot. Diversify across many instance types and AZs and let a capacity-optimized allocation strategy choose the deepest pools, because interruption risk is per-pool rather than global. The discount is large but varies continuously by pool and Region, so read the current Spot advisor rather than trusting any number you have memorised. Details in EC2 Spot instances.

Graviton and the migration question

Graviton instances — the ones with a g in the type name — run AWS's own ARM64 processors rather than x86. AWS positions them as offering better price-performance for most general-purpose, memory and compute workloads, and for a large class of server software that holds up. The interesting question is not whether they are fast; it is what a migration actually costs you.

The answer depends entirely on how far your code sits from the metal. Anything running on a managed runtime — JVM, .NET, Node, Python, Ruby, PHP — is usually a redeploy on an ARM64 AMI and nothing more, because the runtime has been ported for years. Go and Rust are a cross-compilation flag. The friction is always in the layer underneath: native extensions compiled from source, a vendored shared library with only an x86 build, a proprietary agent or database driver the vendor never shipped for ARM, or hand-written intrinsics. Container fleets have a second failure mode — every base image and every sidecar in the chain needs an arm64 variant, so you need multi-architecture image builds and a registry holding both, and a single x86-only sidecar pins the whole pod.

The pragmatic path is to migrate a stateless, high-volume service first, where the savings are real and a rollback is a deployment. Run both architectures behind the same load balancer during the transition, and benchmark rather than assume: performance differences are workload-specific and per-core comparisons across architectures are not meaningful. The porting checklist is in AWS Graviton.

Operating a fleet: access, right-sizing and health

Two habits separate a fleet that is pleasant to run from one that is not. The first is getting rid of SSH keys. A key pair's public half is injected by cloud-init on first boot only, so it cannot be rotated by changing the launch template, and a long-lived private key on a laptop is exactly the credential you did not want. Attach the SSM agent's IAM role and use Session Manager instead: access becomes an IAM decision, sessions are logged and auditable, and the instance needs no inbound port, no public address and no bastion.

The second is right-sizing on evidence. CloudWatch's default EC2 metrics come from the hypervisor, which can see CPU, network and EBS activity but cannot see inside the guest — there is no memory-utilization or disk-space metric until you install the CloudWatch agent. Fleets that never install it are the ones where every instance is sized by guesswork and half are twice as big as they need to be. Install the agent, then let Compute Optimizer use the history to recommend, and re-check after every generation launch, because a newer generation is frequently both faster and cheaper than the one you are on.

Finally, treat instances as replaceable. Health is reported as two separate status checks: a system check covering the underlying host and network, and an instance check covering the guest's own reachability, with newer attached-EBS status surfaced alongside. A failed system check usually means the host is sick and a stop/start — which migrates you to a different host — is the fix; a failed instance check usually means you broke the guest. Wire both into an Auto Scaling group that replaces unhealthy members automatically, put an Elastic Load Balancer in front, and the single most common EC2 incident stops being an incident at all.