Skip to main content

Posts

Showing posts with the label Amazon

What caused the AWS outage that broke the internet on Oct 21st 2025

Here is a technical explanation for the AWS Outage due to a DNS failure. AWS's own internal control plane is built on top of DynamoDB. It's a hidden dependency. When AWS's internal services couldn't find the IP for DynamoDB, the entire management layer collapsed. Stage 1: DNS Fails. The internal DNS servers for dynamodb.us-east-1.amazonaws.com stopped working. Stage 2: Control Plane Fails. AWS's own services that depend on DynamoDB immediately broke. This included: IAM (for authentication and session state) The EC2 instance launch subsystem (which uses DynamoDB for metadata) Network Load Balancer (NLB) health checks (which, it turns out, write their health state to a DynamoDB table) Stage 3: Circular Dependency. This is the crazy part. When the NLB health checks failed (because they couldn't write to DynamoDB), it caused more network connectivity issues, which in turn impacted the (already struggling) DynamoDB service itself. It created a vicious feedback loop....