Outage in AWS

DNS Propagation Delays for EC2 Instance Names

Resolved Minor
July 27, 2020 - Started about 5 years ago - Lasted over 1 year

Incident Report

7:59 AM PDT We are investigating VPC DNS propagation delays in the US-EAST-1 Region. This issue only affects resolution of newly launched EC2 instances that rely on public to private IP DNS mapping, such as newly launched Relational Database Service (RDS) instances. It will also affect configuration changes to VPC peering associations. We have identified root cause and are actively working towards identifying a mitigation.

8:24 AM PDT We have identified a mitigation path and are working towards resolution. This issue only affects resolution of newly launched EC2 instances that rely on public to private IP DNS mapping, such as newly launched Relational Database Service (RDS) instances from within a VPC. It will also affect configuration changes to VPC peering associations.

9:26 AM PDT We have confirmed our mitigation is correct, and are actively deploying it. Recovery is in progress now.

10:43 AM PDT We are continuing to deploy the mitigation to this issue. Impact is limited to workloads that require resolving public DNS names to private IP addresses for instances launched in a newly created VPC, as well as reverse DNS lookups from within a newly created VPC. Resolution of all instances in existing VPCs continue to work normally.

12:04 PM PDT We are continuing to deploy the mitigation to this issue. Customers may see partial recovery as we work towards full resolution.

12:57 PM PDT Mitigation efforts continue, and customers should see increased recovery as we work towards full resolution.

1:41 PM PDT Between 6:32 AM and 1:35 PM PDT, customers experienced issues resolving newly created public EC2 instance names to private IPs within newly created VPCs in the US-EAST-1 Region. Creation of new Route 53 Resolver Endpoints was also delayed during this time. Newly created instances within existing VPCs resolved normally, and all other DNS functionality worked normally in all VPCs during this time. The issue has been resolved and all DNS queries are being answered normally.
The Status Page Aggregator Built for IT Teams

One place to monitor all your cloud vendors. Get instant alerts when an outage is detected.

Try IsDown risk-free 14-day free trial · No credit card required

The Status Page Aggregator Built for IT Teams

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 4522 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook