Increased Error Rates for Cluster Upgrades - N. Virginia

We are making progress towards addressing the elevated error rates for cluster upgrade operations in the US-EAST-1 Region. In an abundance of caution, we left EKS traffic shifted out of the previously affected Availability Zone (use1-az2) while additional mitigations for the previous event were put in place. This meant traffic was shifted away from the affected zone, which would prevent workloads from running in the affected Availability Zone. During this period, customers would be unable to perform cluster upgrades and configuration updates in the Region. We are nearly complete with shifting traffic back in for EKS dependencies, and we will then enable cluster upgrade operations for EKS. We expect to see recovery within the next 30 minutes.

We can confirm that customers are experiencing elevated error rates for cluster upgrade operations in the US-EAST-1 Region. This is a result of actions we took to prevent additional customer impact following the EC2 network propagation event yesterday in a single Availability Zone (use1-az2). For EKS customers, we have purposely disabled cluster upgrade operations to prevent any impact to existing clusters and running applications. We have identified the root cause for the underlying issue and are working towards mitigating the current impact. We will enable cluster upgrade operations once the issue is resolved. We will provide an update within the next 60 minutes, or sooner if we have additional information to share.