[Resolved] Increased API error rates and latencies - N. Virginia

Between 8:52 PM on April 7, and 10:30 AM PDT on April 8, we experienced increased API error rates and latencies for the following Bedrock models in the US-EAST-1 Region: Llama 3.2 and 3.8, Claude 3.5 Sonnet, Claude 3 Opus, Claude Instant v1, Titan, Cohere, and Mistral. Initially, the issue was only impacting the Titan, Cohere and Mistral models. At 8:54 PM on April 7, we communicated to impacted customers via the Personal Health Dashboard. This morning, at 6:47 AM on April 8, we began seeing impact to Claude models. Engineers were engaged at the beginning of the impact. By 10:27 AM on April 8, we began to see initial signs of recovery. By 10:30 AM, we had fully mitigated the issue. We do not expect this issue to recur. Customers who made use of an alternate AWS Region during this time may now return to using US-EAST-1. The issue has been resolved and the service is operating normally.

We are beginning to see early signs of recovery for the increased API error rates and latencies for Bedrock APIs in the US-EAST-1 Region. We still recommend customers use an alternate AWS Region while we work toward full resolution. We will provide additional updates as we have more information to share, or by 11:30 AM PDT.

We continue to work toward mitigating the issue resulting in increased latencies and error rates for Bedrock APIs in the US-EAST-1 Region. This issue is affecting multiple model types, including Llama 3.2 and 3.8, Claude 3.5 Sonnet, Claude 3 Opus, Claude Instant v1, as well as Titan, Cohere, and Mistral models. We are actively working on multiple parallel paths to mitigate the issue. During this time, customers may experience increased latencies, or 503 "ServiceUnavailableException" errors when accessing the Bedrock APIs for the impacted models in the US-EAST-1 Region. Other Bedrock models continue to operate normally. We recommend customers who can, use an alternate AWS Region. We will provide you with another update by 11:30 AM PDT, or sooner if we have any additional information to share.

We can confirm increased latencies and increased API error rates for Amazon Bedrock when using Titan, Cohere and Mistral models in the US-EAST-1 Region. We can confirm the impact is related to the Amazon Titan Text Embedding v1, Amazon Titan Text Embedding v2, Titan Multimodal Embeddings G1, Mistral 7b Instruct, Cohere English V3 and Cohere Multilingual V3 models in the US-EAST-1 Region. We can also confirm this impact is limited to the US-EAST-1 Region. We are actively working on multiple parallel paths to mitigate the issue. During this time, customers may experience increased latencies, or 503 "ServiceUnavailableException" errors when accessing Bedrock APIs for impacted models. Other Bedrock models continue to operate normally. We recommend customers who can, use an alternate AWS Region. We will provide you with another update by 10:30 AM PDT, or sooner if we have any additional information to share.