US region: Ongoing upstream provider degradation causing intermittent errors across multiple services

This incident has been resolved.

In the last three hours we observed a single brief spike in 5xx responses around 20:45 UTC in the US region; since then, 5xx has held at 0% across monitored endpoints. For transparency, we'll keep the incident open in Monitoring with no components marked as affected through the end of the day while we continue to observe.

Update - Error rates are trending down. Over the last few minutes the US region is averaging about 0.4% 5xx responses. Core paths (Authentication, Viewing Content & Search, Self‑Service, and content edits) are each ≤0.5%, while Integrations and Insights & Analytics fluctuate slightly higher but remain under 1%. We'll keep mitigations in place and continue to monitor.

Update - This remains a brownout, not a full outage. Most requests in the US region are succeeding, but ongoing upstream degradation is causing intermittent errors and latency. Authentication, Viewing Content & Search, Self‑Service Portals, Emails & Notifications, and Making Changes to Content are largely available; Integrations and Insights & Analytics are the most affected with higher delays/timeouts. We continue to mitigate and monitor.

Reopened. Our earlier resolution was premature. Our cloud infrastructure provider continues to report widespread degradation in their US‑East region (about 80 services affected, several in a brownout state). This is causing intermittent errors, elevated latency, and timeouts across multiple Shelf services in the US region, Answer Assist and Search, and integrations. EU and CA regions remain healthy. We are working with the provider and have applied mitigations to reduce impact. Next update within 30 minutes.

We're still observing occasional timeouts in Agent Assist searches for certain real-time suggestions.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

We are continuing to investigate this issue.

We are currently investigating this issue.