Outage in Buildkite
Buildkite service disruption
Resolved MajorIncident Report
Summary AI Generated
Buildkite experienced a major service disruption caused by an autoscaling feedback loop that overwhelmed their Redis cluster with connections. This resulted in widespread impact across the Web UI, Agent API, REST API, and job queue — affecting job scheduling, artifact uploads, and generating elevated error rates — in two windows: 18:43–19:21 UTC and 20:02–20:28 UTC. Full recovery was confirmed by 20:28 UTC, with a post-incident review to follow later that week.
We've spotted that something has gone wrong. We're currently investigating the issue, and will provide an update soon.
Trusted by 1,000+ teams
The Status Page Aggregator with Early Outage Detection
Stop finding out about outages from your users. Monitor 6,320+ cloud services and get alerted the second something breaks.
Latest Updates ( sorted recent to last )
RESOLVED
2 days ago - at 09/10/2026 01:54AM
We have seen full recovery for customers since 20:28 UTC.
We experienced an autoscaling feedback loop which increased the number of connections to our redis cluster above its ability to respond. This had widespread impact for all of our customers with Web UI, Agent API, REST API and job queue impact between 18:43-19:21 UTC, and again between 20:02-20:28 UTC. A full post incident review will be available later this week.
MONITORING
3 days ago - at 09/09/2026 08:53PM
We are seeing improvements across the affected services, and are seeing services return to normal functionality. We are continuing to monitor and are determining the root cause.
INVESTIGATING
3 days ago - at 09/09/2026 08:28PM
We are continuing to investigate elevated error rates across multiple services, and are working to determine the cause.
INVESTIGATING
3 days ago - at 09/09/2026 07:41PM
We are continuing to investigate this issue. We are seeing impact on the Agent API which will affect job scheduling, artifact uploads, and an increase in 5xx responses from the Agent API endpoints.
INVESTIGATING
3 days ago - at 09/09/2026 07:07PM
We've spotted that something has gone wrong. We're currently investigating the issue, and will provide an update soon.
Latest Buildkite outages
Hosted Agents - Increased latency and error rates - 8 days ago
Increased latency and error rates - 8 days ago
Scheduling delay for hosted agents - 9 days ago
Delays in Hosted Agents job dispatch - 10 days ago
Delivery issues with email notifications - 11 days ago
The Status Page Aggregator with Early Outage Detection
With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.
Start free trialNo credit card required · Cancel anytime · 6320 services available
Integrations with
