Circle CI Outage History

Circle CI experienced delays in workflow and job data delivery through their Usage API, with some data from the last 24 hours not being fully reflected due to a backlog of events. The Insights UI remained unaffected during the incident. The issue was resolved after 7.3 hours, though a 4-hour period of RAM and CPU utilization data from April 7th was permanently lost and will not be backfilled.

CircleCI experienced a 40-minute degradation affecting UI elements, specifically approval jobs and SSH key navigation functionality. The issue was limited to the interface only, with builds continuing to operate normally throughout the incident. The problem was resolved after implementing a fix and monitoring the results.

Circle CI's Usage API experienced incomplete data for April 1st, with multiple columns showing null values while data from other dates remained unaffected. The issue was identified and a fix was implemented, with the corrected data becoming available on April 3rd after a 7-hour resolution period.

Circle CI experienced a major service incident where some pipelines failed to trigger due to high error rates from GitHub's `GET /repos/{owner}/{repo}/commits` API endpoint. The issue affected manually-triggered pipelines, OAuth hooks, and scheduled pipelines for 33 minutes. The incident was resolved when GitHub's API endpoint became stable and responsive again.

Circle CI's Insights API and dashboard displayed stale data for reporting windows longer than 24 hours (7, 30, 60, and 90-day views), while the 24-hour window continued showing current data. The issue was limited to aggregated metrics display only, with all pipeline pages, job runs, and builds remaining fully operational. The engineering team identified the root cause and deployed a fix within 54 minutes, though full data refresh required 24-36 hours to complete.

CircleCI experienced an issue where users were unable to view their organization's orb list on the Organization Settings page, with the page appearing blank or failing to load the expected public and private orbs. The incident only affected the CircleCI UI while pipeline execution and orb usage in builds continued to function normally. Engineers deployed a fix after 28 minutes, fully restoring the ability to view orb lists on the Organization Settings page.

Circle CI experienced issues with Docker functionality on convenience images (cimg:*) caused by a Docker update deployed that morning. The problem affected Docker Jobs for 25 minutes before being resolved. A fix was deployed and the service was restored to normal operation.

Circle CI users experienced inconsistent failures during checkout steps due to an issue with GitHub as the upstream VCS provider. The incident lasted 1.5 hours and was resolved when GitHub acknowledged and fixed the underlying problem on their platform.