GitLab experienced a 1.8-hour outage where CI jobs failed to process on GitLab SaaS Shared Runners, specifically affecting GitLab Hosted Runners and Hosted Runners on Linux. The issue was caused by a recent change that was identified and rolled back, restoring normal error rates and job processing. The backlog of queued jobs was expected to clear within 30 minutes after the fix was implemented.
Trusted by 1,000+ teams
Stop finding out about outages from your users. Monitor 6,320+ cloud services and get alerted the second something breaks.
We are investigating an issue where CI jobs are not processing. For more info, see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22256
We have identified a recent change as a possible root cause and we are working on rolling back the change. For more info, see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22256
We are still working on rolling back the change that we believe is the root cause. This issue impacts GitLab Hosted Runners and Hosted Runners on Linux. For more info, see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22256
We are still working on rolling back the change that we believe is the root cause. This issue impacts GitLab Hosted Runners and Hosted Runners on Linux. For more info, see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22256
The change has been reverted and we are seeing that error rates are recovering. For more info, see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22256
Error rates have recovered back to normal levels and we started processing jobs again. We expect that the job queue should be processed over the next 30 minutes. For more info, see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22256
With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.
Start free trialNo credit card required · Cancel anytime · 6320 services available
Integrations with