GitLab experienced a 17-hour minor incident where customers received HTTP 500 errors when attempting to open projects, affecting both the website UI and Git operations. The root cause was identified as an abuse vector targeting Gitaly nodes; mitigation efforts including node restarts and reconfiguration proved ineffective, ultimately requiring a full Gitaly rollback to resolve the issue. Service stabilized following the completed rollback deployment.
Trusted by 1,000+ teams
Stop finding out about outages from your users. Monitor 6,320+ cloud services and get alerted the second something breaks.
We're investigating reports of customers unable to open projects, receiving error 500 instead.
No material updates are available at this time. Our team continues to investigate the root cause of the issue.
We’ve identified an abuse vector as the root cause and begun mitigation efforts. Early signs indicate that the service is recovering.
We’re continuing mitigation efforts and have observed some recovery. Customers may still encounter HTTP 500 errors in the UI and during Git operations when accessing certain projects.
We are continuing mitigation and monitoring recovery. Customers may still encounter HTTP 500 errors in the UI and during Git operations for certain projects. We will provide a further update as mitigation progresses.
Mitigation has not yet fully restored the GitLab UI and Git operations in certain projects. We are still investigating HTTP 500 errors and will provide a further update as more information becomes available.
Investigation continues. We will shortly be restarting the affected Gitaly node after reconfiguration. Further updates will be provided accordingly.
The restart of the affected Gitaly node did not improve the issue; HTTP 500 errors continue affecting the GitLab UI and Git operations in certain projects. We are investigating and will provide further updates as the situation changes.
Further mitigation is being rolled out while we continue to investigate HTTP 500 errors affecting the GitLab UI and Git operations. We will provide further updates as the mitigation progresses.
A further mitigation change has been applied, and we are assessing its impact. HTTP 500 errors continue affecting the GitLab UI and Git operations in certain projects. Further updates will follow.
Further mitigation change are currently being applied, and we are continuing to investigate the issue. HTTP 500 errors may continue affecting the GitLab UI and Git operations in certain projects. Further updates will follow.
We are currently rolling back the deployment of Gitaly. We're waiting for the deploy to land, and we are continuing the investigation.
A rollback release has been deployed to all Gitaly nodes. They are stable at present. However, we are continuing to monitor the situation.
Gitaly nodes have stabilized following the completed rollback deployment. The incident has been mitigated and we remain focused on continued monitoring.
With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.
Start free trialNo credit card required · Cancel anytime · 6320 services available
Integrations with