GitLab experienced degraded performance in their AI-assisted services and AI gateway, causing users to encounter higher latency and errors when using Duo features. The incident lasted 4.6 hours and was caused by request saturation and potential segfaults affecting the AI services. GitLab resolved the issue by increasing maximum concurrency limits to reduce latency.
Trusted by 1,000+ teams
Stop finding out about outages from your users. Monitor 6,320+ cloud services and get alerted the second something breaks.
We are investigating issues related to our ai-assisted service and ai-gateway. Some users may experience higher latency or errors affecting Duo features. More details here: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21867
We are investigating which model and provider are causing the latency. Apdex for ai-assisted service is not stable yet. More details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21867
The root cause is still under investigation. We are taking steps to address the sustained request saturation in order to mitigate latency issues. More details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21867
We have increased max concurrency and are monitoring the impact on latency. We'll further investigate segfaults, which may have contributed to this incident. More details in: gitlab.com/gitlab-com/gl-infra/production/-/work_items/21867
With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.
Start free trialNo credit card required · Cancel anytime · 6320 services available
Integrations with