Outage in GitLab

Sidekiq job delays resulting in updates to MR approval policies being ineffective

Resolved Minor
October 03, 2024 - Started 3 months ago - Lasted about 17 hours

Need to monitor GitLab outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including GitLab, and never miss an outage again.
Start Free Trial

Outage Details

We are currently investigating issues with updating MR Approval policies as a result of Sidekiq performance degradation.

More details about this incident can be found in gitlab.com/gitlab-com/gl-infra/production/-/issues/18660.
Components affected
GitLab Website GitLab API
Latest Updates ( sorted recent to last )
3 months ago - at 10/03/2024 02:12PM

We are currently investigating issues with updating MR Approval policies as a result of Sidekiq performance degradation.

More details about this incident can be found in gitlab.com/gitlab-com/gl-infra/production/-/issues/18660.

3 months ago - at 10/03/2024 02:28PM

We've identified the root cause and are recovering from a high volume of request traffic and are seeing improvements. Updates to MR approval policies are currently processing but with delays.


More details about this incident can be found in gitlab.com/gitlab-com/gl-infra/production/-/issues/18660

3 months ago - at 10/03/2024 02:46PM

We're continuing to see improvements and are monitoring until a fix is deployed. Updates to MR approval policies are still processed with delays.


More details about this incident can be found in gitlab.com/gitlab-com/gl-infra/production/-/issues/18660

3 months ago - at 10/03/2024 03:55PM

We're continuing to monitor the performance of Sidekiq until the fix is deployed. Updates to MR approval policies are still processed with delays. The MR to fix the root cause is currently waiting to be merged.


More details about this incident can be found in gitlab.com/gitlab-com/gl-infra/production/-/issues/18660

3 months ago - at 10/03/2024 05:06PM

We are actively monitoring the performance of Sidekiq until the fix is deployed. Updates to merge request approval policies are still experiencing delays. Details about this incident can be found in gitlab.com/gitlab-com/gl-infra/production/-/issues/18660

3 months ago - at 10/03/2024 06:02PM

We continue to closely monitor Sidekiq’s performance while awaiting the deployment of the fix. In the meantime, updates to MR approval policies remain delayed. More details about this incident can be found at gitlab.com/gitlab-com/gl-infra/production/-/issues/18660

3 months ago - at 10/03/2024 07:02PM

We are still awaiting the deployment of the fix and will continue to monitor Sidekiq’s performance in the meantime. Updates to MR approval policies are delayed until the issue is resolved. For more details, visit gitlab.com/gitlab-com/gl-infra/production/-/issues/18660

3 months ago - at 10/03/2024 08:00PM

The fix is expected to be merged soon. In the meantime, we’ll continue monitoring Sidekiq’s performance. Updates to MR approval policies will remain delayed until the issue is fully resolved. For more details, visit gitlab.com/gitlab-com/gl-infra/production/-/issues/18660

3 months ago - at 10/03/2024 09:04PM

The fix has been successfully merged. We will continue to monitor Sidekiq’s performance to ensure everything is functioning as expected. For details, visit gitlab.com/gitlab-com/gl-infra/production/-/issues/18660

3 months ago - at 10/03/2024 10:23PM

The fix has been successfully merged, but we are still waiting for the changes to reach production. We will continue to monitor Sidekiq’s performance, and the next update will be provided once the incident has been mitigated. For more details, visit gitlab.com/gitlab-com/gl-infra/production/-/issues/18660

Start monitoring all your vendors in just 5 minutes

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 3278 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook

Setup in 5 minutes or less

How much time you'll save your team, by having the outages information close to them?

14-day free trial · No credit card required · Cancel anytime