Delay in background job processing

We are currently investigating an issue where prefilled CI/CD variables are not loading when starting a new pipeline on GitLab.com. More details can be found at: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We continue to investigate the issue where prefilled CI/CD variables are not loading when starting a new pipeline on GitLab.com. More details can be found at: gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We continue to investigate the issue where prefilled CI/CD variables are not loading when starting a new pipeline on GitLab.com. More details can be found at: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We continue to investigate the issue where prefilled CI/CD variables are not loading when starting a new pipeline on GitLab.com. More details can be found at: gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We continue to investigate the issue where prefilled CI/CD variables are not loading when starting a new pipeline on GitLab.com. More details can be found at: gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We are investigating a possible cause related to slow queries, this incident could impact more than prefilled CI/CD variables. More details can be found at: gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We are investigating possible solutions to relieve pressure on the database and job queues. This incident could affect pipeline job executions as well as prefilled CI/CD variables. More details can be found at: gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We are seen an improvement in the situation, CI/CD jobs should execute normally. As we are still working on fixing the underline issue we will keep monitoring the situation, next update expected in 3 hours. gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We continue to monitor the situation and there is no user impact at the moment. To resolve the underlying issue, we are currently applying mitigating steps. Next update expected in one hour. https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

No material updates to report. We are still applying mitigating steps. More details at https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We are reverting back the incident as active, we are still implementing mitigation steps. Users might experience delays in job creation or intermittent errors linked to background jobs queuing. More details at https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We are still applying mitigating steps. Impact for our users should be minimal at the moment. More details at gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

No material updates to report. We are still applying mitigating steps and the user impact is still minimal. More details at gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

GitLab is continuing to monitor this incident. User impact is minimal and we are waiting for deployments to complete. No further updates are anticipated until the deployment completes. More details at gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

We are continuing to investigate recent reports of prefilled CI/CD variables not loading when starting a new pipeline on GitLab.com. We are currently applying mitigating steps. More details can be found at: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

No material updates to report. We are still applying mitigating steps. More details at gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

The additional replica has been added and is taking traffic. We are now monitoring GitLab.com. CI Job processing and background jobs should be returning to normal operations. More details can be found at https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20101.

No material updates to report. We continue to monitor GitLab.com. More details can be found at gitlab.com/gitlab-com/gl-infra/production/-/issues/20101

No material updates to report. We continue to monitor GitLab.com. More details can be found at gitlab.com/gitlab-com/gl-infra/production/-/issues/20101