Runner authentication verification API errors

We are seeing elevated error rates on the POST /runners/verify endpoint. We are currently investigating. See the infrastructure issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/18792

We have identified the cause of the errors, and are currently working on the fix. For more details, see the incident issue: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/18792

We have identified and pushed the fix, and it is currently in the process of being deployed. For more details, see the production issue: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/18792

Deployment of the fix has completed, but it is not having the impact we hoped for. We're currently weighing options for a second fix. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/issues/18792.

We're finalizing the next method we'll be using to mitigate the incident and will be attempting another fix shortly. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/issues/18792.

We've manually applied a fix that has stopped the error rate and mitigated the incident. We'll be monitoring for a time to ensure the issue doesn't recur. Details: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/18792.