Degraded REST API success rates for some customers

This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.

The deployment has completed and we expect customers who have been impacted to see recovery. We are continuing to monitor.

We are in the process of deploying the PR to revert the change that was causing timeouts to this endpoint. We will update again once that deployment is complete.

We have identified a deployed change that correlates with the increase in 5XX errors to the GitRefs REST API. This is particularly affecting requests for repos with very large numbers of commits. We are working on rolling back this change which we expect will resolve the issue.

API Requests is experiencing degraded performance. We are continuing to investigate.

Customers are experiencing 504 responses for some API requests for regarding repo refs/tags. We are investigating.

We are currently investigating this issue.