Experiencing increased queue times and build performance issues

A third party provider experienced degraded performance which as a result impacted our network performance as well. The issue has been resolved and performance degradation can be no longer observed on our end. The engineering team is continuing to monitor.

This incident was composed of a mix of two issues. The core issue still being investigated happens to be a networking issue spread across different stacks. Initially some stacks such as Xcode 14.0 and 14.1 experienced higher than average queues. Our team was able to resolve the high queue issue that was delaying build start. We thank you for your patience as we continue to work on the networking issue.

We are currently experiencing higher-than-average queues on our Xcode 14.0 & 14.1. Our infra team is currently deploying additional resources to compensate and we are looking to also have positive improvements on the build performance across other stacks.

We are currently investigating this issue.