Degraded build data performance

We've made some additional small adjustments to mitigate the impact of future scans and builds, and we will continue working to optimize / improve as we move forward. At this time, we are currently stable and not expecting any additional slowdowns, queue backups, or blocked jobs and are hereby declaring this incident as resolved.

Backlogs have been largely resolved, we know the root cause and will be working to implement several permanent fixes.

A fix has been implemented and we are monitoring the results.

We are continuing to investigate the issue, part of the backlog has reduced, however, we are still seeing some elevated build times at this time.

We are presently experiencing a high backlog load of builds. As a result, database performance is degraded and build scanning is slowed. We are actively investigating the cause and will provide updates.