Actions runs using large hosted runners delayed for some customers

This incident has been resolved.

We are continuing to hold this incident open while the team ensures that mitigation put in place is stable.

Larger hosted runners job starts are stable and starting within expected timeframes. We are monitoring job start times in preparation to resolve this incident. No enqueued larger hosted runner jobs were dropped during this incident.

Over the past 30 minutes, all larger hosted runner jobs have started in less than 5 minutes. We are continuing to investigate delays in larger hosted runner job starts

We are still investigating delays in customer’s larger hosted runner job starts. Nearly all jobs are starting under 5 minutes. Only 1 customer larger hosted runner job was delayed by more than 5 minutes in the past 30 minutes.

We are seeing improvements to the job start times for larger hosted runners for customers. In the last 30 minutes no customer jobs are delayed more than 5 minutes. We will continue monitoring for full recovery.

We are seeing run delays for larger hosted runners for a limited number of customers. We are deploying mitigations to address these delays.

We are currently investigating this issue.