Snyk PR Checks for Azure Repositories are Degraded

The fix has now been deployed to our app.snyk.io environment. The fix has now been applied to all multi-tenant environments. Our engineers are monitoring at this time.

The fix has now been deployed to our app.eu.snyk.io and app.au.snyk.io environments.

We will share another update when the rollout has concluded.

The environment on which the fix was initially rolled out was app.us.snyk.io and not app.snyk.io as reported earlier.

Once the rollout to our remaining environments is complete, we will provide a further update.

The fix implemented to our app.snyk.io environment has been successful in mitigating the issue. Our Engineers will proceed with deploying the fix across our remaining environments.

We will provide a further update once the rollout is complete.

Our Engineers have rolled out a fix to our app.snyk.io environment due to the higher volume of expected traffic in this environment. This is to ensure that the issue is mitigated and that services respond well to the changes made.

We will continue to monitor throughout the evening and roll out the fix to all other environments once we have full confidence that the issue is resolved.

Our internal testing remains ongoing. However, it is almost complete.

During our testing and investigations, we have identified that all our multi-tenant environments will experience this issue.

Further updates will be shared once internal testing is completed.

Our internal testing remains ongoing, and preliminary indications suggest we are nearing a point where we can effectively mitigate the impact.

Considering the complexity of the solution and the potential risk to other services, we would like to continue with our internal testing until we are completely confident before deploying the fix.

Our root cause investigation has led us to what we believe is a configuration issue, which causes intermittent behaviours with Azure Repos.

We're currently testing a fix internally to ensure it mitigates the issue.

Our Engineers have determined that this issue only affects a small percentage of Azure Repos PR Checks, as we have successfully reproduced this internally.

We are now capturing additional logs for analysis to help us identify and understand the root cause of the issue.

Our investigations remain ongoing. However, we have validated that this only impacts PR Checks for Azure repos for Snyk Code and Open Source within our app.snyk.io environment.

We will keep you updated on the progress of our investigation.

Our Engineers are aware of an issue affecting our PR Checks for Snyk Code and Open Source.

Initial impact assessment is ongoing, however, so far we believe the issue only impacts a small subset of customers within our app.snyk.io environment.

Affected customers may be observing their PR Checks within Snyk Code and Open Source getting stuck in 'waiting' status.

The Snyk Incident Response has been invoked, and initial investigations are ongoing.