Spaces, App Platform, and Container Registry

Since November 03, 16:55 UTC, we have not seen further recurrences of impact from this incident, indicating the mitigations put in place have been successful as a temporary fix. Due to this, we will be closing this incident.

Our Engineering team will continue to closely monitor error rates and the overall health of Spaces, App Platform, and Container Registry. At the same time, they will continue work to address the root cause in order to put a more permanent fix in place.

If we experience further issues, we will communicate via a new status page, linking back to this one for reference.

We sincerely appreciate your patience throughout this incident. If you continue to experience issues or have any questions, please open a ticket with our Support team.

Our Engineering team has taken steps to reduce the impact of this issue to users and as of 17:00 UTC, has not observed any further errors.

We are continuing to work on a permanent fix to ensure this issue does not reoccur in the future. In the meantime, we're monitoring the situation and will provide another update as soon as more information is available. Thank you for your patience and we apologize for the inconvenience.

Our Engineering team has observed a recurrence of this issue and is taking steps to remediate user impact now.

During this time, users might experience errors while performing CRUD operations with the Container Registry. App Platform users may also experience build failures. Spaces users might experience errors while generating Spaces Access Keys.

We will provide an update as soon as further information is available.

Our Engineering team has continued to investigate the root cause of this issue. At this time, improvements and a potential fix are being deployed.

Due to the recurrences of this issue, we'll be monitoring the situation longer than usual to ensure the fix is effective in remediating the root cause.

We believe the following incidents may have the same root cause/were recurrences:
https://status.digitalocean.com/incidents/7zwkwckjwkvj
https://status.digitalocean.com/incidents/r4c3gbqypx7n
https://status.digitalocean.com/incidents/xd9fjy43fs7f
https://status.digitalocean.com/incidents/bwt457lhw66l

We will post updates here if this issue recurs, otherwise, we will post an update by 17:00 UTC tomorrow, November 3.

We appreciate your patience and understanding. If you have any questions or concerns in the meantime, please open a support ticket from within your account.

Our Engineering team has taken action to remediate impact from this issue. As of 15:31 UTC, error rates have subsided and Spaces, Container Registry, and App Platform users should no longer face impact from this incident.

Our Engineering team continues to investigate the root cause and solutions to avoid recurrence of this issue.

We are monitoring the situation closely to ensure stability, and will post another update as soon as we have further information.

As of 14:56 UTC, our engineers have observed recurring problems with the Spaces, Container Registry, and App platform products. More details about the original incident can be found here:
https://status.digitalocean.com/incidents/bwt457lhw66l

At this moment, users may experience errors when interacting with Spaces. There may be issues while performing CRUD operations with the Container Registry. App Platform users may also experience build failures. We are actively working to find a quick resolution for these issues and we will post an update as soon as additional information is available.