Networking and Downstream services in NYC1, NYC2, FRA1, TOR1

Our Engineering team has implemented a fix for the issue impacting multiple services in NYC1, NYC2, FRA1, TOR1. Our team completed the Networking maintenance in these regions and is currently monitoring the situation.

Thank you for your patience, and we will post an update as soon as the issue is fully resolved.

Our team is continuing work to complete maintenance in NYC2 and FRA1 . Maintenance in NYC1 and TOR1 is complete. We are not currently seeing any impact from the identified root cause, but there could be certain impact as mentioned in previous updates.

We are working as quickly as possible and will provide updates as necessary. Thank you.

Our team is continuing work to complete maintenance in TOR1 and NYC2. We are starting to work on FRA1. NYC1 is complete. We are not currently seeing any impact from the identified root cause, but there could be certain impact as mentioned in the previous update.

We will share an update as soon as we have more information.

Our team is continuing work to complete maintenance in TOR1. NYC1 is complete. We are not currently seeing any impact from the identified root cause, but until all regions have undergone emergency maintenance, recurrences of the networking and downstream service issue are possible in regions which have not had maintenance completed.

We are working as quickly as possible and will provide updates as necessary. Thank you.

Our team has completed maintenance in NYC1 at this time. We are seeing recurrence of the same issue in NYC2, TOR1, and FRA1, due to the same root cause and users may experience errors as noted in previous updates. We have confirmed that the completed maintenance in NYC1 mitigated the issue recurring there, so NYC1 is not impacted. Our team is now working on maintenance for TOR1.

Our Engineering team is now beginning emergency maintenance in NYC1, NYC2, FRA1, and TOR1 to implement a long-term fix for the multiple issues referenced in this incident. As this work progresses, we do not expect user impact to networking connectivity with regards to Droplet and Droplet-based products (such as Managed Kubernetes and Databases).

We do not have a firm ETA on completing maintenance but will post updates as we have more information.

As of 00:56 UTC, we saw a recurrence of this incident, impacting users similarly to our previous updates. We are now seeing recovery again for services and the root cause has been identified.

Our Engineering team is working to implement a long-term fix to ensure continued stability. Thank you for your patience and we will post another update as soon as we have further details.

Our Engineering team is currently investigating a recurrence of this issue and users may once again see issues with multiple products and services as noted in our last updates.

We apologize and will provide another update as soon as we have more information.

Our Engineering team has identified the root cause of the multiple failures and a fix has been implemented. To add to our earlier update, we've confirmed this incident impacted Droplet networking as well and users may have experienced packet loss/latency and loss of connectivity with Droplets in NYC1, NYC2, FRA1, and TOR1, including Managed Kubernetes.

At this time, users should see error rates subsiding and normal connectivity returning.

Thank you for your patience.

As of approximately 23:40 UTC, our Engineering team is investigating multiple failures with DigitalOcean services, including our Cloud Control Panel, API, events such as creation of all resources, and WWW endpoints failing to resolve. We are starting to see recovery but are still investigating root cause and users may continue to experience elevated errors.

We will post another update as soon as further information is available.