DigitalOcean's SFO3 region experienced increased error rates from 02:53 to 04:32 UTC, affecting the ability to create and manage Droplets, Volumes, Load Balancers, Reserved IPs, Cloud Firewalls, Managed Databases, and Kubernetes clusters via the API and Control Panel, with some users also facing connectivity issues with Load Balancers and newly created or restarted Droplets. The root cause was a network routing issue that disrupted connectivity to a core database system after a load balancer node became unavailable. The issue was mitigated by 04:32 UTC and fully resolved by 18:04 UTC, with failed operations requiring a manual retry.
Trusted by 1,000+ teams
Stop finding out about outages from your users. Monitor 6,320+ cloud services and get alerted the second something breaks.
Our Engineering team has confirmed that the increased error rates in the SFO3 region are resolved.
From 02:53 UTC to 04:32 UTC, users may have experienced errors, timeouts, or delays when creating or managing Droplets, Volumes, Load Balancers, Reserved IPs, Cloud Firewalls, Managed Databases, and Managed Kubernetes clusters through the API or Cloud Control Panel. Some users also experienced connectivity issues with Load Balancers and with newly created or restarted Droplets.
The impact was caused by a network routing issue that disrupted connectivity to one of our core database systems after a load balancer node became unavailable. Services have been stable since mitigation. Operations that failed during the incident may need to be retried.
We apologize for the inconvenience. If you continue to experience issues, please open a support ticket from your DigitalOcean account.
Our Engineering team has mitigated the issue, and services in the SFO3 region are currently operating normally. The team is continuing to monitor the situation to ensure stability given the scope of the impact.
We will continue monitoring and share another update once the situation is fully resolved. In the meantime, if you encounter any further errors or timeouts, please retry the affected operations. If the issue persists, please submit a ticket through your Cloud Support page so our team can assist you further: https://cloudsupport.digitalocean.com/s/createticket
Our Engineering team has mitigated the increased error rates impacting multiple services in the SFO3 region and is monitoring the situation. The impact was caused by a network routing issue that disrupted connectivity to one of our core database systems after a load balancer node became unavailable.
As of 04:32 UTC, users should no longer experience errors or timeouts when creating or managing resources in SFO3. Operations that failed during the incident may need to be retried.
We will post an update as soon as the issue is fully resolved.
As of 02:53 UTC, our Engineering team is investigating increased error rates impacting multiple services in the SFO3 region. At this point, users with resources in SFO3 may experience errors, timeouts, or delays when creating or managing Droplets, Volumes, Load Balancers, Reserved IPs, Cloud Firewalls, Managed Databases, and Managed Kubernetes clusters through the API or Cloud Control Panel. Some users may also experience connectivity issues with Load Balancers and with newly created or restarted Droplets.
We apologize for the inconvenience and will share an update once we have more information.
With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.
Start free trialNo credit card required · Cancel anytime · 6320 services available
Integrations with