Connectivity outage in London data center

This incident has been resolved.

Connectivity is now fully restored. We are still seeing some very minor issues but expect these to get resolved shortly. They are not likely to be noticable to any end users.

We are continuing to work on fixing this issue.

The connectivity is redudant but there are problems with both the primary and secondary fiber link providers currently.

Customers on ACD, CDN or Cloudflare should have working sites as we have been able to re-route most of it.

IPv6 traffic should also be restored. If you are NOT using ACD or CDN you should make sure your DNS is set up with the AAAA / IPv6 records shown in the control panel. This should improve the reachability of your site. But DO NOT remove the A / IPv4 record.

Most outbound API traffic to major providers has been restored as well.

There are still issues with reaching SSH/SFTP, and changing things in the control panel do not work for sites in London.

Reachability is partial, some clients can reach London while others cannot. Data center networks appears to be more affected than access networks. We are working on a fix.

We are currently investigating this issue.