ns2.rrpproxy.net and ns2.dnsres.net affected by network outage

Update

As already posted in our last update we have set up new IP addresses replacing the systems that were affected by the incident in the data center last Wednesday, March 10.

The new systems are up and running at their new permanent location. We continue to monitor the situation. For future adjustments we will inform appropriately.

Update on Nameservers:

We finished setting up new IP addresses at their new permanent location for ns2.rrpproxy.net and ns2.dnsres.net:

ns2.rrpproxy.net: IPv4 66.206.3.124 / IPv6 2604:4500:c:3:66:206:3:124
ns2.dnsres.net: IPv4 66.206.3.126 / IPv6 2604:4500:c:3:66:206:3:126

Also, we finished setting up a new IP address for our nameserver providing DNS Zone Xfer IN/OUT at its new permanent location:

ns2-eu.key-systems.net: IPv4 66.206.25.194 / IPv6 2604:4500:c:3:66:206:25:194

Note: Please make sure this IP address can connect to your DNS server (Master, Hidden Master or Slave) for outgoing/incoming DNS Zone transfers (notify/AXFR). I.e. respective firewall rules need to be set up.

All IP addresses are live as of now. Please be aware that depending on TTL setting and caching it may take a while until these new NS IP addresses are populated.

Documentation will be adjusted once all work has been finalised.

Update on Web Forwarding:

Meanwhile all Domains using Web Forwarding have been adjusted and are now using our new IP addresses:

IPv4 66.206.3.38
IPv6 2604:4500:c:4:66:206:3:38

Short update:

We're still working on bringing back systems to normal state.In particular, we're busy setting up new nodes for ns2.rrpproxy.net and ns2.dnsres.net and will communicate new IP addresses as soon as they are available. Until then the communicated IP addresses for our NS serve as replacement.

For Web Forwarding scripts are still running to update affected domains.

Update on Web Forwarding:

We deployed a new Web Forwarding set and are currently adjusting domains. The new IP addresses are:

IPv4
old: 51.91.60.4 --> new: 66.206.3.38

IPv6
old: 2001:41d0:301:100:51:91:60:4 --> new: 2604:4500:c:4:66:206:3:38

The scripts to adjust Web Forwarding for affected domains are running. We expect to have all domains adjusted in a few hours.

Update:

ns2.dnsres.net came back online at its location in the data center having an incident. As a consequence, it answers on queries.

However, we will keep the hostname switched to the IP address communicated in the previous update and will switch back when we're confident the issues at the data center have been resolved.

Update on NS:

ns2.rrpproxy.net came back online at their location in the data center having an incident. As a consequence, they answer on queries.
However, we will keep the hostname switched to the IP address communicated in the previous update and will switch back when we're confident the issues at the data center have been resolved.

EDIT: removed information about ns2.dnsres.net; only ns2.rrpproxy.net came back online

Update:

As a workaround we're switching our Nameserver IP addresses for ns2.rrpproxy.net and ns2.dnsres.net as follows:

ns2.rrpproxy.net from:
IPv4 178.33.34.29
IPv6 2001:41d0:d:7ea:178:33:34:29
to:
IPv4 51.68.180.207
IPv6 2001:41d0:700:1bcf:51:68:180:207

ns2.dnsres.net from:
IPv4 188.165.164.171
IPv6 2001:41d0:d:7ea:188:165:164:171
to:
IPv4 51.75.87.7
IPv6 2001:41d0:700:1bcf:51:75:87:7

Please regard that depending on TTL setting and caching it may take a while until new NS IP addresses are populated.

Update: Our Web Forwarding is affected as well and currently unavailable.
We continue to work on fixing these issues.

Unfortunately the network reachability of both ns2.rrpproxy.net and ns2.dnsres.net is affected by a major incident in one data center location. We are in contact with the responsible network team of the data center to get the traffic for these systems rerouted to a different data center location.

The data center team is still working on resolving the issues. Unfortunately we cannot provide any ETA at this point of time.

The data center team is already working on resolving the issues with all resources available.

ns2.rrpproxy.net and ns2.dnsres.net are currently unavailable due to a large network outage in the backing data center.