Outage in Scaleway

[K8S] CoreDNS timeouts

Resolved Minor
February 15, 2024 - Started 3 months ago - Lasted 5 days
Official incident page

Need to monitor Scaleway outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including Scaleway, and never miss an outage again.
Start Free Trial

Outage Details

Some nodes experience up to 10% DNS resolution failures with I/O timeouts to 169.254.53.53:53. The issue is identified, we are working on a fix
Latest Updates ( sorted recent to last )
RESOLVED 2 months ago - at 02/20/2024 03:49PM

The DNS resolution failure has been resolved. To fix the issue, please proceed as follows:
Replace or reboot your nodes to get the fix.
Use a fqdn to resolve your addresses inside the Private Network. Meaning if you have a foo service in the mypn pn foo will not be resolved and you must use foo.mypn
Important notes:
Please avoid using an existing TLD (https://en.wikipedia.org/wiki/Top-level_domain) as PN name if you do so you can have some problem. For instance, if you name your pn fr you will not be able to resolve google.fr externally, meaning if you have not a service named google in your PN you won’t revsolv google.fr and if you have one, it will not be the good one.
Please note that prod and dev are TLD (https://data.iana.org/TLD/tlds-alpha-by-domain.txt)”

INVESTIGATING 3 months ago - at 02/16/2024 08:30AM

We are still working on a permanent fix for this issue.
The issue can be temporarily mitigated by replacing forward . /etc/resolv.conf by forward . 169.254.169.254 for clusters using private networks and forward . [public resolver of choice] for multicloud and legacy public clusters in the ConfigMap kube-system/coredns

INVESTIGATING 3 months ago - at 02/15/2024 10:19AM

We are continuing to investigate this issue.

INVESTIGATING 3 months ago - at 02/15/2024 10:00AM

Some nodes experience up to 10% DNS resolution failures with I/O timeouts to 169.254.53.53:53.

The issue is identified, we are working on a fix

Monitor outages in real-time, stay one step ahead

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 3155 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook

Setup in 5 minutes or less

How much time you'll save your team, by having the outages information close to them?

14-day free trial · No credit card required · Cancel anytime