Outage in Azure

Azure customers running Canonical Ubuntu 18.04 experiencing DNS errors

Resolved Minor
August 30, 2022 - Started about 1 year ago - Lasted 1 day

Need to monitor Azure outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including Azure, and never miss an outage again.
Start a Free Trial

Outage Details

Starting at approximately 06:00 UTC on 30 Aug 2022, a number of customers running Ubuntu 18.04 (bionic) VMs who have Ubuntu Unattended-Upgrades enabled would receive systemd version 237-3ubuntu10.54. A bug in this version will lead to DNS resolution errors. Reports of this issue are confined to this single Ubuntu version.This bug and a potential fix have been highlighted on the Canonical / Ubuntu website, which we encourage impacted customers to read: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1988119A potential workaround customers can consider is to reboot impacted VM instances so that they receive a fresh DHCP lease and new DNS resolver(s). If you are running a VM with Ubuntu 18.04 image, and you are experiencing connectivity issues, you can evaluate the above mitigation options.Our engineering team deployed an auto-remediation for AKS clusters in all regions and noticed a substantial progress with impacted clusters recovering to a healthy state. However, it was determined that there was a subset of cases where the AKS nodes were not covered by the auto-remediation detection and were not fixed as a result.For the Azure Public Cloud, our engineering team has completed the deploying of the first step of the fix to all the clusters across all regions worldwide. The second step has begun, which will recognize the impacted clusters and then execute the actual mitigation. Some of our customers should already see mitigation for their impacted components as we are making steady progress.For the Azure Government Cloud and Azure China Cloud, our engineering team is currently deploying the first step of the fix to all the clusters across all regions. Once the first step is completed, the second step will begin, which will detect unhealthy clusters and execute the actual mitigation. We are making steady progress on this deployment and will continue to share an update.Also please note, the offending package Ubuntu updates have been removed until further investigation is completed.The next update will be provided in 6 hours, or as events warrant.

Monitor Azure and all your third-party dependencies in one place

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 2761 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook

Setup in 5 minutes or less

Try it out! How much time you'll save your team, by having the outages information close to them?

  • 14-day free trial
  • No credit card required to start
  • Cancel anytime
  • +2500 services available