Azure customers running Canonical Ubuntu 18.04 experiencing DNS errors

Starting at approximately 06:00 UTC on 30 Aug 2022, a number of customers running Ubuntu 18.04 (bionic) VMs who have Ubuntu Unattended-Upgrades enabled would receive systemd version 237-3ubuntu10.54. A bug in this version will lead to DNS resolution errors. Reports of this issue are confined to this single Ubuntu version.This bug and a potential fix have been highlighted on the Canonical / Ubuntu website, which we encourage impacted customers to read: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1988119A potential workaround customers can consider is to reboot impacted VM instances so that they receive a fresh DHCP lease and new DNS resolver(s). If you are running a VM with Ubuntu 18.04 image, and you are experiencing connectivity issues, you can evaluate the above mitigation options.Our engineering team deployed an auto-remediation for AKS clusters in all regions and noticed a substantial progress with impacted clusters recovering to a healthy state. However, it was determined that there was a subset of cases where the AKS nodes were not covered by the auto-remediation detection and were not fixed as a result.For the Azure Public Cloud, our engineering team has completed the deploying of the first step of the fix to all the clusters across all regions worldwide. The second step has begun, which will recognize the impacted clusters and then execute the actual mitigation. Some of our customers should already see mitigation for their impacted components as we are making steady progress.For the Azure Government Cloud and Azure China Cloud, our engineering team is currently deploying the first step of the fix to all the clusters across all regions. Once the first step is completed, the second step will begin, which will detect unhealthy clusters and execute the actual mitigation. We are making steady progress on this deployment and will continue to share an update.Also please note, the offending package Ubuntu updates have been removed until further investigation is completed.The next update will be provided in 6 hours, or as events warrant.