365 synchronization errors

The performance of syncing Microsoft 365-backed calendars was degraded for about 90 minutes.

Microsoft 365 began returning an elevated number of errors and timing out just after 07:05 UTC until 08:30 UTC. This was part of a wider issue within Azure, where Microsoft 365 is hosted.

Steps were taken to increase capacity and be more aggressive in timing out connections to Microsoft 365 to mitigate the impact on background processing. The sudden change to the volume of background jobs did lead to a slight delay in processing being present for much of the issue.

Normal service resumed a few minutes after the Microsoft 365 APIs began to operate normally, once a surge of calendar update notifications was processed.

We will be reviewing our mechanisms for reducing the side effects of such incidents in the future.

Error rates for Microsoft 365-backed calendars have returned to normal levels, and we have processed a surge in calendar updates.

We will continue to monitor the service but believe this issue to now be resolved.

Microsoft have sent a 365 health notification (MO502273) relating to some users being unable to access multiple Microsoft 365 services. This is likely related to the issue we are seeing.

Errors for 365-backed calendar continue to be elevated.

We are seeing a higher than usual number of errors affecting calendars hosted on Microsoft 365 across all our data centers for connections using Exchange Web Services (EWS) and Microsoft's Graph API.

Background processing has scaled up to compensate but may be degraded as a result.