Elevated Google Calendar errors

API success rates have remained stable since our previous message, and Google have resolved their own incident: https://www.google.com/appsstatus/dashboard/incidents/oFcAZTr4EVieF5Fr6Ee9

Between the times of 14:57 UTC - 15:35 UTC, we saw elevated rates of errors, and Google Calendar users may have had slight delays to calendar reads and writes. Since 15:35 UTC everything has been functioning as normal.

Google's API error rate has dropped again and held at a normal level for the past 10 minutes, and both read and write operations to Google Calendar profiles appear to be back to our normal level of performance.

We'll continue to monitor and confirm this holds out.

Google is returning a higher rate of 5xx errors when we try to perform token refresh operations. Approximately 1/3rd of our attempts to refresh tokens are seeing this error.

The increase in errors started approximately 14:57 UTC.

It appears our normal mechanisms are allowing retries to be processed successfully, and limiting the impact to the wider service.

Updates to and from Google Calendar profiles may see slight delays during this time.

Google appear to have acknowledged the incident: https://www.google.com/appsstatus/dashboard/incidents/oFcAZTr4EVieF5Fr6Ee9

We've observed an elevated rate of errors from Google's Calendar API and are investigating the scale and impact.

Updates to and from Google Calendar profiles may see slight delays during this time.

We are monitoring the situation.