Degraded Google calendar sync

Calendar sync for Google-backed calendars has remained healthy since the previous message, so we are considering this as resolved.

Google have updated their incident record of the underlying issue, where they likewise consider it resolved: https://www.google.com/appsstatus/dashboard/incidents/7uJZ5F1Uy4n1n74iMacQ

Error rates from Google's API have returned to normal levels, and calendar syncs for Google-backed calendars are healthy again.

Google Calendar sync performance dropped starting at 13:30 UTC, and was heavily degraded until improvements were seen starting at 16:00 UTC. A full recovery of service was reached around 16:10 UTC.

We're observing a much improved success rate from Google's Calendar API since 16:00 UTC.

Google have advised that a fix has been made on their side and is rolling out currently, and expect to be fully resolved within the hour: https://www.google.com/appsstatus/dashboard/incidents/7uJZ5F1Uy4n1n74iMacQ

We’ll continue to monitor the situation and update here.

Google’s Calendar API is still returning a high error rate, with no notable change to the situation since the previous update.

Google are tracking the incident at their incident dashboard
at https://www.google.com/appsstatus/dashboard/incidents/7uJZ5F1Uy4n1n74iMacQ

We’ll continue to monitor the situation and update here.

We continue to observe a high error rate on Google’s Calendar API. This appears to affect some user cohorts more than others, rather than being spread evenly across all of our syncronized Google calendars.

While this doesn’t affect the error rate of Cronofy’s API, it means that we’re failing to pick up the latest changes to affected users’ calendars, and may present stale availability information. Events written via the Cronofy API may be delayed before being pushed successfully to the external Google calendar. We queue and retry such operations automatically and do not anticipate any remedial action to be necessary as a result.

We'll continue to monitor things and update accordingly.

Errors when communicating with Google calendar increased significantly across all data centers from 13:31 UTC.

We have taken action to reduce the impact on other calendar providers and continue to monitor.

We are seeing a significantly higher than normal amount of errors when trying to interact with Google calendars.