US data center scaling problems

Our US data center experienced degraded performance today, primarily impacting background processing.

There was an issue with the Amazon Elastic Kubernetes Service (EKS) cluster which affected our ability to scale capacity with demand.

This was resolved by AWS intervening to resolve an issue with our EKS cluster. We will be receiving a RCA from AWS relating to this.

Once we receive AWS's RCA, we will follow our own process to publish a postmortem of this incident.

If you have any queries in the interim, please contact us at support@cronofy.com

A fix has been implemented in conjunction with AWS support and our ability to scale our resources has returned.

Capacity has then been able to increase to match demand and service levels appear to have returned to normal.

We are continuing to monitor to ensure service remains stable.

We are working through an issue which is preventing our US data center from scaling as normal.

Performance of background processing has been degraded somewhat through the past hour with processing taking up to 60 seconds to be started rather than being near instant.

We are in contact with AWS to resolve the underlying problem.

We are working through an issue which is preventing our US data center from scaling as normal.

Performance of background processing was degrading somewhat through our peak time around 17:00 UTC with processing taking up to 60 seconds to be started rather than being near instant.

As we are now past that peak, we anticipate any impact on background processing to be reduced but still possible until scaling operations return to normal.

We are working through an issue which is preventing our US data center from scaling as normal. Performance may be degraded as we approach our peak time (around 17:00 UTC) until this is resolved.

This will primarily affect background processing times, with synchronization potentially taking longer than usual.