[USA] Potential impact due to AWS outage

We are aware of global outages impacting AWS infrastructure on which AlayaCare solutions are hosted. This is a pre-emptive alert to let you know that there is a possibility that parts of the AlayaCare solution(s) you are using may be impacted by these outages.

We are monitoring closely and will provide further updates as they are available.

We are aware of global outages impacting AWS infrastructure on which AlayaCare solutions are hosted. This is a pre-emptive alert to let you know that there is a possibility that parts of the AlayaCare solution(s) you are using may be impacted by these outages.

We are monitoring closely and will provide further updates as they are available.

The most recent update from Amazon regarding AWS Services will be shared here as well -
"We have identified a potential root cause for error rates for the DynamoDB APIs in the US-EAST-1 Region. Based on our investigation, the issue appears to be related to DNS resolution of the DynamoDB API endpoint in US-EAST-1. We are working on multiple parallel paths to accelerate recovery. This issue also affects other AWS Services in the US-EAST-1 Region. Global services or features that rely on US-EAST-1 endpoints such as IAM updates and DynamoDB Global tables may also be experiencing issues. We recommend customers continue to retry any failed requests. We will continue to provide updates as we have more information to share, or by 2:45 AM PST.

----------------------------------------------------------------------------------
Nous sommes informés de pannes mondiales affectant l'infrastructure AWS sur laquelle sont hébergées les solutions AlayaCare. Cette alerte préventive vous informe qu'il est possible que certaines parties des solutions AlayaCare que vous utilisez soient affectées par ces pannes.

Nous suivons la situation de près et vous tiendrons informés dès que possible.

La dernière mise à jour d'Amazon concernant les services AWS sera également partagée ici :
« Nous avons identifié une cause potentielle expliquant les taux d'erreur des API DynamoDB dans la région US-EAST-1. D'après notre enquête, le problème semble être lié à la résolution DNS du point de terminaison de l'API DynamoDB dans la région US-EAST-1. Nous travaillons sur plusieurs solutions parallèles pour accélérer la récupération. Ce problème affecte également d'autres services AWS dans la région US-EAST-1. Les services ou fonctionnalités globaux qui dépendent des points de terminaison US-EAST-1, tels que les mises à jour IAM et les tables globales DynamoDB, peuvent également rencontrer des problèmes. Nous recommandons à nos clients de continuer à réessayer toute requête ayant échoué. Nous continuerons à vous tenir informés dès que nous aurons plus d'informations à partager, ou avant 2 h 45 (heure du Pacifique). »

We are aware of global outages impacting AWS infrastructure on which AlayaCare solutions are hosted. This is a pre-emptive alert to let you know that there is a possibility that parts of the AlayaCare solution(s) you are using may be impacted by these outages.

We are monitoring closely and will provide further updates as they are available.

The most recent update from Amazon regarding AWS Services will be shared here as well -
"We have applied initial mitigations and we are observing early signs of recovery for some impacted AWS Services. During this time, requests may continue to fail as we work toward full resolution. We recommend customers retry failed requests. While requests begin succeeding, there may be additional latency and some services will have a backlog of work to work through, which may take additional time to fully process. We are seeing significant signs of recovery. Most requests should now be succeeding. We continue to work through a backlog of queued requests. We will continue to provide additional information. We will continue to provide updates as we have more information to share, or by 3:15 AM PST (615 EST)"

----------------------------------------------------------------------------------

Nous sommes informés de pannes mondiales affectant l'infrastructure AWS sur laquelle sont hébergées les solutions AlayaCare. Cette alerte préventive vous informe qu'il est possible que certaines parties des solutions AlayaCare que vous utilisez soient affectées par ces pannes.

Nous suivons la situation de près et vous tiendrons informés dès que possible.

La dernière mise à jour d'Amazon concernant les services AWS sera également partagée ici :
« Nous avons appliqué les mesures d'atténuation initiales et observons des signes précoces de reprise pour certains services AWS impactés. Pendant ce temps, les requêtes pourraient continuer d'échouer, le temps que nous approchions d'une résolution complète. Nous recommandons à nos clients de réessayer les requêtes ayant échoué. Bien que les requêtes commencent à aboutir, une latence supplémentaire peut survenir et certains services auront du travail en attente, ce qui peut prendre plus de temps pour être entièrement traités. Nous constatons des signes significatifs de reprise. La plupart des requêtes devraient maintenant aboutir. Nous continuons de traiter les requêtes en attente. Nous continuerons de fournir des informations complémentaires. Nous vous tiendrons informés dès que nous aurons plus d'informations à partager, ou avant 3 h 15 PST (6 h 15 EST). »

We are aware of global outages impacting AWS infrastructure on which AlayaCare solutions are hosted. This is a pre-emptive alert to let you know that there is a possibility that parts of the AlayaCare solution(s) you are using may be impacted by these outages.

We are monitoring closely and will provide further updates as they are available.

The most recent update from Amazon regarding AWS Services will be shared here as well -
"We continue to observe recovery across most of the affected AWS Services. We can confirm global services and features that rely on US-EAST-1 have also recovered. We continue to work towards full resolution and will provide updates as we have more information to share."
----------------------------------------------------------------------------
Nous sommes informés des pannes mondiales affectant l'infrastructure AWS hébergeant les solutions AlayaCare. Cette alerte préventive vous informe qu'il est possible que certaines parties des solutions AlayaCare que vous utilisez soient affectées par ces pannes.

Nous suivons la situation de près et vous tiendrons informés dès que possible.

La dernière mise à jour d'Amazon concernant les services AWS sera également publiée ici :
« Nous continuons d'observer une reprise sur la plupart des services AWS concernés. Nous confirmons que les services et fonctionnalités mondiaux dépendant de US-EAST-1 sont également rétablis. Nous continuons de travailler à une résolution complète et vous tiendrons informés dès que nous aurons plus d'informations à partager. »

We are aware of global outages impacting AWS infrastructure on which AlayaCare solutions are hosted. This is a pre-emptive alert to let you know that there is a possibility that parts of the AlayaCare solution(s) you are using may be impacted by these outages.

We are monitoring closely and will provide further updates as they are available.

The most recent update from Amazon regarding AWS Services will be shared here as well -
"The underlying DNS issue has been fully mitigated, and most AWS Service operations are succeeding normally now. Some requests may be throttled while we work toward full resolution. Additionally, some services are continuing to work through a backlog of events such as Cloudtrail and Lambda. We continue to work toward full resolution. If you are still experiencing an issue, we recommend flushing your DNS caches. We will provide an update by 4:15 AM Pacific, or sooner if we have additional information to share."
-------------------------------------------------------------------------------
Nous sommes informés des pannes mondiales affectant l'infrastructure AWS hébergeant les solutions AlayaCare. Cette alerte préventive vous informe qu'il est possible que certaines parties des solutions AlayaCare que vous utilisez soient affectées par ces pannes.

Nous suivons la situation de près et vous tiendrons informés dès qu'elles seront disponibles.

La dernière mise à jour d'Amazon concernant les services AWS sera également partagée ici :
« Le problème DNS sous-jacent a été entièrement résolu et la plupart des opérations des services AWS fonctionnent désormais normalement. Certaines requêtes pourraient être limitées pendant que nous travaillons à sa résolution complète. De plus, certains services continuent de gérer un arriéré d'événements, comme Cloudtrail et Lambda. Nous continuons de travailler à sa résolution complète. Si le problème persiste, nous vous recommandons de vider vos caches DNS. Nous vous tiendrons informés d'ici 4 h 15 (heure du Pacifique), ou plus tôt si nous avons des informations supplémentaires à vous communiquer. »

We are aware of global outages impacting AWS infrastructure on which AlayaCare solutions are hosted. This is a pre-emptive alert to let you know that there is a possibility that parts of the AlayaCare solution(s) you are using may be impacted by these outages.

We are monitoring closely and will provide further updates as they are available.

The most recent update from Amazon regarding AWS Services will be shared here as well -
"We are continuing to work towards full recovery for EC2 launch errors, which may manifest as an Insufficient Capacity Error. Additionally, we continue to work toward mitigation for elevated polling delays for Lambda, specifically for Lambda Event Source Mappings for SQS. We will provide an update by 5:00 AM PDT."

-------------------------------------------------------------------------------
Nous sommes informés des pannes mondiales affectant l'infrastructure AWS hébergeant les solutions AlayaCare. Cette alerte préventive vous informe qu'il est possible que certaines parties des solutions AlayaCare que vous utilisez soient affectées par ces pannes.

Nous suivons la situation de près et vous tiendrons informés dès qu'elles seront disponibles.

La dernière mise à jour d'Amazon concernant les services AWS sera également publiée ici :
« Nous continuons de travailler à une récupération complète des erreurs de lancement EC2, qui peuvent se manifester par une erreur de capacité insuffisante. De plus, nous continuons de travailler à l'atténuation des délais d'interrogation élevés pour Lambda, notamment pour les mappages de sources d'événements Lambda pour SQS. Nous vous fournirons une mise à jour d'ici 5 h 00 HAP. »

We are aware of global outages impacting AWS infrastructure on which AlayaCare solutions are hosted. This is an alert to let you know that there is a possibility that parts of the AlayaCare solution(s) you are using may be impacted by these outages.

We are monitoring closely and will provide further updates as they are available and will confirm when the AWS outage is fully resolved.

The most recent updates from Amazon regarding AWS Services can be viewed directly at https://health.aws.amazon.com/health/status?path=service-history

----------------------------------------------------------------------------
Nous sommes informés des pannes mondiales affectant l'infrastructure AWS sur laquelle sont hébergées les solutions AlayaCare. Cette alerte vous informe qu'il est possible que certaines parties des solutions AlayaCare que vous utilisez soient affectées par ces pannes.

Nous suivons la situation de près et vous tiendrons informés dès qu'elles seront disponibles. Nous vous confirmerons la résolution complète de la panne AWS.

Les dernières mises à jour d'Amazon concernant les services AWS sont disponibles directement à l'adresse suivante : https://health.aws.amazon.com/health/status?path=service-history

We are aware of global outages impacting AWS infrastructure on which AlayaCare solutions are hosted. This is an alert to let you know that there is a possibility that parts of the AlayaCare solution(s) you are using may be impacted by these outages.

We are monitoring closely and will provide further updates as they are available and will confirm when the AWS outage is fully resolved.

The application response time is slowly recovering, but it still higher than usual.

We will post another update in 30 minutes.

The most recent updates from Amazon regarding AWS Services can be viewed directly at https://health.aws.amazon.com/health/status?path=service-history

We are aware of global outages impacting AWS infrastructure on which AlayaCare solutions are hosted. This is an alert to let you know that there is a possibility that parts of the AlayaCare solution(s) you are using may be impacted by these outages.

We are monitoring closely and will provide further updates as they are available and will confirm when the AWS outage is fully resolved.

The application response time is slowly recovering, but it still higher than usual.

AWS is observing a substantial improvement in successful EC2 launches in one of the affected Availability Zones after making a change. They are now validating this change and will proceed with applying it to the remaining two affected Availability Zones.

The most recent updates from Amazon regarding AWS Services can be viewed directly at https://health.aws.amazon.com/health/status?path=service-history

We are aware of global outages impacting AWS infrastructure on which AlayaCare solutions are hosted. This is an alert to let you know that there is a possibility that parts of the AlayaCare solution(s) you are using may be impacted by these outages.

We are monitoring closely and will provide further updates as they are available and will confirm when the AWS outage is fully resolved.

AWS is observing a substantial improvement in successful EC2 launches in one of the affected Availability Zones after making a change. They are now validating this change and will proceed with applying it to the remaining two affected Availability Zones.

We will provide another check-in at 11AM EST.

The most recent updates from Amazon regarding AWS Services can be viewed directly at https://health.aws.amazon.com/health/status?path=service-history

We have applied additional mitigations to reduce the impact of the ongoing AWS outage. AlayaCare Cloud should now be available for general use.

Please note that AWS problems have not been fully resolved and there is a potential that you may experience service interruptions.

We are monitoring closely and will provide further updates as they are available and will confirm when the AWS outage is fully resolved.

We will provide another update by 12PM EST.

The most recent updates from Amazon regarding AWS Services can be viewed directly at https://health.aws.amazon.com/health/status?path=service-history

We are continuing to monitor the changes applied to reduce the impact of AWS outage with AlayaCare Cloud. We will confirm when the AWS outage is fully resolved.

We will provide another update by 2PM EST if not before.

The most recent updates from Amazon regarding AWS Services can be viewed directly at https://health.aws.amazon.com/health/status?path=service-history

We continue to monitor the changes that were applied to reduce the impact of AWS outage with AlayaCare Cloud. We will confirm when the AWS outage is fully resolved.

We will provide another update by 4PM EST if not before.

The most recent updates from Amazon regarding AWS Services can be viewed directly at https://health.aws.amazon.com/health/status?path=service-history

We are continuing to monitor the changes that were applied to reduce the impact of AWS outage with AlayaCare Cloud. We will confirm when the AWS outage is fully resolved.

We will provide another update by 5PM EST if not before.

The most recent updates from Amazon regarding AWS Services can be viewed directly at https://health.aws.amazon.com/health/status?path=service-history

As AWS continues to finalize a full recovery across all services, the most recent updates from Amazon regarding AWS Services can be monitored directly at https://health.aws.amazon.com/health/status?path=service-history

We will provide another update by 9AM EST, if not before.

As per AWS' latest update, the global outage incident has reached a point of resolution. We will keep this status in a monitoring state until AWS shares a AWS post-event summary.

We are continuing to monitor the changes that were applied to reduce the impact of AWS outage with AlayaCare Cloud.

https://health.aws.amazon.com/health/status

In the wake of the AWS outage we have been running a internal assessment that indicates that most reported impacts were related to system performance degradation rather than service failures.

The primary confirmed exceptions are within the Data Explorations and EVV domains.
By 3pm EST, a full data refresh will be run for all tenants observing impacted DE 2.0 functionality.

API services are now stable, and background processing has fully recovered, though some temporary delays may have occurred during the recovery period. APIs and background processing was observed to have recovered around noon EST on the 20th.

We will continue to monitor data services connected to Snowflake, as residual issues in this area also impacted payroll operations.

A comprehensive root cause analysis is underway, and detailed findings will be shared once the investigation is complete, supplemented by AWS' official post-event summary.