App deployment degradation

Deployments of Forge apps in all EU cells are confirmed to be proceeding normally. We are now setting this issue as resolved.

Timeline
2024-10-22 08:58 UTC: deployments in one EU cell began to fail intermittently
2024-10-22 09:48 UTC: we moved deployments off the impacted cell, which resolved the issue
2024-10-23 00:25 UTC: all EU cells are confirmed to be operating normally

If you deployed apps between 2024-10-22 08:55 UTC to 2024-10-23 00:25 UTC and require them to deploy to the eu-west-1 region, we recommend deploying those apps again.

Between 08:55 UTC and 9:50 UTC on 2024-10-22 we have observed increased error rate on app deployments.

After investigation we found an condition in one of cells in EU region which was causing deployments to fail.

We applied change to migrate new deployments off the impacted EU cell which mitigated impact - no new errors were since 9:50 UTC.

We are keeping this incident in monitoring state and will resolve it once underlying issue is resolved and deployments are resumed to the affected cell.

From perspective of App developers this means that deployments of apps (new apps and updates to existing ones) will work correctly but may introduce minor performance degradation due to function being invoked from nearest EU cell when customer is located in the affected cell. Because of this we will post an update advising to re-deploy affected apps once this incident is resolved.