Elevated error rates for deployments with volumes on Southeast Asia (Singapore) Metal region

We are currently investigating elevated error rates of deployments in Southeast Asia (Singapore) Metal. A small subset of services with volumes may be failing to deploy with the error message Container failed to start

We have identified a single degraded host on Southeast Asia (Singapore) Metal. We are currently restoring active deployments without a volume on the host to a different host. Workloads with a volume located on the host may be unavailable during this period. Impacted users will see a deployment initiated by Railway as part of this restoration process.

All workloads with a volume on the impacted host is back online. All other workloads have been successfully moved to a different host. We are actively monitoring the recovery. If your deployment failed due to “Container failed to start error, please attempt a re-deploy.