Modal Labs Outage History

Modal Labs experienced degraded container creation for both CPU and GPU functions due to a problematic change. The issue was resolved by rolling back the change, with recovery observed after approximately 2 hours.

Modal Labs experienced increased latency affecting web endpoints for 2 hours. Some requests to web endpoints were impacted by degraded response times during this period. The incident was classified as minor and has been resolved.

Modal Labs experienced degraded performance on their web dashboard for 40 minutes. The frontend was affected, causing metrics to load slower than normal. The incident was classified as minor and has been resolved.

Modal Labs experienced degraded performance on web endpoints in the us-east region, where some requests proxied through the input plane were failing. The incident affected web endpoint functionality for 30 minutes before being resolved.

Modal Labs experienced a major outage affecting their image builds service for 36 minutes. The incident caused image builds to go down completely, preventing users from building container images. The service was restored after 36 minutes of downtime.

Modal Labs experienced delayed task logs starting at 2:46 PM ET, affecting the Web API component. The service disruption lasted approximately 1.4 hours before being resolved. This was classified as a minor incident impacting users' ability to access timely logging information for their tasks.

Modal Labs experienced a major outage affecting GPU functions, Volumes, Sandboxes, and Image builds for 16 minutes. The primary issue was with Image builds going down, which impacted multiple core services. The incident has been resolved.

Modal Labs experienced a major incident where a fraction of new containers encountered errors, affecting CPU functions, GPU functions, web endpoints, sandboxes, and image builds. The service disruption lasted approximately 129.6 hours (about 5.4 days) before all issues were fully resolved.