Fly.io Outage History

Fly.io experienced elevated GraphQL API latency that caused delays and errors when creating apps and accessing dashboard pages. The incident affected the Dashboard component and lasted 49 minutes. The issue was resolved after implementing a fix and monitoring the results.

Fly.io's EWR region experienced low host capacity for 1.6 hours, causing failures when creating new machines or starting stopped/suspended machines. Users were advised to use nearby regions like ORD as a workaround while new host capacity was being provisioned. The incident was resolved after additional capacity was successfully added to the region.

Fly.io experienced degraded networking performance in their BOM and SJC (Sunnyvale, California) regions for approximately 46 hours. The network performance issues affected services in these locations before being restored. The incident was resolved after monitoring confirmed the restoration of normal network performance.

Fly.io experienced an IPv6 connectivity outage affecting some machines in their Singapore (SIN) region for 2.2 hours. The incident was classified as minor and impacted IPv6 network connectivity for affected instances. A fix was implemented and monitoring confirmed the resolution of the connectivity issues.

Fly.io experienced network issues in the Singapore region for 1.5 hours, causing apps and Managed Postgres clusters to become unreachable or experience higher latency. The upstream provider implemented a fix, with ingress traffic temporarily re-routed to other regions, resulting in continued higher latency for users in the Singapore area during recovery. The incident was fully resolved after monitoring confirmed all services were accessible.

Fly.io experienced an IPv6 networking failure affecting some machines in their Chicago (ORD) data center for 2.8 hours. The outage impacted applications relying on IPv6 connectivity in that region. The incident was resolved after working with upstream network providers to restore IPv6 functionality.

Fly.io experienced networking issues in their Singapore (SIN) region lasting 16.2 hours, caused by high packet loss and latency from an upstream provider. The incident affected machines with high packet loss and made some MPGs (Multi-Process Groups) unreachable due to inability to connect to their config store. The issue was resolved after working with the upstream provider to restore normal connectivity.

Fly.io experienced networking issues affecting new machines with assigned egress IP addresses in the Sydney (SYD) region for 3 hours. The incident impacted connectivity for newly deployed applications that required dedicated outbound IP addresses. The issue was identified, fixed, and resolved after a monitoring period to ensure stability.

The Fly.io dashboard experienced loading failures that prevented users from accessing the interface, affecting both the dashboard and deployments functionality. The incident lasted 1.3 hours while the team investigated, identified, and implemented a fix. The issue was resolved after a brief monitoring period to ensure the fix was effective.

Fly.io experienced a major incident affecting logs and metrics in the IAD region, where new logs and metrics from machines were missing while past data remained accessible. Applications continued to run normally throughout the incident. The issue was resolved after 1.7 hours.