Outage in Fly.io

Consul/Nomad outage interfering with app instance scheduling

Resolved Minor
October 27, 2022 - Started almost 3 years ago - Lasted 1 day
Official incident page

Need to monitor Fly.io outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including Fly.io, and never miss an outage again.
Start Free Trial

Outage Details

We are currently investigating an issue with consul and nomad, which is causing issues with deployment monitoring. Deployments are still happening.
Latest Updates ( sorted recent to last )
MONITORING almost 3 years ago - at 10/28/2022 02:30PM

We have cleaned up most lingering issues from the Consul/Nomad outage and are monitoring to ensure that peoples' apps recover properly.

IDENTIFIED almost 3 years ago - at 10/28/2022 01:42PM

We are still working on this issue. Deploys, restarts, and rescheduling of apps are all currently affected by this.

IDENTIFIED almost 3 years ago - at 10/28/2022 11:00AM

We are having continued issues with our consul and nomad clusters. We are working on restoring them.

MONITORING almost 3 years ago - at 10/28/2022 07:29AM

We've repaired our Nomad and Consul clusters and are working through a backlog of queued operations. Deployments and restarts will continue to be delayed ~5-30m until the queue is empty.

INVESTIGATING almost 3 years ago - at 10/28/2022 03:01AM

Deployment monitoring is erroring in most cases. Deployments themselves are working through a queue. It's currently taking upwards of 30 minutes for deploys to take effect.

INVESTIGATING almost 3 years ago - at 10/28/2022 02:16AM

We are currently investigating an issue with consul and nomad, which is causing issues with deployment monitoring. Deployments are still happening.

Latest Fly.io outages

Intermittent 500 errors on the dashboard - about 15 hours ago
Registry pulls using a local docker client return an error - 5 days ago
Elevated Packet Loss - 6 days ago
Intermittent registry errors - 6 days ago
Grafana log search not loading - 7 days ago

Be the First to Know When Vendors Go Down

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 4522 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook