Outage in Fly.io

Docker Hub Registry Timing Out: Fly Platform Impact

Resolved Minor
September 29, 2023 - Started 8 months ago - Lasted about 3 hours
Official incident page

Need to monitor Fly.io outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including Fly.io, and never miss an outage again.
Start Free Trial

Outage Details

Docker Hub has an ongoing outage with their registry: https://www.dockerstatus.com/pages/incident/533c6539221ae15e3f000031/6515dbc89fabff05350ae18d Deployments that use an image from the Docker Hub Registry—any unqualified FROM in a Dockerfile or setting an unqualified image name with `fly deploy --image`—may fail. Existing machines and allocs are not affected. Creating apps, machines, or allocs from images in the Fly Registry (registry.fly.io/APP_NAME) are working. Remote builders may fail to download the image from Docker Hub. We're working on a fix for that now. Try `fly deploy --local-only` if you experience remote builder issues as result of this issue. That will use a local docker environment on your system to build and push the image. That might work and work around the remote builder issue for now, though we understand local docker environments can be unreliable in certain cases (for example, building linux/amd64 images on macOS M1/M2 chips can seqfault with a known wontfix issue in qemu).
Latest Updates ( sorted recent to last )
RESOLVED 8 months ago - at 09/29/2023 05:30AM

The Docker Hub timing out issue has been marked resolved.

https://www.dockerstatus.com/pages/incident/533c6539221ae15e3f000031/6515dbc89fabff05350ae18d

MONITORING 8 months ago - at 09/29/2023 03:33AM

We updated our remote builders to load their images from docker-hub-mirror.fly.io/flyio/rchab, which is a caching mirror of registry-1.docker.io. We're seeing remote builders more reliably work.

Deployments that use an image from the Docker Hub Registry—any unqualified FROM in a Dockerfile or setting an unqualified image name with `fly deploy --image`—may still fail.

You can try adding docker-hub-mirror.fly.io/IMAGE_NAME to your Dockerfile or the `--image` flag on `fly deploy`. That will work more reliably if we have a cached copy of the image from Docker Hub.

The `fly deploy --local-only` work around is still a good option, if it works for you.

Another work around is finding another registry, like ghcr.io or quay.io or something like that, which has a copy of the image you would normally get from the Docker Hub Registry.

Marking this as monitoring while we wait for further updates from Docker Hub via their status page.

INVESTIGATING 8 months ago - at 09/29/2023 02:45AM

Docker Hub has an ongoing outage with their registry: https://www.dockerstatus.com/pages/incident/533c6539221ae15e3f000031/6515dbc89fabff05350ae18d

Deployments that use an image from the Docker Hub Registry—any unqualified FROM in a Dockerfile or setting an unqualified image name with `fly deploy --image`—may fail.

Existing machines and allocs are not affected. Creating apps, machines, or allocs from images in the Fly Registry (registry.fly.io/APP_NAME) are working.

Remote builders may fail to download the image from Docker Hub. We're working on a fix for that now.

Try `fly deploy --local-only` if you experience remote builder issues as result of this issue. That will use a local docker environment on your system to build and push the image. That might work and work around the remote builder issue for now, though we understand local docker environments can be unreliable in certain cases (for example, building linux/amd64 images on macOS M1/M2 chips can seqfault with a known wontfix issue in qemu).

Latest Fly.io outages

Metrics issue - 3 days ago
BOG - Network Issue - 4 days ago

Never miss when a third-party service is down

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 3170 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook

Setup in 5 minutes or less

How much time you'll save your team, by having the outages information close to them?

14-day free trial · No credit card required · Cancel anytime