Outage in Fly.io

Docker Hub Registry Timing Out: Fly Platform Impact

Resolved Minor
September 29, 2023 - Started over 1 year ago - Lasted about 3 hours
Official incident page

Need to monitor Fly.io outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including Fly.io, and never miss an outage again.
Start Free Trial

Outage Details

Docker Hub has an ongoing outage with their registry: https://www.dockerstatus.com/pages/incident/533c6539221ae15e3f000031/6515dbc89fabff05350ae18d Deployments that use an image from the Docker Hub Registry—any unqualified FROM in a Dockerfile or setting an unqualified image name with `fly deploy --image`—may fail. Existing machines and allocs are not affected. Creating apps, machines, or allocs from images in the Fly Registry (registry.fly.io/APP_NAME) are working. Remote builders may fail to download the image from Docker Hub. We're working on a fix for that now. Try `fly deploy --local-only` if you experience remote builder issues as result of this issue. That will use a local docker environment on your system to build and push the image. That might work and work around the remote builder issue for now, though we understand local docker environments can be unreliable in certain cases (for example, building linux/amd64 images on macOS M1/M2 chips can seqfault with a known wontfix issue in qemu).
Latest Updates ( sorted recent to last )
RESOLVED over 1 year ago - at 09/29/2023 05:30AM

The Docker Hub timing out issue has been marked resolved.

https://www.dockerstatus.com/pages/incident/533c6539221ae15e3f000031/6515dbc89fabff05350ae18d

MONITORING over 1 year ago - at 09/29/2023 03:33AM

We updated our remote builders to load their images from docker-hub-mirror.fly.io/flyio/rchab, which is a caching mirror of registry-1.docker.io. We're seeing remote builders more reliably work.

Deployments that use an image from the Docker Hub Registry—any unqualified FROM in a Dockerfile or setting an unqualified image name with `fly deploy --image`—may still fail.

You can try adding docker-hub-mirror.fly.io/IMAGE_NAME to your Dockerfile or the `--image` flag on `fly deploy`. That will work more reliably if we have a cached copy of the image from Docker Hub.

The `fly deploy --local-only` work around is still a good option, if it works for you.

Another work around is finding another registry, like ghcr.io or quay.io or something like that, which has a copy of the image you would normally get from the Docker Hub Registry.

Marking this as monitoring while we wait for further updates from Docker Hub via their status page.

INVESTIGATING over 1 year ago - at 09/29/2023 02:45AM

Docker Hub has an ongoing outage with their registry: https://www.dockerstatus.com/pages/incident/533c6539221ae15e3f000031/6515dbc89fabff05350ae18d

Deployments that use an image from the Docker Hub Registry—any unqualified FROM in a Dockerfile or setting an unqualified image name with `fly deploy --image`—may fail.

Existing machines and allocs are not affected. Creating apps, machines, or allocs from images in the Fly Registry (registry.fly.io/APP_NAME) are working.

Remote builders may fail to download the image from Docker Hub. We're working on a fix for that now.

Try `fly deploy --local-only` if you experience remote builder issues as result of this issue. That will use a local docker environment on your system to build and push the image. That might work and work around the remote builder issue for now, though we understand local docker environments can be unreliable in certain cases (for example, building linux/amd64 images on macOS M1/M2 chips can seqfault with a known wontfix issue in qemu).

Know the moment Fly.io and other vendors go down

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 4000 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook