Red Hat Outage History

Red Hat's Quay.io container registry experienced HTTP 502 errors that prevented users from pulling and pushing container images for 3.1 hours. The issue was caused by a recent deployment, which the team identified and reverted, gradually restoring service by rolling traffic back to the primary region. Pull operations were restored first, followed by push functionality once monitoring confirmed the fix was stable.

Red Hat's cert-api.access.redhat.com experienced a 3.1-hour outage caused by a gateway configuration mismatch, resulting in HTTP 500 errors for certificate-based requests. The incident impacted Red Hat Lightspeed registrations and data uploads for both direct and Satellite-connected hosts, affecting multiple Insights services including Inventory, Advisor, Compliance, Patch, Remediations, and Vulnerability components. The issue was resolved after implementing a configuration fix and monitoring the results.

Red Hat's Quay.io container registry experienced degraded performance affecting both push and pull API operations for 1.4 hours. The pull API was restored first, followed by the push API being brought back online. The service was fully resolved and returned to normal operation.

Red Hat's Subscription Watch service experienced a backend data processing error that caused incorrect subscription threshold exceeded notifications to be sent to customers using non-Pay-as-you-go products, displaying misleading usage values. The notification capability was temporarily disabled to prevent further erroneous alerts from being sent. The issue was resolved after 312.6 hours by fixing the backend service to process the correct data for non-PAYGO products.

Red Hat's quay.io container registry experienced 502 errors affecting both push and pull operations across multiple registry endpoints for 3.1 hours. The service was temporarily shifted to read-only mode to restore pull functionality while pushes remained unavailable. Full functionality was restored after implementing a fix that resolved both push and pull operations.

Red Hat's SSO Authentication system experienced timeouts that prevented users from accessing access.redhat.com. The issue affected user login capabilities for 23 minutes. The problem was identified and resolved with a implemented fix.

Red Hat experienced intermittent failures affecting RHEL 9 package downloads and Red Hat Update Infrastructure (RHUI), with users encountering duplicate key value errors during AppStream repository synchronization on Red Hat Satellite 6. The incident was a recurrence of a previously reported issue and lasted approximately 189.6 hours before being resolved. The RHEL team investigated and ultimately resolved the download failures that were impacting both direct package downloads and repository synchronization processes.

Red Hat experienced intermittent failures affecting RHEL 9 package downloads and Red Hat Update Infrastructure (RHUI) services, with users encountering duplicate key value errors during AppStream repository synchronization on Red Hat Satellite 6. The incident impacted users' ability to download specific packages and synchronize repositories in RHEL 9 environments. The issue was resolved after 67.9 hours of investigation and remediation efforts.