Pantheon Outage History

Pantheon experienced a dashboard display issue where Git commit logs and deployment histories failed to load on Dev environments for 4 hours. This was purely a visual problem in the dashboard interface - all hosted sites remained fully operational and Git pushes/deployments continued to work normally via CLI and Terminus. The engineering team identified the root cause, implemented a fix, and restored normal Git commit log visibility in the dashboard.

Pantheon experienced reduced performance affecting Dashboard loading for 7.4 hours, while live customer sites remained unaffected throughout the incident. The performance issues impacted user workflows by causing slow Dashboard responsiveness and loading delays. The engineering team identified and implemented a fix, restoring normal Dashboard functionality.

Pantheon experienced a dashboard reporting issue where site traffic metrics incorrectly displayed as zero for 3.4 hours, though all hosted sites remained fully operational. The engineering team implemented a fix to restore the data pipeline, but warned that cached zero-value metrics might persist in browsers for up to six hours. Users were advised to clear their browser cache to immediately view updated traffic data.

Pantheon experienced a major incident where users were unable to create new workspaces due to failures in the workspace creation system. The issue affected the Dashboard component and prevented users from setting up new workspaces for 8.4 hours. Engineers identified and implemented a fix, fully resolving the workspace creation functionality.

Pantheon experienced a major incident where a high-risk configuration in the deployment workflow from Test to Live environments could have caused unintended data and file loss. The issue affected Workflow Operations and required an immediate freeze on all Live deployments to prevent potential data corruption. Engineers identified and implemented a fix within approximately 2 hours, restoring safe deployment capabilities after monitoring confirmed the resolution.

A small number of Pantheon customer accounts were compromised through credential stuffing attacks using passwords stolen from external data breaches unrelated to Pantheon. The unauthorized access potentially allowed attackers to modify customer site code and files. Pantheon worked directly with affected customers to secure their accounts and remediate unauthorized changes, with no further suspicious activity detected after the 30-hour incident.

Pantheon experienced a major Redis endpoint issue that disrupted connections for customer sites, primarily affecting the Europe region. The incident caused intermittent downtime for affected sites over a 28.2-hour period, with manual remediation available through support during the outage. A fix was implemented and monitored before the incident was fully resolved.

Pantheon experienced intermittent access issues affecting the Dashboard and Workflow Operations for approximately 107.5 hours. Users encountered performance problems and difficulty accessing the dashboard, with the engineering team recommending postponing non-critical dashboard and CLI tasks during the outage. The issue was resolved through multiple fix attempts and monitoring phases, with the platform returning to normal operations and confirmed stability.

Pantheon experienced intermittent access issues affecting the Dashboard, Workflow Operations, and Terminus Operations for 1.8 hours. Users encountered 502 responses, degraded performance, and failed commands when trying to access these services. The team implemented a fix and monitored the results before confirming full resolution of the incident.

Pantheon experienced degraded performance affecting Dashboard loading and Terminus-related workflows for 35.4 hours. The engineering team identified the root cause as issues with core data storage performance and asynchronous job runner behavior, requiring a deep investigation and fix implementation. The incident was resolved with all Dashboard and Terminus operations returning to normal functionality.