Harness Outage History

Harness experienced a major outage affecting Infrastructure as Code Management (IaCM) pipelines using Terraform for 1.3 hours. The issue was identified and resolved through an update to the harness_terraform plugin version 0.214.0. Customers using older pinned versions need to update manually, while those not pinning specific versions automatically received the fix.

Harness experienced CI hosted build failures affecting Linux Cloud Builds in their Continuous Integration Enterprise service for 4.8 hours. The issue was identified and a fix was implemented, with the service returning to normal operation after a monitoring period.

Harness experienced an 18.2-hour incident where Feature Flags were unable to update, preventing users from modifying their feature flag configurations. The engineering team implemented a fix and monitored the system before confirming full resolution. The incident was classified as minor severity and has been completely resolved.

Harness experienced an issue where feature flag metrics impact calculations stopped updating, though experiment calculations remained unaffected. The service team deployed a fix and monitored the system for 45 minutes until all metrics impact calculations resumed normal processing.

The Legacy Run Test step in Harness experienced intermittent connectivity failures to the test intel service, affecting all customers in the Prod2 environment. This impacted Continuous Integration Enterprise components including Mac, Windows, and Linux cloud builds as well as self-hosted runners. The issue was identified as a connectivity problem, fixed, and resolved after 5.3 hours of monitoring.

Harness experienced login issues affecting multiple services including Continuous Integration Enterprise (Mac, Windows, and Linux Cloud Builds), Continuous Delivery Next Generation, and Feature Flags. The incident lasted 59 minutes, during which users were unable to access the platform. The issue was identified and resolved with a fix that restored normal login functionality.

Harness experienced issues that impacted pipeline executions in their Continuous Delivery - Next Generation (CDNG) service for 6.6 hours. The incident affected the ability to run pipelines properly. A fix was implemented and monitored before the issue was fully resolved.

The Harness Internal Developer Portal (IDP) experienced accessibility issues that prevented users from accessing the service. A fix was implemented and deployed to restore functionality. The incident was resolved after 17 minutes of monitoring confirmed the service was operating normally.

Harness experienced degraded performance in their Pipeline Insights Dashboards, specifically affecting Custom Dashboards functionality. The issue persisted for 21.1 hours before being identified and resolved with an implemented fix. The incident was fully resolved after a monitoring period to ensure stability.

Harness experienced a CI degradation affecting steps that used AWS connectors with inherited authentication, impacting the Continuous Delivery - Next Generation (CDNG) service. The incident lasted 2.7 hours before being resolved. The degradation specifically affected CI pipeline steps configured with AWS connectors using inherited authentication methods.