Kong Outage History

Kong Konnect Cloud experienced a major incident where analytics data from data plane nodes in the US region was being lost for 3.1 hours. The engineering team identified the root cause, deployed a fix, and monitored the system to ensure stability. The incident was fully resolved with analytics data collection restored to normal operation.

Kong's 3.14 release introduced "secure by default" changes that caused plugin configuration defaults for ssl_verify and hide_credentials to flip from false to true, breaking existing configurations for customers using deck sync without explicitly defining these values. This affected numerous plugins including authentication, AI, logging, and proxy plugins, causing unexpected behavior changes when configurations were updated. Kong resolved the incident by rolling back to the original default values after 5.3 hours.

Kong Konnect Cloud experienced degraded functionality for Personal Access Token (PAT) management operations, affecting the ability to create, read, and delete PATs through both the API and UI. Authentication using existing PATs remained unaffected during the incident. The issue was resolved after 1 hour of monitoring and remediation.

Kong Konnect Cloud experienced a major incident where Dev Portals became unreachable in the ME region for 2.5 hours. The service was restored and monitored for stability before the incident was fully resolved.

Kong experienced a major service degradation in the ME region that impacted the DevPortal, Debugger, and node listing functionality within Kong Konnect Cloud. The issue was identified as a problematic instance that required removal, and the service was restored after 23.4 hours of downtime.

Kong Konnect's Singapore region experienced partial degradation affecting portal emails and application registration, causing delays in message processing due to a backlog in the message handling infrastructure. The incident was complicated by concurrent cloud service provider maintenance that initially prevented the team from scaling up infrastructure to address the backlog. The issue was resolved after 3.3 hours once the cloud provider completed maintenance and the message handling infrastructure was scaled up to process the accumulated backlog.

Kong Identity experienced a complete failure to mint tokens across all regions, affecting Kong Konnect Cloud services. The team identified the root cause and applied fixes progressively, starting with US and EU auth servers before expanding to other regions. The incident was fully resolved after 53 minutes of downtime.

Kong Konnect Cloud experienced a partial outage in the US region where configuration changes failed to propagate from the control plane to data plane nodes, though some API requests continued processing successfully and existing data plane traffic remained unaffected. The issue was resolved after 22 minutes when a fix was implemented to restore proper configuration propagation.