LaunchDarkly Outage History

LaunchDarkly experienced a 54-minute outage where Observability alert notifications failed to be delivered to users. The engineering team identified the root cause, deployed a fix, and redelivered all delayed notifications. All other notification types remained operational throughout the incident.

LaunchDarkly experienced a metrics ingestion delay in the EU region that caused under-reporting of customer metrics for approximately 16 minutes (11:11-11:27am ET). The service team identified the issue, confirmed no data was lost, and spent 2 hours processing the backlogged metrics data. The incident was fully resolved once all delayed metrics were successfully ingested and caught up.

LaunchDarkly's documentation site experienced a major outage for 27 minutes due to an issue with their documentation provider. The incident was identified as being linked to an outage on the doc provider's side, and a fix was implemented to restore service.

LaunchDarkly experienced an issue where observability alerts failed to deliver Slack notifications to channels, though direct message alerts continued working normally. The problem was identified after approximately 1.5 hours of investigation and fully resolved after 2.3 hours total duration.

LaunchDarkly experienced delayed processing of Session Replay and Error event data for 1.1 hours due to elevated database utilization. Customers observed temporary delays in session and error data appearing in the LaunchDarkly UI, though all incoming data was queued rather than lost. The issue was resolved by addressing the database utilization problem, and all queued events were successfully processed.

LaunchDarkly experienced an issue where experiment reporting data was missing for some experiment iterations, causing affected users to see incomplete data in the UI for a subset of accounts. The team first restored data availability for current experiment iterations after about 2 hours, then worked on recovering historical data for previously affected iterations. After 46.4 hours, complete data restoration was finished and all experiment iterations showed full data in the UI.

LaunchDarkly experienced event processing delays that caused stale data to appear in autogenerated metric creation, Data Export, and Experimentation features for 5.2 hours. Flag Delivery remained operational throughout the incident. The issue was resolved with a fix implementation, and no data was lost during the outage.

LaunchDarkly experienced inaccurate usage reporting for their Observability "Errors" product, affecting some customers' February and March usage data. The engineering team identified the root cause and corrected the February reporting issues while finalizing corrections for March. The incident was resolved after 4.6 hours with usage reporting restored to accuracy.

LaunchDarkly experienced a major incident where customers could not access the web application and encountered authentication failures, with some users also seeing errors during flag evaluation. The Flag Delivery Network remained generally functional throughout the 1.4-hour outage. The issue was identified, fixed, and resolved with stable performance restored.