PostHog US Outage History

PostHog US experienced an issue where LLM analytics traces older than approximately 17 hours failed to load in the dashboard, displaying "trace not found" errors instead. The problem affected the main application and prevented users from accessing historical LLM trace data. The issue was resolved after 33 minutes with a fix implemented and monitoring in place.

PostHog US experienced processing delays in their logs ingestion pipeline for 16 minutes, causing logs to take longer than usual to appear in the product. While no data was lost, logs were temporarily unavailable in PostHog apps and queries until the processing delays were resolved. The incident was classified as minor and affected the Logs Ingestion component.

An attacker exploited PostHog's invite system to send phishing emails containing malicious links between April 22-24, affecting users who received unexpected invite emails. PostHog deployed a fix to prevent further abuse, blocked the attacker, and confirmed no compromise of their systems or data occurred. The incident was resolved after 12.9 hours with continued monitoring for additional malicious activity.

PostHog experienced issues with their overflow topic that caused deprioritized workloads to be processed with additional delays beyond normal expectations. Normal realtime destination delivery continued to function without delays and no data was lost. The issue was resolved after 27 minutes of investigation and remediation.

PostHog experienced analytics query timeouts and failures that affected dashboards, insights, and other query-driven views, making them slow or unavailable. The incident lasted 12 minutes and was classified as minor. The service was investigating the issue but no resolution details were provided in the available updates.

PostHog experienced an issue where users were unable to add new users to static cohorts. The problem was identified after investigation and a fix was implemented. The incident lasted 1.9 hours with minor impact to the cohort management functionality.

PostHog experienced analytics query timeouts and failures for 23 minutes, causing dashboards, insights, and other query-driven views to be slow or unavailable. The issue was caused by a failed node in their ClickHouse infrastructure, which was identified and mitigated to restore service.

PostHog experienced issues with realtime destinations causing increased errors and delivery delays for events over 1.3 hours. The underlying issue was fixed and the system caught up on delayed events. No data was lost during the incident.

PostHog experienced a partial outage where session replays could not be played, though no data was lost during the incident. The issue was resolved after 27 minutes of investigation.

PostHog experienced analytics query timeouts and failures for 9.2 hours, causing dashboards, insights, and other query-driven views to be slow or unavailable. The issue was identified and queries returned to normal operation with continued monitoring.