Alerts API & FlightStats.com Flight Alerts degraded performance

This incident has been resolved.

Alerts performance has been stable over the past 24 hours, with no issues reported for the Alerts API. The root cause was identified as increased database data from sweepers running, which led to SQL query delays. We addressed this by vertically scaling the database and are currently working to clear stale data.

All systems are stable with no further performance issues detected. Root cause investigation is underway. A further update will be provided no later than 2025-11-11 06:30 UTC.

System performance remains stable, with alerts being delivered promptly and no further issues detected. We are continuing to investigate the temporary spike, however have yet discovered the cause. The database clean up is ongoing, and we are monitoring the system closely. A further update will be provided no later than 2025-11-10 14:30 UTC.

The performance seems stable now after a temporary spike from 5.30 UTC to 6.20 UTC.
The alerts are getting delivered without delays.

Next Actions:
1) We are checking why there is a temporary spike during this timeframe
2) Team is working on the Database cleanup to optimize the storage space

Next update will be 2025-11-10 10:30 UTC.

The performance of the Flight Alerts API looks good and the requests are getting processed.
We will continue monitoring the system and the cleanup the backlog and stale data. Next update - 2025-11-10 6:30 UTC

The Flight Alerts API service have been restored and all the operations have been working. Alerts are getting delivered. We are continuously monitoring the system for performance. There could be some latency which is expected due to the heavy backlog. Next update, 2025-11-09 6:30 UTC.

The database is performing better, but the main delay is due to the large volume of stale data that must be manually cleaned up before normal operations can resume. They are actively working on the cleanup now and anticipate that services will resume in the next update 9th November 1:00 UTC.

We are continuing to work on a fix for this issue.

We are continuing the recovery process here, still no major breakthrough. Database is bloated with huge volume and less memory available. Database Backup recovery is running for a while and we are waiting for the process to complete. We will provide a further update no later than 2025-11-08 5:00 UTC. Thank you for your continued patience and understanding.

Our engineering team has identified the cause of the latency and are working to clear the alerts queue. We will provide a further update on or before 2025-11-08 01:17 UTC.

We are continuing to experience latency issues during the processing of flight events. Our engineers are investigating. We will provide a further update no later than 2025-11-07 20:37 UTC. Thank you for your continued patience and understanding.

The queue size has returned to normal, and alert delivery to customers is operating as usual. The connection timeout issue is stable, and the team is monitoring it. The original issue of missing alerts for several Rule IDs affecting some customers has been resolved, alerts will now be delivered as expected. We will continue to monitor the system and provide updates.

We’d like to provide you with an update on this case. As per our engineering team, the system has shown stable connectivity over the past six hours, and we continue to monitor its performance closely. The next update will be provided by 13:00 UTC on 2025-11-07, or sooner if there are any further developments.

We are continuing to investigate this issue.

The issue was initially limited to some Alerts API and FlightStats.com customers but is now affecting our alerting service for other airlines. Previous mitigation efforts have not resolved the problem. Our development team will continue working on a fix to restore normal service and we will provide the next update at 2025-11-07 06:45 UTC.

We have identified an event processing timeout issue that is preventing one airline's flights from triggering Alerts API callbacks. Our engineers are continuing to investigate the cause of these timeouts. A further update will be provided no later than 2025-11-06 23:54 UTC.

Our Engineering Team has identified that the Alert APIs are affecting one specific airline. They are actively investigating the root cause of the issue. Thank you for your patience and understanding.

A subset of Alerts API customers are currently experiencing degraded performance with our Alerting service. Our engineering team is actively investigating the issue and will provide updates as soon as more information becomes available. We will provide the next update on 2025-11-06 at 07:54 UTC. Thank you for your patience and understanding.