TrafficInsights processing delayed in US1, US3, US4

After further investigation, we have determined that a portion of the Traffic Insights (TI) data for the US4 cluster has become irretrievable. The TI data loss was limited to 6 PM EDT Thursday, April 20, and 6 AM EDT Friday, April 21. While impactful, this loss was minimized to a 12-hour time frame that took place outside of typical business hours. This issue only affected accounts that had TI turned on at the time of the outage and had no other operational impact on the core functions of Auvik deployments. Please contact your Customer Success Manager if your account was affected and you would like additional detail on the outage.

A full Root Cause Analysis (RCA) will be completed and made available in the coming weeks.

We have identified the underlying root cause of the instability that has caused the delay in the results of processing data in TrafficInsights (TI) in US4. We continue to work through the issue to bring all TI flow data to customers. Current TI flow data is available in the UI. We appreciate your patience as we continue to work through this issue. We will keep you posted on a resolution. The next update will be by April 25th, 18:00 UTC.

We have identified the underlying root cause of the instability that has caused the delay in the results of processing data in TrafficInsights (TI) in US4. We continue to work through the issue to bring all TI flow data to customers. Current TI flow data is available in the UI. We appreciate your patience as we continue to work through this issue. We will keep you posted on a resolution. The next update will be by 18:00 UTC.

We have identified the underlying root cause of the instability that has caused the delay in the results of processing data in TrafficInsights (TI) in US4. We continue to work through the issue to bring all TI flow data to customers. Current TI flow data is available in the UI. We appreciate your patience as we continue to work through this issue. We will keep you posted on a resolution. The next update is scheduled for Monday, 24, 2023, at 14:00 UTC.

We have identified the underlying root cause of the instability that has caused the delay in the results of processing data in TrafficInsights (TI) in US4. We continue to work through the issue to bring all TI flow data to customers. Current TI flow data is available in the UI at this time. We appreciate your patience as we continue to work through this issue. We will keep you posted on a resolution. The next update will be by 16:00 UTC.

We have identified the underlying root cause of the instability that has caused the delay in the results of processing data in TrafficInsights in US4. We continue to work through the issue to bring data current for all customers. Some customers may already see current data, but these results are still inconsistent across the cluster for all customers. We still anticipate having full functionality restored to the TI setup at some point tomorrow afternoon. We appreciate your patience as we continue to work through this large delay in full functionality. We will keep you posted on a resolution. The next update will be tomorrow morning.

We have identified the underlying, root cause of the instability that has caused the delay in the results of processing data in TrafficInsights in US4. We continue to work through the issue to bring data current for all customers. Some customers may already see current data, but these results are still inconsistent across the cluster for all customers. We anticipate having full functionality restored at some point tomorrow afternoon. We appreciate your patience as we continue to work through this delay. We will keep you posted on a resolution. Next update by 21:00 UTC.

We have identified the underlying root cause of the instability that has caused the delay in the results of processing data in TrafficInsights in US4. We are taking action to address the issue. Some customers may already see up to date data, but it will be inconsistent across the cluster. We continue to monitor and work toward a successful conclusion. Overnight progress was good and continues into this morning. We appreciate your patience as we continue to work through this delay. We will keep you posted on a resolution. Next update by 14:00 UTC

We have identified the underlying root cause of the instability that has caused the delay in the results of processing data in TrafficInsights in US4. We are taking action to address the issue. Some customers may already see up to date data, but it will be inconsistent across the cluster. We continue to monitor and work toward a successful conclusion overnight . We expect our next update tomorrow morning based on progress overnight. We appreciate your patience as we continue to work through this delay. We will keep you posted on a resolution.

We have identified the underlying root cause of the instability that has caused the delay in the results of processing data in TrafficInsights in US4. We are taking action to address the issue. Some customers may already see up to date data, but it will be inconsistent across the cluster. We continue to monitor the situation and work toward a successful conclusion. We appreciate your patience as we continue to work through this delay. We will keep you posted on a resolution.

We are continuing to monitor the situation as we work to resolve the delayed results in TrafficInsights in US4. Some customers may already see up to date data, but we continue to work to restore data across all tenants. We appreciate your patience as we continue to work through this delay. We will post another update by 21:00 UTC

We are continuing to monitor the situation as we work to resolve the delayed results in TrafficInsights in US4. Some customers may already see up to date data, but we continue to work to restore data across all tenants. We appreciate your patience as we continue to work through this delay. We will post another update by 17:00 UTC

We are continuing to monitor the situation as we work to resolve the delayed results in TrafficInsights in US4. Some customers may already see up to date data, but we continue to work to restore data across all tenants. We appreciate your patience as we continue to work through this delay. We will keep you posted on a resolution.

We are continuing to monitor the situation as we resolve the delayed results in TrafficInsights in US4. Some customers may already see up to date data but we continue to work to restore data across all tenants.

We are continuing to monitor the situation as we resolve the delayed results in TrafficInsights in US4. Some customers may already see up to date data but we continue to work to restore data across all tenants.

Issues with US1 and US3 are resolved. We are continuing to monitor US4 as it recovers.

Issues with TrafficInsights in US3 are now resolved. We expect that issues in US1 will be resolved within 1 hr. US4 is continuing to improve but we are still refining our estimate for when it will be fully functioning. We'll continue to keep you posted as this progresses.

We’ve identified the source of the service disruption with TrafficInsights and continue to monitor the situation. We are now starting to see data process in US4 and expect further improvements in US1 & US3 shortly. We expect to return to fully functioning across all clusters by end of day. We’ll keep you posted on a resolution.

Starting at 10:04am ET a delay in processing TrafficInsight data occurred in US1, US3, US4. No data has been lost, but results will be delayed by about 90 minutes. A fix has already been deployed, but it will take some time to process the backlog of messages. We are monitoring results as the system recovers.