IT Visibility - NA - Data Processing is currently delayed

We have deployed additional optimizations to production. The Dashboard updates are currently in progress, and we are continuing to monitor the environment.

The investigation is ongoing. We are currently testing a new set of optimizations to be deployed in production later today.

We are still observing a slow response in the application. Our staff deployed a new set of optimizations in production, but that did not yield the desired results. Discussions are ongoing, and we continue to mitigate problems while working on a long-term solution.

We have deployed additional optimizations to production, following which new queries have been running normally, and no new anomalies have been observed so far. The investigation is still ongoing. We are continuing to isolate problems and work on a long-term solution.

Performance has significantly improved following some of the remediation actions. Technical teams are continuing to make optimizations in the environment and investigate a long-term fix.

During the investigation, technical teams discovered that some of the queries to export data from IT Visibility to ServiceNow are in a hung state and as a result some of the US customers with IT Visibility ServiceNow integration may experience issues while completing their export activities.

We have deployed additional changes to provide relief in the environment. Technical teams are continuing to work on a fix and long-term solution for this issue.

The investigations are still ongoing. Technical teams are working on isolating any performance issues in the environment and making enhancements.

Technical teams continue to eliminate any contributing factors to mitigate the problem and simultaneously investigate a long-term solution.

Performance has improved since the changes were deployed on Friday; however, we are still observing delays in processing. Technical teams have identified some of the contributing factors and continue to work towards a resolution.

The backlog processing has been ongoing and we are continuing to make progress. Technical staff will be monitoring the environment overnight, and the next update will be provided tomorrow morning.

Production deployment is still ongoing, and we are continuing to make progress.

The configuration change in Production is progressing as planned. We have processed about 1/3 of the backlog and validated that IT Visibility Dashboards are being updated.

We have successfully tested the configuration change and are on track to deploy it in production.

We are continuing to test a fix to re-organize the configuration to enable faster data processing, following which it will be deployed in the production environment.

We are currently testing a fix to re-organize the configuration to enable faster data processing, following which it will be deployed in the production environment.

Incident Description:
IT Visibility Data Processing in North America is currently delayed by up to 20 hours, resulting in some data being out of date.

As a result of this incident, some of the US customers with IT Visibility ServiceNow integration may also experience issues while completing their exports.

Priority: 2

Restoration activity:
Technical teams have been engaged and are currently investigating.