Flexera One - IT Asset Management - NA - Batch Failures Resulting in Temporary Active Directory User Import Suspension

We have extended our monitoring throughout this week and have detected no anomalies or recurrence of the issue. Our temporary measures have ensured sustained stability, and our long-term action plans are progressing smoothly towards implementation. Consequently, we have marked this incident as resolved and transitioned it to a problem management investigation.

This investigation will oversee the completion of the long-term fix, expected by the end of this week. A post-mortem report will be published in the coming days, detailing the actions taken for long-term measures.

There have been no new reports of issues, and our temporary measures continue to maintain system stability. According to our technical team's latest analysis, our testing and validation processes are on track. We anticipate moving forward with a full deployment by the end of this week.

The recent updates have successfully passed initial functionality tests. We are currently performing performance tests to ensure these enhancements work reliably. Our goal is to maintain consistent stability and efficiency of our systems.

Further updates will be provided as progress is made.

The testing and validation phase is still ongoing, and our technical teams continue to make significant progress. While manual interventions and temporary measures have prevented any new issues, we have decided to prolong our monitoring process to ensure that the fixes remain effective and contribute to a stable, long-term solution. We appreciate your understanding as we work diligently to enhance our systems. We will continue to provide updates as we progress.

We have actively monitored the environment throughout the weekend and can confirm that there have been no batch failures or alerts related to connection issues. Our team continues to oversee batch processing tasks and Active Directory imports to ensure system stability.

Additionally, our technical team has completed the coding process and is now engaged in thorough testing and validation to ensure that all updates function as anticipated before their production deployment.

We are committed to maintaining a stable and reliable service and will provide updates as new information becomes available.

We have been actively monitoring the environment throughout the weekend. As of our latest evaluation, there have been no alerts or additional complications reported. Our team continues to closely observe the batch processing tasks and Active Directory imports. We are diligently working to identify the specific causes of the blockages to prevent future occurrences and ensure a stable service environment.

Thank you for your patience and understanding as we address this issue. We will provide further updates as they become available.

Our teams have been actively discussing strategies to mitigate the problem and expedite the recovery process. In line with the proposed approach to managing recurring AD import failures, our technical team has proactively lifted the previously enforced AD block. This action is intended to enhance our monitoring and management capabilities throughout the weekend. We are closely monitoring for any blocked session alerts and are prepared to immediately terminate any processes that pose a risk, to prevent further database and connection failures.

Our primary objective continues to be ensuring a seamless and successful import/reconcile process for our customers. We plan to reassess the situation after the weekend and will decide whether to reimpose the AD block based on the outcomes. This strategy is aimed at maintaining system stability while addressing the underlying issue.

We will continue to provide updates as more information becomes available and further actions are taken.

Our technical teams have made substantial progress on the fix. Once development is finalized, we will conduct comprehensive testing and validation before proceeding with implementation. Concurrently, we are monitoring the environment to ensure ongoing stability. We will keep you updated as we make progress.

Due to the AD inventory block, we expect no further batch processing failures to occur. However, the temporary AD disablement is affecting the normal flow of user data imports for all customers.

Incident Description: We are currently experiencing a recurring issue with connection errors during overnight batch processing tasks on the IT Asset Management platform in the NA region. This issue is impacting multiple organizations, resulting in batch task failures and delayed reconciliations.

Our assessment indicates that these connection errors are likely due to a potential bug in the Active Directory (AD) user import process, which has been causing a high number of blocked sessions and subsequent connection failures over the past few nights.

Priority: P2

Restoration Activity: To mitigate the connection errors affecting overnight batch processing, we have temporarily disabled the AD user import process for all customers and adjusted our monitoring thresholds for quicker issue detection. Our team is working on a permanent fix to address the bug and ensure long-term stability. Continuous monitoring is in place to prevent further disruptions.