CDP - Delayed data processing

This incident has been resolved.

Based on our efforts from yesterday, the system has stabilized overnight and the system is now operating as expected.
Our team is actively monitoring the system to ensure continued stability and is focused on retrying any failed event data to keep all processes up-to-date and accurate.

A fix has been implemented and we are monitoring the results.

Following the latest update, we have decided to proceed with another system maintenance window. Although errors are decreasing, we aim to fine-tune and expedite this improvement further to enhance performance.
Please note that during this maintenance window, no events will be processed, and workflows will not be triggered. We will perform this maintenance between 19:00 and 20:00 CET time.

We observe a positive decrease in errors and queues following our recent maintenance window. Our team will closely monitor these encouraging trends over the next hour to assess whether further emergency maintenance would be beneficial. We will provide our next update in 60 minutes or sooner if significant developments occur.

Our ongoing database maintenance is progressing successfully. However, to ensure we achieve the best possible outcome, we require an additional hour beyond our original schedule. We now anticipate concluding the maintenance around 16:00.

As mentioned in the previous update, we will perform maintenance between 14:00 and 14:30 CET time. During this timeframe no events and workflows will be processed, after the maintenance window this will be picked up again and continue.

We are going to perform maintenance on our systems to help mitigate the ongoing issues. During this timeframe no events and workflows will be processed, after the maintenance window this will be picked up again and continue.

Please understand we are doing everything we can to mitigate the ongoing issues with the help of several partners. As soon as we know the timeframe we will post another update, but please be aware this is upcoming.

Update: We are actively working together with our third-party providers to resolve the ongoing issues. Our team is fully committed to restoring normal operations as quickly as possible, we are sorry for the inconvenience.

We are working on implementing the solution to resolve these issues.

We have identified the issue causing delays, as well as failures in processing events and syncing segments to the address book. We are working on implementing a solution to resolve these issues.
No events have been lost during this issue, and failed events will be reprocessed once the fix is implemented.

We are continuing our investigation into this issue.

We are experiencing delays in data processing and loading times. Our team is actively investigating the issue.