CEP & MEP Delivery Issues

The system has been stable for the last 12 hours.

As of August 14, 18:04 CEST:
- Late MEP & CEP Trigger Automation messages have been sent
- Some MEP Push Campaigns & Recurring Automations have been partially send and will not be retried
- API error rates are back to normal
- Webhooks are back to normal


This incident is now considered as resolved.

The system has now fully processed the backlog of CEP and MEP automation messages, including Email, SMS, and Push v1 and v2.
All previously delayed messages have been sent.

We are now moving into a monitoring phase, during which the on-call team will closely watch system performance.

The system is still catching up with the backlogged work.
All late CEP & MEP Automation messages (Email, SMS, Push v1 & v2) will be sent.

We will provide further updates as soon as we have more information or in 1 hour, whichever comes first.

The database issue has been resolved.
All components are gradually returning to normal.
Processing speed is being increased progressively, with priority given to maintaining system stability.

We will provide further updates as soon as we have more information or in 1 hour, whichever comes first.

We are still working on fixing the database problem.

The affected components are still the same.

We have been experiencing a database issue since August 14th, 12:10 CEST.
We are working on a complete report of what is affected.

In the meantime, here is a brief overview of the platform’s status

* CEP Automations (Email, SMS, Push v2) are not being sent
* MEP Automations (Push v1) are not being sent
* MEP Campaigns are partially being sent
* CEP Campaigns are partially being sent
* The transactional API (MEP) is working
* Data sent on APIs for which we sent back a 200 HTTP status code may not be processed live but is enqueued and will eventually be processed
* Some APIs might experience elevated error rate over the afternoon
* Webhooks are partially working


We will provide further updates as soon as we have more information or in 1 hour, whichever comes first.