SMS processing and delivery delay

This incident has been resolved.

Update - Our partner team investigation is ongoing, and we have not yet been able to definitively identify the root cause of the issue affecting outbound message delivery within the platform.
To prevent further message backlog growth and reduce the risk of additional duplicate deliveries, we have implemented the following mitigation measures:

Message submissions are being temporarily blocked and will return Failed responses.
Message workers have been stopped to prevent further processing of impacted traffic.
Queued messages are being redirected to a new queue for post-incident analysis and recovery efforts.

Customers using SMPP may still be able to submit messages during this period. As a result, some message duplication may continue to occur until the incident is fully resolved.

As part of our recovery strategy, messages queued from the start of the incident will not be automatically reprocessed. While some of these messages may already have been successfully delivered after experiencing delays, others may not have been sent. Customers should be aware that message reconciliation and potential resubmission may be required once service has been restored.

Our engineering teams remain actively engaged in identifying the root cause and restoring normal service. We will continue to provide updates as new information becomes available.

Update - Our SMS service partner identified an issue within the internal systems responsible for routing messages to downstream mobile network operators.

Since approximately 14:30 AEST, messages within our platform have been intermittently becoming stuck in a router worker process, resulting in delays to outbound message delivery.

While the issue is ongoing, small batches of queued messages are continuing to be processed and delivered approximately every few minutes. Customers may also observe duplicate message deliveries in some cases as a result of this issue.

Our engineering teams are actively investigating the root cause and working to restore normal service as quickly as possible. Once the issue has been resolved, we expect queued messages to resume processing and be delivered automatically.

We will provide further updates as more information becomes available.

An update has been posted

We are still investigating an issue impacting outbound message delivery and engaging additional resources to help restore normal service as quickly as possible.

Further updates will be provided as more information becomes available.

Update - We are investigating an issue impacting outbound message delivery across all regions.

Customers will experience delays in message processing and delivery, with message statuses remaining in "Enroute" for longer than expected. This issue is affecting outbound message sending across our platform.

Our engineering teams are actively investigating the root cause and working to restore normal service as quickly as possible.

We will provide further updates as more information becomes available.