Web Application Errors

This incident has been resolved.

The Engineering team continues to monitor system performance. There have been no performance degradations. Next update by 12 pm PT or sooner. If you are still seeing any impact please reach out to support@iterable.com

We are continuing to monitor for any further issues.

Engineering team has confirmed that we continue to process at normal rates. Our processing of messaging backlog has caught up. Engineering is continuing to monitor our systems, if you are still seeing any impact please reach out to support@iterable.com

The engineering team has completed the instances scale up of various services. Access to the web app, and message send rates are back to normal processing. We continue to see improvements across the app as we catch up on the messaging backlog. Most systems are caught up with processing. Next update by 3:15PM PT or sooner.

The issue is identified. The engineering team continues to scale up instances of various services. Customers should continue seeing improvements in access to the web app, and message send rates continue to recover. We continue to see improvements across the app as we catch up on the messaging backlog. Next update by 2:15PM PT or sooner.

The issue is identified. The engineering team continues to scale up instances of various services. Customers should begin seeing improvements in access to the web app, and message send rates are beginning to recover. We should continue to see improvements across the app as we catch up on the messaging backlog. Next update by 1:15 PM PT or sooner.

The issue is now identified. The engineering team is slowly bringing up the scaled down instances of various services. Customers should begin seeing improvements in access to the web app, and message send rates are beginning to recover. All services except webhooks are scaled up now. We should continue to see improvements across the app as we catch up on the messaging backlog. Next update by 12:45 PM PT or sooner.

The Engineering team is continuing to address DB issues. We are slowly bringing up the scaled down instances of various services. Web errors and journey latency are going down and send rate is going up. Customers may still experience issue with web app failing to fully load and an inability to switch projects. Next update by 12:15 PM PT or sooner.

The Engineering team is continuing to investigate and implement mitigations to address DB issues. Web errors and journey latency are going down and send rate is going up. Customers may still experience degradation and issues with pages of the web app failing to fully load and an inability to switch projects. Next update by 11:45 AM PT or sooner.

The Engineering team is continuing to investigate and implement mitigations to address DB issues causing web and API errors. Sends, journey processing, and webhooks will continue to see delays. Customers may also experience issues with pages of the web app failing to fully load and an inability to switch projects. Next update by 11:15 AM PT or sooner.

The Engineering team is continuing to investigate and implement mitigations to address DB issues causing web and API errors. Sends, journey processing, and webhooks will continue to see delays. Customers may also experience issues with pages of the web app failing to fully load and an inability to switch projects. Next update by 10:45 AM PT or sooner.

The Engineering team is implementing mitigations to address the intermittent web and API errors as well as send and webhook delays. Next update by 10:15 AM PT or sooner.

The Engineering team is investigating web and API errors. Next update by 9:45 AM PT or sooner.

We have put in further mitigations to restore send speeds and are working through the send backlog. We are continuing to work on recovering webhooks. Next update by 9:30 AM PT or sooner

We're seeing continued improvements in the web application and API performance. We are still working to resolve delays in sends and webhooks. Next update by 9:00 AM PT or sooner

Iterable engineering has put in mitigations and is seeing recovery in the web application and API endpoints. We are continuing to work on remediating delays to sends and journey processing. Next update by 8:30 am PT

Beginning at 5 AM PST we were alerted to a spike in API errors and Web App errors.

Our engineering team is investigating. Next update will be at 7:30 AM PST or sooner. If you have questions please reach out to support@iterable.com