Increased web console latencies and Live Tail issues

This incident has been resolved.

All issues affecting the web application, public API, and streaming authentication service have been fully mitigated, and these systems are now operating normally.

All issues affecting the web application, public API and streaming authentication service have been mitigated and these systems are now fully operational.

Data pipelines involved in our outbound S3 integration are showing signs of recovery as well.

Our team will continue to monitor and provide updates.

Our team has mitigated the issues affecting our web application and public API, with both being operational at this time. We continue to monitor in case the underlying AWS outage may impact further.

Customer client event consupmtion remains healthy, and impressions are being processed normally. However, web application is impacted with non-migrated (only login through app.split.io) customers experiencing intermittent login failures and configuration operation of feature flags and experiments.

SDKs continue to receive flag updates, and customer applications won't see any disruptions. Customers may see timeouts for auth.split.io, with the SDK falling back to polling for updates resulting in slightly higher propagation times.

Customer may see errors and intermittent timeouts using the public API, which may prevent them from updating flags.

We have confirmed the first disruption occured from 12:44 AM to 1:31 AM PST, with the second being at 7:32 AM PST and is ongoing.

Our teams are actively monitoring the situation and working with AWS to restores the functionality.

The ongoing AWS incident is causing timeouts for some of our internal services and providers, which can result in intermittent login issues.

Our SDKs and data processing are unaffected and continue to operate without issues.

An issue affecting AWS is resulting in increased latencies for our web console and preventing the Live Tail functionality from working properly. We will continue to monitor the situation and provide updates here.

More details can be found on AWS status page: https://health.aws.amazon.com/health/status