Multiple Segment features are experiencing intermittent issues

This incident has been resolved.

AWS has resolved the issues with the usw2-az1 and usw2-az2 Availability Zones in the US-WEST-2 Region.

As the AWS issues is resolved, Tracking API, UI and other APIs are recovered too. We will continue to monitor our sub-systems as It could take some time to work through the lag that has built up in the system.

AWS continues to work towards resolving the increased networking latencies and errors affecting Availability Zones (usw2-az1 and usw2-az2) in the US-WEST-2 Region. They have seen network mapping propagation latencies return to normal levels in both usw2-az1 and usw2-az2 Availability Zones. At this stage, all EC2 instance launches and network configuration changes are operating normally. They continue to monitor network mapping latencies and will provide a final update once all has been confirmed to be operating normally.

As AWS issues resolve, Tracking API, UI and other APIs should recover too. It could take some time to work through the lag that has built up in the system. We will continue to provide updates as we get them

AWS continues to work towards resolving the increased networking latencies and errors affecting Availability Zones (usw2-az1 and usw2-az2) in the US-WEST-2 Region. Over the last 30 minutes, they have seen a strong improvement in network mapping latencies in both Availability Zones. They have also seen a reduction in latencies and error rates for other AWS services affected by this event. At this stage, they would expect that the vast majority of EC2 instance launches and network configuration changes would complete within the normal latencies. They continue to monitor until all network propagation latencies have returned to normal levels and will continue to provide updates.

As AWS issues resolve, Tracking API, UI and other APIs should recover too. It could take some time to work through the lag that has built up in the system. We will continue to provide updates as we get them

AWS continues to work toward resolving the increased networking latencies and errors affecting Availability Zones (usw2-az1 and usw2-az2) in the US-WEST-2 Region. They have now completed all updates to mitigate the resource contention being experienced by the subsystem responsible for network mapping propagation. They're seeing early signs of improvements as the system works through the backlog of network mapping propagations. They expect to see network mapping propagation latencies continue to recover over the next 2 hours.

Soon after Tracking API, UI and any other Segment APIs should recover from the degraded performance

AWS has identified the issues as errors affecting Availability Zones. They expect to fully recover in the next 90 to 120 mins. We expect to the 500 errors on Tracking API and any errors on UI & API to resolve at that time. The pipeline latency has improved but would take some time to work through the lag.

The segment ecosystem is impacted by the AWS US West 2 incident. There will be intermittent UI and API errors until AWS becomes stable again.

We are observing a small number of 500 errors on Tracking API due to the AWS Incident. We observed a lag in our end-to-end processing, which is recovering.

We are currently investigating an issue with multiple features within the Segment ecosystem. This includes Public API, the segment web app, monitoring features, and other interface features. At the time we do not believe this is impacting event delivery or data processing.