EU - elevated error rate for feature flags, increased latency

This incident has been resolved.

we're monitoring the systems after fixes were applied and they are stable.

We're processing the delayed data which should be completed soon.

Web app and feature flags are fully operational again for a while now.

We update this once the delayed data is fully processed.

We applied some more fixes and seeing signs of recovery. We're monitoring the situation.

We still need some time to crunch through the data. We will send an update.

No data has been lost in all of this.

We identified the cause for the increased latency. The error rates for feature flags have decreased (from 1.5%) and the ingestion lag is coming down.

We're investigating increased error rates in feature flags and an increase in latency in the ingestion pipeline.

There is no data loss, but we're currently processing events with a delay.

We're still investigating but also discovered that the web app is showing increased latency.

We've spotted elevated error rates in feature flags. We're currently investigating the issue, and will provide an update soon.