Degraded logs querying performance

This incident has been resolved.

Things have been stable for a while now. There is still a backlog of logs, so some logs may continue to be delayed until things catch up, but all processes looking nominal now.

We are seeing stable log queries from the dashboard now. There may be delays in logs appearing or possibly missing logs from the duration of the incident. We are currently assessing that now that things are looking more stable.

We have identified a few issues. Some fixes have gone out and the team is working on others. Things are stabilizing, and error rates are reducing; however, they are not entirely solved. But folks should be seeing improvements in logging in the dashboard now. We'll continue to update as the situation continues to improve.

We are still working on resolving the issue. We are deploying a new version of Logflare that can solve one of the underlying causes.

Please note that Edge Functions (and all other services) are fully operational, but as logs are degraded, you will not be able to see their logs in the dashboard.

We have identified possible underlying issues, and our engineers work hard on mitigation measures. We have balanced some queries to an additional cluster and limited queries with high resources consumption to be available only for teams and enterprise tier customers. We are still working on immediate steps solve the problem and will post updates as soon as we are able to mitigate the issue.

We are experiencing degraded logs querying performance. Our team is aware of the issue and is already implementing mitigation issues. Logs ingestion is unaffected.