Outage in Grafana

Issues Querying Metrics - US Central

Resolved Major
April 10, 2024 - Started 22 days ago - Lasted about 3 hours
Official incident page

Need to monitor Grafana outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including Grafana, and never miss an outage again.
Start Free Trial

Outage Details

As of 18:30 UTC, we were alerted to a high volume of errors querying metrics in both Graphite and Prometheus within the prod-us-central-0 cluster only. Users encountering this issue may experience an error stating "500 Internal Server Error: the bucket index is too old" when querying metrics. Engineering is actively engaged and assessing possible remediation paths. We will provide updates as more information is shared.
Components affected
Grafana GCP US Central - prod-us-central-0: Querying Grafana US-CENTRAL: Querying
Latest Updates ( sorted recent to last )
RESOLVED 22 days ago - at 04/10/2024 10:26PM

As of 22:20 UTC, querying has been restored for both Graphite and Prometheus, and users should no longer be experiencing the 500 internal Server Error.

MONITORING 22 days ago - at 04/10/2024 07:41PM

As of 19:17 UTC, we have observed marked recovery and users in the `prod-us-central-0` cluster should now be able to query Graphite and Prometheus metrics without errors.

We will monitor this issue and provide updates as more information is shared.

INVESTIGATING 22 days ago - at 04/10/2024 07:05PM

As of 18:30 UTC, we were alerted to a high volume of errors querying metrics in both Graphite and Prometheus within the prod-us-central-0 cluster only. Users encountering this issue may experience an error stating "500 Internal Server Error: the bucket index is too old" when querying metrics.

Engineering is actively engaged and assessing possible remediation paths. We will provide updates as more information is shared.

Latest Grafana outages

AWS Germany -- Metrics Generator Issues - 6 days ago
Service Interruption: Metric Queries Unavailable - 20 days ago
Grafana Managed Alerts - Pending periods and group intervals are incorrectly displayed in editing page - 22 days ago
Service Interruption: Metric Range Queries Unavailable - 22 days ago
Investigating Increasing Read Error Rate Detected in US Central Region - 27 days ago

The easiest way to monitor Grafana and all cloud vendors

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 3153 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook

Setup in 5 minutes or less

How much time you'll save your team, by having the outages information close to them?

14-day free trial · No credit card required · Cancel anytime

Start today for FREE