Outage in The Graph

[Grafana]: Base: Block ingestor lagging behind

Resolved Minor

April 23, 2025 - Started 12 months ago - Lasted about 2 hours
Official incident page

Incident Report

[FIRING:1] Base: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure) https://thegraph.grafana.net/alerting/list **Firing** Value: B0=0 Labels: - alertname = Base: Block ingestor lagging behind - cmp = partial_outage - component = indexers - grafana_folder = Production - priority = P2-High - statuspage = true - team = Infrastructure Annotations: - description = Triggered when the block ingestor is lagging - statuspage = Base: Block ingestor lagging behind Source: https://thegraph.grafana.net/alerting/grafana/ddjeivil1kc8we/view?orgId=1 Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=__alert_rule_uid__%3Dddjeivil1kc8we&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1 Dashboard: https://thegraph.grafana.net/d/7rcuDImZk?from=1745434600000&orgId=1&to=1745438235315 Panel: https://thegraph.grafana.net/d/7rcuDImZk?from=1745434600000&orgId=1&to=1745438235315&viewPanel=118

Latest Updates ( sorted recent to last )

RESOLVED 12 months ago - at 04/23/2025 10:02PM

This incident has been resolved.

INVESTIGATING 12 months ago - at 04/23/2025 08:10PM

[Comment from Opsgenie]Vlad Sirenko noted: "https://graphprotocol.slack.com/archives/CL43LRZC6/p1745438908907459?thread_ts=1745433420.253159&cid=CL43LRZC6

syncing, should be self resolved soon " on alert: "[Grafana]: Base: Block ingestor lagging behind"

INVESTIGATING 12 months ago - at 04/23/2025 08:05PM

[Comment from Opsgenie]Vlad Sirenko noted: "The problem is being resolved on the Pinax end. they have just rolled out the fix. " on alert: "[Grafana]: Base: Block ingestor lagging behind"

INVESTIGATING 12 months ago - at 04/23/2025 08:04PM

[Comment from Opsgenie]Vlad Sirenko acknowledged alert: "[Grafana]: Base: Block ingestor lagging behind"

INVESTIGATING 12 months ago - at 04/23/2025 07:57PM

[FIRING:1] Base: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Base: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Base: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivil1kc8we/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=__alert_rule_uid__%3Dddjeivil1kc8we&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1
Dashboard: https://thegraph.grafana.net/d/7rcuDImZk?from=1745434600000&orgId=1&to=1745438235315
Panel: https://thegraph.grafana.net/d/7rcuDImZk?from=1745434600000&orgId=1&to=1745438235315&viewPanel=118

Latest The Graph outages

Degraded Indexing Performance (Gnosis) - 3 months ago

Subgraph Indexing Issue - 3 months ago

Upgrade Indexer - DB Unavailable and Subgraph not allocated - 3 months ago

DatasourceError - 3 months ago

Polygon (Matic): Block ingestor lagging behind - 3 months ago

The Status Page Aggregator with Early Outage Detection

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 6320 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook