Outage in Datadog US1

Delayed Processing for a Subset of Metrics

Resolved Minor
August 01, 2023 - Started 9 months ago - Lasted about 9 hours
Official incident page

Need to monitor Datadog US1 outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including Datadog US1, and never miss an outage again.
Start Free Trial

Outage Details

We are investigating increased latency processing Processing for Distribution Metrics coming from the agent. As a result of this issue, some users may see delays or gaps for a subset of their metrics on graphs. RUM/Synthetics/Logs/Processes measures that depend on distribution metrics will also see delays. To prevent spurious alerts, we have temporarily disabled monitors based on this data.
Components affected
Datadog US1 Metrics Pipeline Datadog US1 Synthetics Datadog US1 Logs Datadog US1 Alerting Engine Datadog US1 Processes
Latest Updates ( sorted recent to last )
RESOLVED 9 months ago - at 08/02/2023 05:45AM

This incident has been resolved.

MONITORING 9 months ago - at 08/02/2023 03:08AM

Distribution metrics lag has been resolved. As a result of the issue, some users may see gaps for distribution metrics on graphs. Monitors evaluation is now re-enabled for monitors based on this data.

MONITORING 9 months ago - at 08/02/2023 12:43AM

We continue to make progress on processing the lag on distribution metrics and are monitoring any further issues.

MONITORING 9 months ago - at 08/01/2023 10:57PM

We continue to process the lag on distribution metrics

MONITORING 9 months ago - at 08/01/2023 09:42PM

A fix has been implemented and we are monitoring the results.

IDENTIFIED 9 months ago - at 08/01/2023 09:37PM

The issue has been identified and a fix is being implemented.

INVESTIGATING 9 months ago - at 08/01/2023 09:37PM

We are continuing to investigate this issue.

INVESTIGATING 9 months ago - at 08/01/2023 09:07PM

We are continuing to investigate this issue.

INVESTIGATING 9 months ago - at 08/01/2023 09:05PM

We are investigating increased latency processing Processing for Distribution Metrics coming from the agent.
As a result of this issue, some users may see delays or gaps for a subset of their metrics on graphs.
RUM/Synthetics/Logs/Processes measures that depend on distribution metrics will also see delays.
To prevent spurious alerts, we have temporarily disabled monitors based on this data.

Latest Datadog US1 outages

Partial outage of metrics query - 4 days ago
Elevated Error Rates for Metrics Submission - 25 days ago
Google SSO login issues for web application - 2 months ago
Elevated Error Rates for Metrics Submission - 3 months ago
Logs Status Elevated Error Rates - 4 months ago

Monitor vendor outages in real-time. Stay one step ahead

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 3155 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook

Setup in 5 minutes or less

How much time you'll save your team, by having the outages information close to them?

14-day free trial · No credit card required · Cancel anytime

Start today for FREE