Outage in Netdata

Slow and failing Agent chart data responses

Resolved Minor
June 29, 2022 - Started over 3 years ago - Lasted about 23 hours
Official incident page

Need to monitor Netdata outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including Netdata, and never miss an outage again.
Start Free Trial

Outage Details

Users with nightly versions of the Netdata Agent are experiencing slow responses between Cloud and Agent, resulting in failing or slow charts in their Cloud dashboards. We are investigating the issue.
Components affected
Netdata Agent-Cloud Link (ACLK)
Latest Updates ( sorted recent to last )
MONITORING over 3 years ago - at 06/30/2022 08:03AM

For completeness, the affected versions are v1.35.0-84-nightly and v1.35.0-96-nightly. Latest, corrected version is v1.35.0-104-nightly.

MONITORING over 3 years ago - at 06/30/2022 07:20AM

The new nightly version of the Netdata Agent has been published and installed by a large portion of the agents that auto-update. We are monitoring the results.

IDENTIFIED over 3 years ago - at 06/29/2022 02:11PM

We have identified part of the cause of failing responses for alarm values. In yesterday's nightly build of the Agent, we enabled the use of the newer MQTT5 library by default. We will create another build to revert that. In the meanwhile, you can explicitly disable this library using the mqtt5 setting in your configuration as described here: https://github.com/netdata/cloud-backend/issues/178.

Additionally the other latencies appear to be another instance of a known issue that causes responses with a small payload to be delayed. We are working on resolving this issue.

INVESTIGATING over 3 years ago - at 06/29/2022 08:54AM

Users with nightly versions of the Netdata Agent are experiencing slow responses between Cloud and Agent, resulting in failing or slow charts in their Cloud dashboards. We are investigating the issue.

Be the First to Know When Vendors Go Down

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 4522 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook