Cloud and Enterprise Agents showing DNSSEC failures.

This incident is now resolved.

Please follow the instructions using one of the options below if any of your Enterprise Agents cannot automatically
update themselves.


Option 1: Add the New DNSSEC Key
Adding the new DNSSEC key can restore the Agent’s connection to the ThousandEyes platform. After the Enterprise Agent
reconnects, the Test Data collected since the Enterprise Agent’s last check-in should be retroactively pushed to the
Test timeline as long as the Test Data remains in the Enterprise Agent’s local cache.

The local cache retains Test Data for up to 36 hours after connectivity is lost; data older than that may be
permanently lost.
Cisco Application-Hosting (CAF) Agent
Access the CLI of the Cisco device hosting the CAF Agent.

Open a terminal in the app container:

app-hosting connect <app-id> session bash

Append the new DNSSEC key to the key file (note – this is all one line):

echo ". 172800 IN DNSKEY 257 3 8
AwEAAa96jeuknZlaeSrvyAJj6ZHv28hhOKkx3rLGXVaC6rXTsDc449/cidltpkyGwCJNnOAlFNKF2jBosZBU5eeHspaQWOmOElZsjICMQMC3aeHbGiShvZsx4wMYSjH8e7Vrhbu6irwCzVBApESjbUdpWWmEnhathWu1jo+siFUiRAAxm9qyJNg/wOZqqzL/dL/q8PkcRU5oUKEpUge71M3ej2/7CPqpdVwuMoTvoB+ZOT4YeGyxMvHmbrxlFzGOHOijtzN+u1TQNatX2XBuzZNQ1K+s2CXkPIZo7s6JgZyvaBevYtxPvYLw4z9mR7K2vaF18UYH9Z9GNUUeayffKC73PYc="
>> /usr/lib/te-agent/dns-rootserver.key
Docker Agent
Run the following command (note – this is all one line) on the Docker host, replacing
<CONTAINER_NAME> with the Agent’s container name:

docker exec <CONTAINER_NAME> sh -c 'echo ". 172800 IN DNSKEY 257 3 8
AwEAAa96jeuknZlaeSrvyAJj6ZHv28hhOKkx3rLGXVaC6rXTsDc449/cidltpkyGwCJNnOAlFNKF2jBosZBU5eeHspaQWOmOElZsjICMQMC3aeHbGiShvZsx4wMYSjH8e7Vrhbu6irwCzVBApESjbUdpWWmEnhathWu1jo+siFUiRAAxm9qyJNg/wOZqqzL/dL/q8PkcRU5oUKEpUge71M3ej2/7CPqpdVwuMoTvoB+ZOT4YeGyxMvHmbrxlFzGOHOijtzN+u1TQNatX2XBuzZNQ1K+s2CXkPIZo7s6JgZyvaBevYtxPvYLw4z9mR7K2vaF18UYH9Z9GNUUeayffKC73PYc="
>> /usr/lib/te-agent/dns-rootserver.key'
Linux Package Agent
Append the new DNSSEC key to the key file (note – this is all one line):

echo ". 172800 IN DNSKEY 257 3 8
AwEAAa96jeuknZlaeSrvyAJj6ZHv28hhOKkx3rLGXVaC6rXTsDc449/cidltpkyGwCJNnOAlFNKF2jBosZBU5eeHspaQWOmOElZsjICMQMC3aeHbGiShvZsx4wMYSjH8e7Vrhbu6irwCzVBApESjbUdpWWmEnhathWu1jo+siFUiRAAxm9qyJNg/wOZqqzL/dL/q8PkcRU5oUKEpUge71M3ej2/7CPqpdVwuMoTvoB+ZOT4YeGyxMvHmbrxlFzGOHOijtzN+u1TQNatX2XBuzZNQ1K+s2CXkPIZo7s6JgZyvaBevYtxPvYLw4z9mR7K2vaF18UYH9Z9GNUUeayffKC73PYc="
>> /usr/lib/te-agent/dns-rootserver.key

Please note: Customers will still need to complete Option 2 for a long-term fix, even if Option 1 is applied in
the interim.

Option 2: Upgrade the Agent Image
Cisco Application-Hosting (CAF) Agent
Run the following command on the Cisco device hosting the CAF Agent:

If you have x86 architecture:

app-hosting upgrade appid package
https://downloads.thousandeyes.com/enterprise-agent/thousandeyes-enterprise-agent-x86_64-5.2.1.cisco.tar

If you have ARM architecture:

app-hosting upgrade appid package
https://downloads.thousandeyes.com/enterprise-agent/thousandeyes-enterprise-agent-aarch64-5.2.1.cisco.tar

Please note: Upgrading the Cisco App-Hosting Image (CAF) Agent image will clear the Enterprise Agent’s local Test
Data cache. The
Test timeline will have a data gap for the affected period, and the cached Test Data will not be backfilled.
Docker Agent
Use the latest image tag (or latest-agent if you are not using BrowserBot), or Docker image version
0.105.0 or later.

docker pull <name>

docker stop <running-agent>

docker rm <running-agent>

docker run <same options as previous launch>
Linux Package Agent
Upgrade the te-agent package to the latest version (1.247.2) using the system package manager
(yum,
dnf, or apt, depending on your distribution).

For more details, see href="https://docs.thousandeyes.com/product-documentation/global-vantage-points/enterprise-agents/managing/upgrading-enterprise-agents">Upgrading
Enterprise Agents.

New updates are being pushed for Enterprise Agents.
Enterprise Agents showing online are out of scope.

Customers with offline Enterprise Agents should reach out to Standard ThousandEyes Support Channels to manually update configurations.

A fix has been implemented. Operations will continue to monitor performance.

We are continuing to investigate this issue.

Cloud and enterprise agents are showing failure message in DNSSEC tests due intermediate validation.
The issue started at 14:00 PM UTC.