Outage in Logit.io

Logit.io Response to Apache Log4j 2 CVE-2021-44228

Resolved Minor
December 13, 2021 - Started over 3 years ago
Official incident page

Need to monitor Logit.io outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including Logit.io, and never miss an outage again.
Start Free Trial

Outage Details

Follow updates here on how our teams are currently responding with the highest priority to vulnerability CVE-2021-44228 that is impacting multiple versions of the Apache Log4j 2 which was disclosed publicly via Github on December 9th 2021. Logit.io engineers and security incident teams continue to actively analyse, identify and where necessary patch all affected log4j versions across all Logit.io data centre instances. Logit.io are initially updating and following the current remediation advice as a first priority as detailed here https://github.com/advisories/GHSA-jfh8-c2jp-5v3q "In previous releases (>=2.10) this behaviour can be mitigated by setting system property "log4j2.formatMsgNoLookups" to “true” or by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class)" This remediation will be applied to all versions of Logit.io instances and we will post further updates here as this is rolled out. In addition the Logit.io security team has analysed its logs and updated its monitoring of all internal services to include active alerting of any attempts to issue remote code execution via the JndiLookup class. The information leak in Log4j does not permit access to data within an Elasticsearch cluster but allows an attacker to extract certain environmental data via DNS. The data leaked is limited to that available via Log4j lookups. If you have any questions or concerns, please reach out to us via the normal support channels or email support@logit.io. You can also choose to subscribe to updates at the top of this page to receive all future notifications.
Latest Updates ( sorted recent to last )
RESOLVED over 3 years ago - at 12/14/2021 12:23AM

All stacks have been patched and are operational. We have removed the org/apache/logging/log4j/core/lookup/JndiLookup.class file from all installations of log4j

IDENTIFIED over 3 years ago - at 12/13/2021 08:44PM

All Elasticsearch clusters in the UK and US regions have been patched and updated. We continue in parallel to update the EU region

IDENTIFIED over 3 years ago - at 12/13/2021 08:05PM

We can confirm we have applied the patches to all Logstash Inputs for Stacks in the EU Regions.

IDENTIFIED over 3 years ago - at 12/13/2021 05:09PM

Update: We can confirm that we have now also applied the patches to Logstash for Filter nodes in the EU region.

IDENTIFIED over 3 years ago - at 12/13/2021 04:58PM

Update: We can confirm that we have now also applied the patches to Logstash for Filter nodes in the EU region.

IDENTIFIED over 3 years ago - at 12/13/2021 02:10PM

Update: Our engineers are in the process of rolling out a patched version of Logstash for both Input and Filter nodes for all active versions of ELK. We can confirm we have applied the patches to all Stacks in the UK and US Regions, and we are awaiting the completion of the EU Region.

IDENTIFIED over 3 years ago - at 12/13/2021 02:07PM

Update: Our engineers are in the process of rolling out a patched version of Logstash for both Input and Filter nodes for all active versions of ELK.
We can confirm we have applied the patches to all Stacks in the UK and US Regions, and we are awaiting the completion of the EU Region.

IDENTIFIED over 3 years ago - at 12/13/2021 10:29AM

Follow updates here on how our teams are currently responding with the highest priority to vulnerability CVE-2021-44228 that is impacting multiple versions of the Apache Log4j 2 which was disclosed publicly via Github on December 9th 2021.

Logit.io engineers and security incident teams continue to actively analyse, identify and where necessary patch all affected log4j versions across all Logit.io data centre instances.

Logit.io are initially updating and following the current remediation advice as a first priority as detailed here https://github.com/advisories/GHSA-jfh8-c2jp-5v3q

"In previous releases (>=2.10) this behaviour can be mitigated by setting system property "log4j2.formatMsgNoLookups" to “true” or by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class)"

This remediation will be applied to all versions of Logit.io instances and we will post further updates here as this is rolled out.

In addition the Logit.io security team has analysed its logs and updated its monitoring of all internal services to include active alerting of any attempts to issue remote code execution via the JndiLookup class.

The information leak in Log4j does not permit access to data within an Elasticsearch cluster but allows an attacker to extract certain environmental data via DNS. The data leaked is limited to that available via Log4j lookups.

If you have any questions or concerns, please reach out to us via the normal support channels or email support@logit.io. You can also choose to subscribe to updates at the top of this page to receive all future notifications.

Be the first to know when Logit.io and other third-party services go down

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 4200 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook