Outage in Presslabs

Emergency Kubernetes Upgrade on US nodes

Resolved Maintenance

March 17, 2022 - Started about 4 years ago - Lasted 12 days
Official incident page

Incident Report

We have identified a GKE bug that is causing taints on our nodes and websites downtimes. We are going to deploy an emergency upgrade on all GKE nodes in the US cluster. The expected downtime is ~10-20 minutes only for the wp-admin area of all the US-based instances. The cache for public-facing pages will be frozen during this time. For any issues please e-mail support@presslabs.com and we'll be happy to help you. More details about this bug can be found in this report: https://stackoverflow.com/questions/71065016/gke-taints-come-back-after-removing-them-when-clustermanagerinternal-patchcluste

Components affected

Presslabs US

Need to monitor Presslabs outages?

Monitor all your external dependencies in one place
Get instant alerts when outages are detected
Be the first to know if service is down
Show real-time status on private or public status page
Keep your team informed

Start monitoring for free

Latest Updates ( sorted recent to last )

IDENTIFIED about 4 years ago - at 03/17/2022 10:31AM

We have identified a GKE bug that is causing taints on our nodes and websites downtimes.

We are going to deploy an emergency upgrade on all GKE nodes in the US cluster.

The expected downtime is ~10-20 minutes only for the wp-admin area of all the US-based instances. The cache for public-facing pages will be frozen during this time.

For any issues please e-mail support@presslabs.com and we'll be happy to help you.

More details about this bug can be found in this report: https://stackoverflow.com/questions/71065016/gke-taints-come-back-after-removing-them-when-clustermanagerinternal-patchcluste

Latest Presslabs outages

Displaying logs in dashboard not working - 7 months ago

Frontend pages returning 404 on US clusters - 11 months ago

Backend unresponsive on some sites from US cluster - 12 months ago

wp-admin unavailable for some of the sites - about 1 year ago

Internal systems unavailable - over 1 year ago

The Status Page Aggregator with Early Outage Detection

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 6020 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook