Outage in Google Cloud

MDP (an istio component) evicts pods continuously when the image URL is overwritten by Kubernetes-digester for versions of istio: 1.14.6-asm.23 & 1.14.6-asm.24

Resolved Minor

November 21, 2023 - Started about 2 years ago - Lasted about 8 hours

Incident Report

Summary: MDP (an istio component) evicts pods continuously if image url is overwritten by k8s-digester for versions of istio: 1.14.6-asm.23 & 1.14.6-asm.24 Description: We've received a report of an issue with Anthos Service Mesh as of Tuesday, 2023-11-21 06:39 US/Pacific. We will provide more information by Wednesday, 2023-11-29 10:00 US/Pacific. Diagnosis: Anthos Service Mesh (ASM) pods are continuously (and sometimes simultaneously) evicted if the proxy image url is overwritten by k8s-digester. Workaround: Customers should disable MDP if they are impacted. Instruction how to disable MDP is available here: https://cloud.google.com/service-mesh/docs/managed/provision-managed-anthos-service-mesh#disable_the_managed_data_plane_optional Please disable Managed ASM individually on each namespace with the following commands: kubectl annotate --overwrite controlplanerevision -n istio-system \ REVISION_LABEL mesh.cloud.google.com/proxy='{"managed":"false"}'

Components affected

Anthos Service Mesh Anthos Service Mesh (us-central1) Anthos Service Mesh (us-east1) Anthos Service Mesh (us-east4) Anthos Service Mesh (us-east5) Anthos Service Mesh (us-west1) Anthos Service Mesh (us-west2) Anthos Service Mesh (us-west3) Anthos Service Mesh (us-west4) Anthos Service Mesh (northamerica-northeast1) Anthos Service Mesh (northamerica-northeast2) Anthos Service Mesh (southamerica-east1) Anthos Service Mesh (southamerica-west1) Anthos Service Mesh (europe-west1) Anthos Service Mesh (europe-west2) Anthos Service Mesh (europe-west3) Anthos Service Mesh (europe-west4) Anthos Service Mesh (europe-west6) Anthos Service Mesh (europe-west8) Anthos Service Mesh (europe-west9) Anthos Service Mesh (europe-central2) Anthos Service Mesh (europe-north1) Anthos Service Mesh (europe-southwest1) Anthos Service Mesh (asia-east1) Anthos Service Mesh (asia-east2) Anthos Service Mesh (asia-northeast1) Anthos Service Mesh (asia-northeast2) Anthos Service Mesh (asia-northeast3) Anthos Service Mesh (asia-south1) Anthos Service Mesh (asia-south2) Anthos Service Mesh (asia-southeast1) Anthos Service Mesh (asia-southeast2) Anthos Service Mesh (australia-southeast1) Anthos Service Mesh (australia-southeast2)

Need to monitor Google Cloud outages?

Monitor all your external dependencies in one place
Get instant alerts when outages are detected
Be the first to know if service is down
Show real-time status on private or public status page
Keep your team informed

Start monitoring for free

Latest Google Cloud outages

We are investigating elevated error rates with multiple products in us-east1 - 6 months ago

Multiple GCP products are experiencing Service issues - 7 months ago

Google Compute Engine (GCE) issue impacting multiple dependent GCP services across zones - 8 months ago

Customers are experiencing connectivity issues with multiple Google Cloud services in zone us-east5-c - 10 months ago

Apigee customers may experience unable to login to Admin UI portal. - 11 months ago

The Status Page Aggregator with Early Outage Detection

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 5420 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook