Outage in Google Cloud
Cloud TPU Service Activation is impacted.
Resolved Minor
August 07, 2024 - Started over 1 year ago
- Lasted about 2 hours
Incident Report
Summary: Cloud TPU Service Activation is impacted.
Description: We are investigating issues where customers using TPUs through Vertex AI Service or Cloud TPU Service/API directly may be experiencing API activation failures. This can happen for new customers or existing customers who disable and reactivate the TPU Service beginning on Wednesday, 2024-08-07 12:00 US/Pacific.
Our engineering team has rolled out a mitigation and is progressing as expected.
We will provide an update by Wednesday, 2024-08-07 19:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Impacted customers would not be able to enable Cloud TPU APIs in their projects and would encounter an error message that states, e.g. "Failed to upload model. Contact Vertex AI."
We continue to investigate impact to other Google Cloud Services and products.
Workaround: None at this time.
Components affected
Vertex AI Online Prediction Vertex AI Online Prediction (europe-west3) Vertex AI Training (northamerica-northeast1) Vertex AI Training (us-east1) Vertex AI Training (us-central1) Vertex AI Online Prediction (asia-southeast1) Vertex AI Training (asia-south1) Vertex AI Training (europe-west3) Vertex AI Training (europe-west1) Vertex AI Online Prediction (northamerica-northeast2) Vertex AI Online Prediction (us-west1) Vertex AI Online Prediction (us-east4) Vertex AI Online Prediction (asia-northeast1) Vertex AI Training (asia-northeast3) Vertex AI Online Prediction (europe-west2) Vertex AI Online Prediction (northamerica-northeast1) Vertex AI Training (europe-west2) Vertex AI Online Prediction (asia-east1) Vertex AI Training (europe-west4) Vertex AI Training (asia-northeast1) Vertex AI Online Prediction (us-central1) Vertex AI Online Prediction (europe-west1) Vertex AI Online Prediction (us-west2) Vertex AI Online Prediction (europe-west6) Vertex AI Training (asia-southeast1) Vertex AI Online Prediction (asia-east2) Vertex AI Training (northamerica-northeast2) Vertex AI Training (asia-east2) Vertex AI Training Vertex AI Training (asia-east1) Vertex AI Training (us-east4) Vertex AI Online Prediction (europe-west4) Vertex AI Training (australia-southeast1) Vertex AI Online Prediction (asia-south1) Vertex AI Training (europe-west6) Vertex AI Training (us-west1) Vertex AI Online Prediction (us-east1) Vertex AI Online Prediction (australia-southeast1) Vertex AI Online Prediction (asia-northeast3) Google Cloud Infrastructure Components Vertex AI Training (us-west2) Cloud Developer Tools Operations Cloud Machine LearningTrusted by 1,000+ teams
The Status Page Aggregator with Early Outage Detection
Stop finding out about outages from your users. Monitor 6,320+ cloud services and get alerted the second something breaks.
Start Free Trial
- No credit card
- 14-day trial
- 2-minute setup
Latest Google Cloud outages
Multiple GCP products are experiencing Service issues - 10 months ago
Google Compute Engine (GCE) issue impacting multiple dependent GCP services across zones - 11 months ago
Customers are experiencing connectivity issues with multiple Google Cloud services in zone us-east5-c - about 1 year ago
Apigee customers may experience unable to login to Admin UI portal. - about 1 year ago
The Status Page Aggregator with Early Outage Detection
With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.
Start free trialNo credit card required · Cancel anytime · 6320 services available
Integrations with
