Cloud Providers Health Report - December 2022

Check our December 2022 health report on the top most popular cloud providers. We analyze the health of the cloud providers based on the number of outages and problems during the month. The source of the data is made available by the cloud providers themselves via their status page. We normalize it and use it to generate the report.

Please analyze the information with a grain of salt as each provider has their own framework for reporting incidents, and in no way one provider having more outages, means that they are worse than any other. Also we are comparing providers with very different sizes and market share.

In this report, we compare AWS, Azure, DigitalOcean, Fly.io, Google Cloud, Heroku, Linode, Netlify, Vercel according to their reported outages.

AWS

Metrics

List of Outages

• 05/12MINORInternet Connectivity
• 06/12MINORIncreased errors viewing or updating payment information
• 06/12MINORIncreased API error rates
• 13/12MINORIncreased API Error Rates and Latencies
• 13/12MINORNetwork Connectivity
• 17/12MINORIncreased API Error Rates
• 24/12MINORImpaired EC2 instances

Azure

No incidents reported for December, 2022

DigitalOcean

Metrics

List of Outages

• 04/12MINORNetwork Connectivity in BLR1
• 06/12MINORNetwork Connectivity in TOR1
• 06/12MINORNetwork Connectivity in NYC3
• 06/12MINORCloud Control Panel and API
• 09/12MINORPayments via Paypal
• 12/12MAINTENANCECSS-Tricks Connectivity
• 17/12MINORApp Platform
• 29/12MINORNetwork Connectivity in SYD1

Fly.io

Metrics

List of Outages

• 05/12MAINTENANCEHost failure in ORD
• 06/12MINORCDG host network issues
• 10/12MINORDelayed private DNS entries, machines list and proxy upstreams
• 13/12MAJORDeployment issues
• 13/12MINORDeployment issues
• 17/12MAJORDNS resolution failures for fly.dev
• 19/12MAINTENANCEDeplyoment Issues
• 23/12MINORApps with a shared IPv4 and a dedicated IPv6 are inaccessibly over IPv6
• 28/12MINORDeployments may take 10-15min in fra region

Google Cloud

Metrics

List of Outages

• 01/12MINORMonarch Tcube Query Availability drop in cloud_prod
• 01/12MAJORApp Engine Flexible Control Plane Create Update and Delete endpoints are failing
• 02/12MAJORGoogle Compute Engine. ssh fails with ERROR: (gcloud.compute.start-iap-tunnel) Error while connecting [4047: 'Failed to lookup instance']
• 07/12MINORIntermittent failures (ERROR: PERMISSION_DENIED: The caller does not have permission) when trying to list/describe the OAuth client via gCloud or Terraform
• 07/12MINORGoogle Cloud iOS mobile app errors
• 08/12MINORGlobal: Cloud VPN tunnel creation failures via Terraform
• 08/12MINORClusters in us-central1 are getting DEADLINE_EXCEEDED errors
• 09/12MINORPerformance degradation of Cloud Monitoring notifications
• 09/12MINORGoogle Kubernetes Engine: INTERNAL_ERROR when performing ClusterCreation in Asia regions
• 09/12MINORGoogle Cloud Storage IAMBACKENDINVALIDARGUMENT errors. INTERNALERROR when performing ClusterCreation in Google Kubernetes Engine and Artifact Registry in Asia
• 09/12MINORMultiple products may return error messages across Asia/Australia
• 12/12MINORGlobal: G Cloud Command loading performance issue
• 13/12MINORApigee Live Portals management operations are currently experiencing intermittent issues
• 14/12MINORLogs delayed in europe-west3
• 15/12MINORCloud Monitoring: Missing Bigtable Replication Metrics
• 15/12MINORDelay in Data Stretching
• 15/12MINORIncrease in CreateTimeSeries DEADLINE_EXCEEDED errors globally
• 15/12MINORGlobal: BigQuery Federated Queries are returning errors for VPC-SC customers
• 16/12MINORCalico Node pods failing to start and crash-looping
• 16/12MINORMulti Zonal: Google Compute Engine Experiencing VM failure
• 20/12MINORGoogle engineers are investigating a global issue with Cloud Memorystore.
• 21/12MINORCustomers are unable to create or modify L7 Internal Load balancers using the Cloud Console UI with the logging option enabled
• 21/12MINORus-central1: Degraded performance for Cloud Networking
• 21/12MINORus-central1: Cloud Build High Queue Latency
• 22/12MINORIncorrect Enterprise/High Scale latency metrics in the Cloud Filestore
• 28/12MINORWe are experiencing increased tail latency for predictions on Vertex AI Online Prediction.

Heroku

Metrics

List of Outages

• 03/12MINORIssues with DNS provider
• 06/12MINORPerformance degradation with Heroku Webhooks
• 07/12MINORDNS propagation delays in Common Runtime and Private Spaces
• 08/12MINORPlatform API errors affecting Dashboard and Heroku CLI
• 09/12MINORRouting Performance Disruption
• 29/12MINORIssues with DNS provider

Linode

Metrics

List of Outages

• 09/12MINORService Issue - Object Storage - AP-South (Singapore)
• 13/12MINORConnectivity Issue - Dallas, Cloud Manager, API

Netlify

Metrics

List of Outages

• 05/12MINORFailure of Some GitHub-triggered Builds
• 05/12MINORInvestigating build logs not showing in our UI
• 08/12MINORIncreased errors in Serverless functions
• 08/12MINORIncreased Errors in Edge Functions
• 13/12MINORIncreased errors and latency on the Standard Edge Network
• 13/12MINORBuilds failing to build
• 17/12MINORBuild failures
• 21/12MAJORIssue with logging into Netlify
• 21/12MINORAsset Optimization Content serving 404

Vercel

Metrics

List of Outages

• 05/12MINORElevated Errors in Region CLE1
• 07/12MINORBuild Pipeline Degradation
• 08/12MINORDNS Record and Wildcard Cert issue Delays
• 11/12MINORElevated error rates in Serverless Function invocations
• 12/12MINORElevated Errors in the Edge Network