Google Cloud Outage

We're happy to report that the issue has been resolved, and everything is back to normal. We apologize for any inconvenience this may have caused and appreciate your patience while we worked to resolve the issue. Please let us know if you have any further questions or concerns.

Message from the Google team:

"The following Google Cloud products are still experiencing residual impact:

Google Cloud Dataflow: Customers may experience delays with Dataflow operations as the backlog is clearing up gradually.

Vertex AI Online Prediction: Customers may continue to experience elevated 5xx errors with some of the models available in the Model Garden.

Personalized Service Health: Updates on the Personalized Service Health are delayed and we recommend customers to continue using Cloud Service Health dashboard for updates."

From the Google team:

"Most of the Google Cloud products have confirmed full service recovery.

A few services are still seeing some residual impact and the respective engineering teams are actively working on recovery of those services.

We expect the recovery to complete in less than an hour."

This is from the Google Team:

"We have implemented mitigation for the issue in us-central1 and multi-region/us and we are seeing signs of recovery.

We have received confirmation from our internal monitoring and customers that the Google Cloud products are also seeing recovery in multiple regions and are also seeing signs of some recovery in us-central1 and mutli-region/us.

We expect the recovery to complete in less than an hour."

The update from Google states:

"We have identified the root cause and applied appropriate mitigations. Our infrastructure has recovered in all regions except us-central1.

Google Cloud products that rely on the affected infrastructure are seeing recovery in multiple locations.

Our engineers are aware of the customers still experiencing issues on us-central1 and multi-region/us and are actively working on full recovery."

The update from Google states:

"Our engineers have identified the root cause and have applied appropriate mitigations.

While our engineers have confirmed that the underlying dependency is recovered in all locations except us-central1, we are aware that customers are still experiencing varying degrees of impact on individual google cloud products. All the respective engineering teams are actively engaged and working on service recovery."

Update: The issue impacting our Reporting Hub is now also affecting our OCA and Klik products. Customers may continue to see a "Trouble Loading Data" error when accessing these services. Google Cloud is actively working on resolving the issue. We will keep monitoring closely and provide updates as they become available. For the latest status, please visit Google's status page (https://status.cloud.google.com/).

Thank you for your continued patience. For additional assistance, please contact us at support@bizzabo.com.

We are currently experiencing an issue impacting our Reporting Hub. Customers may encounter a "Trouble Loading Data" error when viewing their reports. This issue has been identified as related to an ongoing incident with Google Cloud (https://status.cloud.google.com/). We are actively monitoring the situation and will provide updates as new information becomes available.

Thank you for your patience. For any additional concerns, please contact us at support@bizzabo.com.