DataRobot Outage History

DataRobot experienced intermittent service issues in US Production for 20.1 hours that prevented users from launching new workloads for Notebooks, Custom Models, and Custom Applications, while existing workloads remained unaffected. The disruption was caused by an ongoing AWS Availability Zone outage that led to resource allocation failures. Engineering resolved the underlying workload scheduling issue and confirmed all services were restored.

DataRobot experienced a 37-minute delay in processing messages on the JP MTS due to an autoscaling malfunction. Engineering resolved the issue by manually scaling up the deployment and applying infrastructure configuration changes. The service returned to normal operation with continued monitoring to ensure cluster stability.

A Quay.io outage caused degraded performance across the DataRobot platform for 12.2 hours, affecting the website, AI Catalog, Data Ingest, AI Apps, AutoML, Predictions, and API services. The engineering team monitored the external Quay outage and worked on mitigation efforts. The incident was resolved when Quay.io functionality was restored and all DataRobot environments were fully stabilized.

DataRobot's Managed AI Cloud experienced performance degradation affecting the API for 58 minutes. A fix was implemented and the incident was resolved after monitoring confirmed the solution was effective.

DataRobot experienced intermittent UI disruptions on their Managed AI Cloud platform that affected both the website and API for approximately 6 days. The incident was classified as minor severity. A fix was implemented and the issue was resolved after monitoring confirmed stability.

DataRobot experienced a network issue related to Kubernetes in their US cluster that impacted model deployment and predictions functionality. The incident lasted 2.1 hours, during which engineering identified the root cause and implemented a mitigation solution. The issue was fully resolved after monitoring confirmed the environment had completely recovered.