Power Infrastructure Incident in VIT

We are marking this incident as resolved. The Root Cause Analysis will be published here once compiled.

The last host and VMs have been recovered. We are setting this incident into monitoring status.

Preliminary information on the Incident:
Following a power grid outage, the site suffered a partial interruption of power, which impacted several datacenter support systems. The reason for this is under active investigation. This affected the availability of a subset of infrastructure components and hosts, subsequently impacting the VMs provisioned on them or dependent on those components.

A full Root Cause Analysis (RCA) is currently being compiled and will be shared here once completed.

Thank you for your patience.

All but one hosts are recovered. The remaining host suffered a hardware defect and needs to be replaced to recover the workload provisioned there. This is currently underway. We estimate that this work will be completed within the next minutes.

Most affected hosts have been brought back into service. We are working on restoring the remaining components.

Power has been restored and site support systems are back in operational state. We have started recovering infrastructure components to fully restore service.

We are currently responding to an incident in our VIT datacenter. We will regularly update this status page with information.