Cloud Management Services

This incident has been resolved.

After reviewing detailed error logs, our engineers identified an error indicating a vSan connection issue to an unrelated cluster inside vCenter.

The offending host was put into maintenance mode and restarted. After this action, vCloud Director was able to communicate with T0 and T1 storage for new provisions in vCenter.

No customer services went offline during this time.

The change made earlier was successful and vCloud director is available for customers.

vApps and VMs are still failing provisioning on T0 and T1 storage policies, existing vApps and VMs are unaffected.

All customer workloads are functioning as normal outside of new VM creation on the T0 and T1 storage policies.

We are continuing to work with the vendor.

After working with VMware/Broadcom today as mentioned in the latest update, we need to bring down the customer WebUI for vCloud Director to make modifications to the back end and resolve the issues we are seeing with storage policy availability.

Customer VMs and vApps will remain online during these works.

We will commence this work now.

The vCenter update was successful.

We are seeing some reporting return to normal.

vApps and VMs are failing provisioning on T0 and T1 storage policies, existing vApps and VMs are unaffected.

All customer workloads are functioning as normal outside of new VM creation on the T0 and T1 storage policies.

Our engineers will be working with a specialist VMware/Broadcom engineer in the morning at 9am (AEDT).

Next updates will be following that meeting.

As this is ongoing, in an attempt to resolve the issues we are seeing, our vendor has asked us to upgrade our version of vCenter.

Our team are going to apply this update now.

No running customer workloads should be affected at this time, VMs should remain online as we update this backend system.

Engineers have identified an issue where certain vsan storage policies are not syncing properly to vcloud director. This is causing new vApps to fail provisioning on t0 and t1 storage policies.

Existing vApps are not affected.

Engineers are continuing to work with the software vendor to resolve the issue.

Engineers have successfully rebooted vCenter and all services are online.

We will continue to monitor vCenter throughout the day. If you encounter any issues accessing vCenter please raise a case via the MySAU portal to seek assistance.

Engineers have identified an issue with the vCenter which backs our vCloud Director instance. To resolve the issue, the vCenter services will need to be restarted.

Whilst restarting is underway, the Virtual Machine console and associated power actions will be unavailable. Running workloads will not be affected, this is only relating to Management actions.