Network incident

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

All compute racks have been migrated to the new core switches.
Around 20sec of forwarding cut per rack. No service downtime has been seen.

LAG LACP servers have been migrated to core switch #2 as well (less than 10sec of fw cut per machine).

We are now finalizing the cabling of all remaining active/standby | LAG LACP servers toward the new core switch #1.

We are working to replace a whole switch stack, you may see some latencies increase.

The disk of one of the 2 core switches based on Cumulus of SD6 failed.
We use MCLAG on those core switches.

It was in the roadmap already to replace those Cumulus boxes by another vendor.
Instead of replacing the Cumulus by a spare and doing another migration later in the year, we will proceed with the migration now.

We migrated all the machines using active/standby on the new boxes, without downtime.

Tonight at 22h00 CEST / 20h00 UTC, we will migrate all other compute racks and servers using LAG LACP to the new boxes.
As LAG LACP requires the whole bundle to be brought down for the migration, a short downtime is expected when migrating the racks.

A network equipment is faulty we will proceed to an emergency maintenance.
You may see some lag during few seconds.