Friday 5th October 2018

[IMPORTANT] Partial Cloud Failure Advisory and Resolution, scheduled 8 months ago

On Friday, 05 Oct. 2018 at 17:07 UTC the network connection to one of our BladeSystems was interrupted which made the blades and virtual servers inaccessible. We have identified an issue within the VirtualConnect's firmware and have rebooted both switches to restore communication. Our network is designed with redundancy in mind and our redundancy didn't fail (which is a good thing). During a reboot of a blade, a bug in the firmware was triggered which deadlocked both switches and both LACP channels were interrupted.

We apologise for the inconvenience. We will update the switches to a newer firmware that fixes these issues on Saturday, 06 Oct. 2018 at 10:00 UTC. As some blades are not redundant (non Block Storage/Customer Dedicated Blades) the network connection may be interrupted up to 10 minutes. Cloud Block HDD instances are not affected by this maintenance.

HP has published an advisory here: a00029108en_us

Thank you very much for your understanding!