SSD slow disk in VS affects customer production.

Bug Phenomena

SSD slow disk in VS affects customer production.

Trigger conditions

The hardware failure of the SSD hard disk causes IO to not return or the IO delay is very large. From experience, new hard disks within 3 months and hard disks that have been used for more than 2 years have a higher probability of occurrence.

Root Cause

When the SSD cache disk becomes a slow disk, the disk is still online, and there will still read and write operation to this slow disk. However, as the SSD is a slow disk, it will either not replying to the IO request or having long IO latency towards the request and resulting in VM IO stuck or having large IO latency.

How to check

If a disk has no IO (r/s and w/s are both <=1), but the utilization (%utils) is 100%, it indicates that the disk is a slow disk, and the IO is stuck and will not reply to IO request.

As old version of HCI do not support the slow cache disk isolation function, when the SSD disk becomes a slow disk, the disk cannot be isolated.  

Solution

Upgrade to version 6.3.0R3 or 6.7.0R2.