A follow-up to the previous article about Create LVM thin provisioning with SSD cache. After a while with performance issues I have upgraded the mirror of two WD Black with a RAID5 of WD Red.
I had smartmontools tell me every now and then that a sector of one of the WD Black in the RAID1 mirror was bad. Often the system was slow and the CPU idling. What I have come up with is that the NVMe SSD throttled down when getting hot. Not good when used as a cache. Also a faulty drive is bound to cause issues.
I have now upgraded two WD Black to three WD Red, new chassis to be able to house them and I utilized my old 3Ware RAID controller.
I first went with LVM Thin Provisioning but when the volume has to grow all the time it was also really slow. I reinstalled with ordinary LVM. The VMs are still on a XFS volume as QCOW2 files but that is to be migrates. I attached an LVM volume to a VM and tested performance. I get near gigabit speeds over CIFS from that volume.
The 3Ware have a BBU cache so writes are fast too.
Fredrik Rambris 