In this post, we will discuss about space efficiency techniques used by Virtual SAN to reduce the amount of space used and to protect the data. The two methods available are Deduplication and Compression and RAID 5/6 Erasure Coding. While Deduplication and compression reduces the space used, Erasure coding technique offers more protection and space-saving over the traditional RAID 1 mirroring. So let’s get into the details
To make the use of the Space Efficiency techniques mentioned, below are the prerequisites to be considered.
- Deduplication and Erasure Coding are available only on all-flash disk groups.
- On-Disk Format version 3.0 is required to implement this.
- A valid license is needed to make use of these.
- Deduplication and Compression can be enabled only when storage-claiming method is set to manual. This can be changed to Automatic once deduplication is enabled.
- RAID 5/6 Erasure Coding does not support stretched clusters but Deduplication and Compression is supported.
Deduplication and Compression
Let’s first understand what does each step in Deduplication and Compression do.VSAN performs block level deduplication and compression; redundant data within each disk group is drastically reduced. These techniques do not reduce redundancy across disk groups.
Deduplication eliminates redundant data blocks in a Disk Group where as compression reduces additional redundant data within each data block. Both the techniques are to be applied at once and cannot be applied individually. Though this is a cluster wide setting, Deduplication and Compression are applied on Disk Group basis.Deduplication is applied when data is de-staged from cache tier to capacity tier and Compression is applied after Deduplication and before the data is written to capacity tier.
VSAN does a rolling reformat of each diskgroup on each host and hence depending on the amount of data available, it takes much time for this process to complete and does not incur any VM downtime but may have performance issues.
Enabling Deduplication and Compression on a Hybrid model fails with below error.
I have recreated my lab as a All-Flash based model and now we are ready to go.
From the VSAN cluster settings, Click Edit and set the Claim mode to Manual and Enable the Deduplication and Compression by checking the box against it. Click OK.
Wait for the deduplication task to Finish. As discussed it takes time and the tasks can be monitored from the monitor section. Observe that changes are made to each disk group on each host. Having a lower version of On-Disk Format and enabling Deduplication will automatically upgrade the On-Disk Format version.
Once the Task completes, you can observe the capacity usage before and after enabling Deduplication and Compression.
The Used Before capacity is the logical space needed before enabling Dedup while the Used After space is the physical space occupied by the VSAN objects. The capacity overhead of the Deduplication and Compression is around 5% of the total raw capacity.
RAID 5/6 Erasure Coding
Using RAID 5/6 Erasure Coding with Deduplication and Compression enabled makes much more difference to the storage usage. RAID 5/6 erasure coding enables VSAN to tolerate two capacity device failures in the datastore and requires less space than the RAID 1 configuration. Below is the space usage when RAID 1 and RAID 5/6 are implemented. Also this gives the number of fault domains needed to implement RAID 5/6.
The below screenshots explain the RAID 5 and RAID 6 configuration in VSAN
RAID5/6 is a policy attribute as we discussed in our earlier post. So we would need to apply this setting via a policy on the virtual machine. Below are the settings that you need to change if you want RAID5/6 configured. RAID5/6 can be configured with 4 or 6 fault domains.
RAID 5/6 Erasure Coding does not support value a of 3 for Number of Failures to Tolerate. Now in my Lab setup, i am going to edit the storage Policy that we created earlier and set it to use RAID 5 as below. Click Ok.
Reapply the storage policy on the VM and we should be good. We now have Deduplication and Compression enabled and RAID 5 configured on top of it.
Thin Swap Objects
Another space saving technique introduced in VSAN 6.2 is the swap objects to be able to be provisioned as thin objects. Prior to 6.2 swap objects were thick provisioned with 100% Object Space Reservation. This option is good only when you do not overcommit because if you overcommit, you would still need the swap files when there is a memory contention. So, that was a warning! This is a host level advanced setting that has to be configured on each host that is participating in the VSAN cluster. Use the below from the SSH session to enable it. The setting is disabled by default.
esxcfg-advcfg -s 1 /VSAN/SwapThickProvisionDisabled
To disable it, set the value to 0.
Hope this was informative. Thanks!