14

One of the new features that I didn't see listed in all the "Windows Server 2012 r2" posts is that Clustering now supports CSVs that are formatted with ReFS. So, naturally, I would like to change the CSVs where I store the VHDX files to be ReFS. But the VHDX files are being used to store database files in VMs running Sql Server 2012.

The thought is that I would then have RAID at the hardware level, protecting against instantaneous failure. Above that, the real OS (Hyper-V Server 2012 r2) would maintain them as ReFS volumes, which would protect data on those drives against bitrot. Finally, VHDXs are NTFS drives, which means the applications being supported continue to use the filesystem they rely on.

So far, the best I can find is that this is technically supported---because Hyper-V reports that you must turn off the "data integrity" setting in the VHDX file (Set-FileIntegrity cmdlet) when you try to use it from the ReFS volume. But I can't find any more solid information than that. Is it really ready for prime-time, or is it effectively just a tech-preview for clustering?

Edit: 2014-01-22

I found that ReFS only detects bitrot by itself. In order to have ReFS both detect and auto-fix, you must also use Storage Spaces to create a RAID-1 volume using multiple ReFS drives. So it's looking like my solution is evolving into having the hardware RAID present its disks as JBOD, then Windows would take care of the RAID-1 part. I'll be testing if this is a viable setup in Production over the next month or so.

Granger
  • 1,150
  • 2
  • 10
  • 26

2 Answers2

14

The answer is a very clear "No".

ReFS only detects bit rot in user data if the file in question has "Integrity Streams" enabled (Sources: official TechNet docs, everyone's favorite blog post, and another spot). Oh, and you also lose COW (Copy-On-Write) when Integrity Streams are disabled. Since you cannot use a VHDX residing on a ReFS volume unless Integrity Streams is disabled, you cannot protect a VHDX against bit rot. Game Over.

It's like the same person who thought a Clustered Storage Space Pool should require at least 3 disks was also the one that made the decision to make the best thing about ReFS something you could turn off, and then got the Hyper-V people to require it to be disabled. It's hard to imagine that amount of "dumb" spread out so far across core teams like that.

Ancillary

While doing some testing, I found the following that may be useful to people who still wish to move forward:

  • You can only SLM (Storage Live Migrate) an in-use VHDX to a ReFS-mirror volume if your destination is a folder where Integrity Streams have been disabled.
    • If you attempt to do SLM onto a ReFS-mirror where Integrity Streams is enabled, you'll get an error with this in it: "The destination '...' is not valid because it is configured with the integrity stream attribute. Select a destination that does not have the integrity stream attribute to continue.". You get the same error when attempting via PowerShell.
  • Copying/Moving a file onto a ReFS-mirror will result in the file having its "integrity bit" set to match the setting from the destination folder.
  • You cannot get/set the integrity bit of a VHDX that is in-use.
  • Otherwise, the performance of a ReFS-mirror volume appears to be good enough (my opinion, of course) for Production. My "differences" test is here if anyone cares.
Granger
  • 1,150
  • 2
  • 10
  • 26
  • 3
    I wouldn't assume the engineers at MS are dumb, rather there are some difficult problems which arise with your desired solution and they either couldn't resolve them in time or it wasn't possible to make it reliable. – Andy Mar 11 '14 at 01:24
  • If you'll notice, this is not "stupid". Linux systems have similar limitations, but don't enforce them. Sure, you can put a qcow2 image on top of a BTRFS volume with checksumming enabled - but it's going to perform like garbage for most workloads. Turn off checksumming, and it's much better - yet you still get the volume features, etc of BTRFS. If you're that worried about it, put a checksumming ReFS in the VM image. – Spooler Oct 30 '18 at 14:36
0

ReFS is supported, with data integrity disabled, as you discovered. What this means is that your VHD is not "protected against bitrot" as you say above. The file system itself would be, but not the VHD itself. If this is measure of protection is interesting to you, go ahead and use ReFS.

Jake Oshins
  • 5,116
  • 17
  • 15
  • You're both right and wrong, considering what I think "protect" means in this case. ReFS by itself will detect and notify you of bitrot, but it won't auto-fix it for you. In order for ReFS to truly protect against bitrot (detect and auto-fix), you have to use Storage Spaces to create a OS-level RAID-1 volume from multiple ReFS drives. ...so my original scenario won't work unless I sacrifice more space (RAID-1 on top of RAID-1). – Granger Jan 23 '14 at 03:38