NTFS compressed folders: is it possible to tweak compression ratio?

Question

As it seems, Windows compresses the NTFS compressed folders with the lowest compression ratio possible. While this maybe good for increasing speed and decreasing CPU load, for the files that are accessed rarely (backup folders) it would be more sensible to have the possibility to increase the compression ratio. Are there some parameters that may be tweaked to achieve this?

Answer 1

Short answer

No, it's not possible at this time.

Long answer

Files and folders¹ are compressed and decompressed by passing a FSCTL_SET_COMPRESSION control code and a compression state to the DeviceIoControl API function. The compression state can be one of the following:

COMPRESSION_FORMAT_NONE = 0
COMPRESSION_FORMAT_DEFAULT = 1
COMPRESSION_FORMAT_LZNT1 = 2

Any nonzero value means the target item is going to be compressed. From the official documentation:

The LZNT1 compression algorithm is the only compression algorithm implemented. As a result, the LZNT1 compression algorithm is used as the DEFAULT compression method.

_{Source: FSCTL_SET_COMPRESSION control code}

The LZNT1 algorithm is designed for speed, and there's no way to set a custom compression level.

_{¹ Folders aren't actually compressed: their compression attribute only gives a default compression state to new files and subfolders.}

Additional information

The NTFS file system volumes support file compression on an individual file basis. The file compression algorithm used by the NTFS file system is Lempel-Ziv compression. This is a lossless compression algorithm, which means that no data is lost when compressing and decompressing the file, as opposed to lossy compression algorithms such as JPEG, where some data is lost each time data compression and decompression occur.

On the NTFS file system, compression is performed transparently. This means it can be used without requiring changes to existing applications.

If you compress a file that is larger than 30 gigabytes, the compression may not succeed.

_{Source: File Compression and Decompression}

The compression algorithms in NTFS are designed to support cluster sizes of up to 4 KB. When the cluster size is greater than 4 KB on an NTFS volume, none of the NTFS compression functions are available.

_{Source: File and Folder Compression}

Further reading

• File Compression and Decompression
• File and Folder Compression
• Understanding NTFS Compression
• LZNT1 Algorithm Details

Answer 2

Generally it's not possible, as indicated in and31415's answer.

However Microsoft has added some new NTFS compression options and algorithms in Windows 10, so now there's a way to change the compression ratio:

COMPACT [/C | /U] [/S[:dir]] [/A] [/I] [/F] [/Q] [/EXE[:algorithm]]
        [/CompactOs[:option] [/WinDir:dir]] [filename [...]]
...
  /EXE       Use compression optimized for executable files which are read
             frequently and not modified.  Supported algorithms are:
             XPRESS4K  (fastest) (default)
             XPRESS8K
             XPRESS16K
             LZX       (most compact)

The new algorithms are intended for the new Compact OS feature (as you can see in the /CompactOs option above). The idea is to compress read-only, backup and less frequently accessed system files with the highest ratio. The recovery partition is removed and the compressed file will then be used for both executing (if it's already the latest version) and recovery purpose

The LZX algorithm is quite efficient at archiving. And despite the option name and description are all about "executable files", the option can be applied to any files

E:\test>compact /a

 Listing E:\test\
 New files added to this directory will not be compressed.

  1050909 :   1050909 = 1.0 to 1   sometext.txt

...

E:\test>compact /c /exe:lzx sometext.txt

 Compressing files in E:\test\

sometext.txt          1050909 :    176128 = 6.0 to 1 [OK]

1 files within 1 directories were compressed.
1,050,909 total bytes of data are stored in 176,128 bytes.
The compression ratio is 6.0 to 1.

However unlike the old algorithm they don't support on-the-fly editing, so writing back to the file uncompresses it.

E:\test>echo x >> sometext.txt

E:\test>compact /a

 Listing E:\test\
 New files added to this directory will not be compressed.

  1050913 :   1050913 = 1.0 to 1   sometext.txt

---

An important note is that you can't set the folder to mark new files to be compressed automatically using those new algorithms, because as stated in the help part of compact (emphasis mine)

  /C         Compresses the specified files.  Directories will be marked
             so that files added afterward will be compressed ***unless /EXE
             is specified***.
  /U         Uncompresses the specified files.  Directories will be marked
             so that files added afterward will not be compressed.  If
             /EXE is specified, only files compressed as executables will
             be uncompressed; if this is omitted, only NTFS compressed
             files will be uncompressed.

Therefore if you want to use them for back-up files you might have to run a script to compress manually after backing up or periodically after some time

Unfortunately this new in Windows 10, therefore it can't be used in older versions. However NTFS-3g does support it, therefore you won't have problem accessing it from Linux. If you don't use Windows 10 you can run a Linux live USB or Windows 10 PE to open those files

Answer 3

You can increase compression ratio by 3rd-party apps. The prime example is zipmagic.co I've used it on my laptop for a year without any problem. But notice that using NTFS or Zipmagic compression needs SSD hard drive because compression will make your drive totally fragmented and make it useless! If you have SSD drive, so go ahead and enjoy!

Answer 4

I would advise to use an archiver such as 7-zip or Winrar for this purpose. With Winrar, it is even possible to access a file that is inside an archiver. Do note that it will first extract the entire archive before it can access the file, and it will update the archive with modifications when you close and save to the file. This can make the entire process slow, but given that it is a backup that should be okay.

Do note that this is not a full backup solution. Its only a way to save space.

Ontop of that, there is a program that can mount 7z and rar archives as a virtual drive: http://www.winarchiver.com/virtual-drive.htm