10
5
I'm thinking about using compressed NTFS on my external USB harddisk. It should be used for backups only.
maaartinus
Posted 2011-03-14T02:58:31.347
Reputation: 3 054
6
1) Yes, it is possible to access it from Liunx. As others explained, ntfs-3g is capable of handling compressed ntfs.
2) reliability: compression is in ntfs for a long time now (since Windows NT, see http://www.ntfs.com/ntfs-compressed.htm). I don't see any reliability problems on windows. I'm not that sure about non-windows implementations (e.g. linux). If your only concern is reading the data on linux, this is poses no reliability problem (mounting read-only is a good idea when restoring backups anyway).
Also data is compressed on cluster level which is usually 4 kiB. A physically corruption of the disk only affects single clusters, not entire files. In this regard compressed ntfs should be just as reliable as non-compressed.
3) Performance: The KB article you cite says performance might be affected on a server system, where the CPU is already saturated. On a desktop system with a current CPU as you describe it, using compressed ntfs for backups, should not have any significant performance impact. On the contrary, if you store compressible data, you might actually gain performance as you have less I/O. That is especially true, if the interface (USB 2.0) is slow compared to the CPU. I guess your CPU should easily be capable of saturating a USB 2.0 link writing or reading compressed ntfs.
4) If you set the compression flag for the entire filesystem you shouldn't have the problem of non-compressed moved files.
georg
Posted 2011-03-14T02:58:31.347
Reputation: 226
1
I think it is accessible from Linux, but I'm not sure.
Reliable in what sense? It's easier for your data to get corrupted, but I'm not sure if that's what you mean.
It's slower processor-wise, but faster I/O-wise. If your processor is much faster than your disk, then it might be worth it to read less data but do some processing, instead of reading huge files but avoiding postprocessing.
You can't avoid this, it depends on how the applications were programmed. I remember seeing a setting in Windows that had to do something with this, but I don't think it does what you need -- I'll update this if it turns out it does.
user541686
Posted 2011-03-14T02:58:31.347
Reputation: 21 330
Ad 2: I mean if there are no bugs in the filesystem or alike. Ad 4: Actually, I'm going to copy files from another non-compressed drive, probably using rsync. I really don't see why an application should care about the internals of the filesystem, IMHO it's the job of the FS to do all the work. – maaartinus – 2011-03-14T04:05:28.247
To pt#1 -> NTFS can be accessed @Mehrdad – Sathyajith Bhat – 2011-03-14T04:05:34.047
1@maartinus: #2: Well, I have no hard data on this, but personally I would trust the Windows version but not the Linux version. #4: It's simply a matter of keeping the default settings, not a matter of the application caring. When a 2-GB file moves from a non-compressed folder to a compressed folder, there's no reason for the file system to spend a long time compressing it -- it just leaves it uncompressed, hence the problem. I don't think there's much you can do about it. @Sathya: Ah thanks. – user541686 – 2011-03-14T04:27:02.270
I suppose this leaving file uncompressed happens only when files get moved from an uncompressed part of the same disk, which will not be the case. So everything is fine. – maaartinus – 2011-03-15T12:15:33.687
1
The ntfs-3g driver supports reading, appending and (recently) modifying compressed files.
Currently reading compressed files is supported by all ntfs-3g versions. Creating new compressed files, clearing contents, and appending data to existing compressed files are supported since ntfs-3g-2009.11.14. Modifying existing compressed files by overwriting existing data (or existing holes) are supported since ntfs-3g-2010.8.8.
The filesystem is as reliable as its usual Linux counterparts, ext3/ext4.
The ntfs-3g driver handles everything really well. (It might still have some bugs in modifying compressed files; as the above quote says, it has only been added in version 2010.8.8.)
(no answer)
When this is caused by programs creating an uncompressed file elsewhere and moving it to its intended location later, the workaround is easy: Re-enable compression on those files.
user1686
Posted 2011-03-14T02:58:31.347
Reputation: 283 655
ntfs-3g -V: 2013.1.13AR.1 As of Ubuntu 14.10, kernel 3.16, I still can not vouch for ntfs-3g's ability to even properly read compressed files on my Win8 GPT partition. Even copying a file over results in a different md5sum. – Marcos – 2014-10-04T22:04:43.783
4NTFS never transfers files compressed, in either the windows or the Linux driver. That optimization doesn't exist, sadly. You'd have to go extremely low level to achieve that: Create the file and metadata, pre-allocate the space, then raw write the compressed data to the MFT and allocated clusters. So in normal use, you won't gain anything I/O-wise. – SilverbackNet – 2014-01-27T02:01:26.443