4
1
I am getting a 50% larger "Size on Disk" than the "Size" of files would lead me to expect, as linked. I have some idea from other posts that I have read about cluster size and small file allocation to clusters causing waste but would that account for 300GB extra space used/wasted?
Any ideas on how I could resolve this disparity and reduce the Size on Disk please?
WorkerD
Posted 2017-07-17T13:43:56.100
Reputation: 49
3
You have applied the NAS tag. Does this mean you determined those statistics on a file share? What kind of NAS is it? A Linux-based one?
Starting from Windows 8.1 (or maybe only on Windows 8.1), Windows erroneously calculates used disk space using the “allocation roundup size” Samba reports. There’s an article explaining the meaning behind this Samba parameter and various other things. It’s 1 MiB by default so it’ll inflate the supposed disk space usage a lot when when lots of smaller files are present.
tl;dr: It’s a bug.
Daniel B
Posted 2017-07-17T13:43:56.100
Reputation: 40 502
This could well explain it. It's definitely in the right ballpark to explain the sheer quantity of extra slack being seen. Is there a way around this to find out the true amount of slack space? I'm assuming for now the only way to know would be from whatever disk management console the NAS provides... – Mokubai – 2017-07-19T06:53:22.870
Other operating systems (including older Windows versions) or du (directly on NAS via SSH) would probably do, I’ll check. – Daniel B – 2017-07-19T06:57:24.147
1Based on further investigation, Windows 7 just seems to assume a block size of 1024 Byte. That makes the wrong calculation less obvious. When mounted on Linux (mount.cifs), all values report correctly (via du). – Daniel B – 2017-07-19T07:08:40.367
1
General with FAT 12/16/32 changing the cluster size results in other difficulties like reduced MAX partition size. FAT16 2gb MAX@16k cluster, whoever thought this was a good idea, was wrong.....dead wrong.
DO NOT FORMAT your hdd with data on it. You should ONLY FORMAT, a brand new partition/drive!! I CAN NOT emphasize this enough.
You could probably use gparted or partedmagic bootable ISO to do this job.
The most effective thing you can do is to resize the clusters, or add another partition (shrink your existing partition, and add in the free space) and format new partition with smaller clusters.(don't accept the defaults when formatting)
In addition, you could place all/most the small files inside a zip file, but that could disrupt the functionality of your programs. Also you need to keep extracting and putting them back in the ZIP file, what a pain.
You could create and mount a VHD file from disk management, and it would get assigned its own drive letter. It would still be a bit slower, but would still allow you to format a virtual drive with a smaller cluster without the hassle of repartitioning nor the extract/compress hassle of ZIP or other file compression.
File/folder compression doesn't help as each file still uses 1 cluster minimum.
You can have .5k,1k,2k,or 4k clusters anything larger would have even more waste.
First you have to identify which files are wasting the most space, and group them by program.
Then depending on how much free space you have, move a couple of them over to the new drive/partition. You decide when the ratio is good enough for you to stop.
cybernard
Posted 2017-07-17T13:43:56.100
Reputation: 11 200
2Standard cautions apply : Formatting removes all data on the drive. If OP has a traditional Windows PC with a single partition, then he won't be able to format it while booted from the partition. – Christopher Hostage – 2017-07-17T17:12:02.880
To say nothing of the possibility that the OP doesn't have a good restore disk of any sort and will completely lose Windows too after doing a format. If you're going to tell someone to nuke their system, you need to spend some time in your advice on how to prepare to recover! The tag says NAS, but there's nothing in the question about that. – Joe – 2017-07-18T11:27:04.713
@Joe There I added a warning! – cybernard – 2017-07-18T11:52:15.127
4check the cluster size and file system type. Is it nfts and 4k, that usually the defaults and has good results. Search your file system see how many files you have smaller than the cluster size. – cybernard – 2017-07-17T13:53:53.360
1Even with an average file size of 0.5Kb with 64Kb clusters you'd be looking at a ballpark figure of about 24Gb of file slack for that many files (63.5Kb * 388285). I'm ignoring folders as iirc they're handled somewhat differently. What you've got there looks excessive to say the least. How is the disk formatted? – Mokubai – 2017-07-17T14:35:53.330
use notepad create a file containing Hi there save it, now how many bytes does that use on disk. – cybernard – 2017-07-17T15:06:11.857
1I agree with @Mokubai; there is over 700KB of unaccounted-for space per file and folder (combined). – lungj – 2017-07-17T15:37:50.290
May I ask why you removed the “nas” tag and the picture from your question? – Daniel B – 2017-07-24T09:07:28.250
I reverted the change. While it might be justified to remove the NAS tag depending on the circumstances, removing the picture while still stating ".. as linked." just doesn't make sense. The only thing resembling a link was that picture. – Seth – 2017-07-24T09:47:26.940