Calculate minimum ext3 partition size for certain amount of data

These following ext3 partitions contain identical data. As we can see, the larger the partition size, the more space is required for the same files:

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/loop11            3965777    561064   3199964  15% [...]
/dev/loop19             573029    543843     29186  95% [...]

Filesystem            Size  Used Avail Use% Mounted on
/dev/loop11           3.8G  548M  3.1G  15% [...]
/dev/loop19           560M  532M   29M  95% [...]

Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/loop11          1024000    1656 1022344    1% [...]
/dev/loop19          1024000    1656 1022344    1% [...]

I start with a partition of fixed size that possibly wasted a lot of space and I want to create a partition that is able to hold that data but with (almost) minimal size. How can I reliably calculate that minimal partition size needed for storing a certain amount of data? The amount of data changes over time, and I need to automate these calculations.

Daniel Beck

Posted 2011-08-05T15:38:32.240

Reputation: 98 421

I'd appreciate if the person down voting would leave a comment explaining it. – Daniel Beck – 2011-09-04T10:12:51.983

Answers

Here's a procedure you could try:

Create a disk image a bit larger than what you need.
Create the filesystem directly in the image. You may want to set filesystem parameters, as billc.cn suggests.
Loop-mount the image and copy your files. Unmount.
Shrink the image to its minimum size using resize2fs -M. If you prefer, use Gparted as a graphical front-end.

Basically, resize2fs can do the calculation you want dynamically, considering the actual space used by the files.

If you just want to calculate the minimum size, rather than actually resizing, use resize2fs -P instead.

~~Since your data is only a few hundred megabytes, you could create the intermediate disk image in RAM (tmpfs) to save time~~ in any file system with free space (e.g. /tmp), resize, and then dd it to your hard disk.

If your data doesn't change once written, you should consider using SquashFS or a similar filesystem instead, which will be more compact than ext3. Otherwise, consider ext2, which avoids the overhead of journalling.

Mechanical snail

Posted 2011-08-05T15:38:32.240

Reputation: 6 625

This will leave the image file the same size after shrinking the partition on it. And while it's larger than the partition anyway, I need the data after that partition to be left untouched by this process. I mentioned that the amount of data varies. It is in almost all cases multiple GB. I'm already using compressed file systems, but in this case, I need it to be ext2. – Daniel Beck – 2011-08-06T05:17:33.420

@Daniel Beck: ext2 or ext3? – Mechanical snail – 2011-08-06T05:31:37.047

@Daniel Beck: On my system (resize2fs 1.41.12), it does shrink the image. What do you mean by "data after that partition"? – Mechanical snail – 2011-08-06T05:37:55.510

Sorry, ext3, just like I wrote in the original question. ALso, there are multiple partitions on the image, complete with boot sector. That's probably why nothing gets resized for me (and I actually don't want any resizing). I admit it's a particular problem with odd constraints. – Daniel Beck – 2011-08-06T15:12:00.770

Why do you need to put multiple partitions on your temporary image? – Mechanical snail – 2011-08-07T04:52:14.133

It's not temporary. Unfortunately, I'm unable to go into the details behind these requirements. – Daniel Beck – 2011-08-07T10:29:17.270

I meant to make a temporary image, resize, and then dd to wherever you want the final partition. – Mechanical snail – 2011-08-08T05:21:13.263

You can use the -b option of mkfs (or mke2fs) to explicitly set the block size or you can use the -T option to specify the type of files. Check the manual page for details.

You can also need to make sure the number of inodes and journal size is appropriate though.

billc.cn

Posted 2011-08-05T15:38:32.240

Reputation: 6 821

I already use -b. I reserve no sectors for root. I'll have to look into journal size though, I'm not sure right now. – Daniel Beck – 2011-08-05T20:16:25.403