33

In Windows, it's very fast to format with NTFS. I have a low powered Linux machine, with little RAM. Formatting a 2TB volume to ext4 takes a long long time.

Is there anything I can do to speed up the format? I can't imagine what takes so long? (what does take so long)

Abdollah
  • 103
  • 4
stuck
  • 667
  • 2
  • 10
  • 23
  • 1
    What commands are you using to format it? – tacotuesday Aug 01 '12 at 04:53
  • As a workaround. try making it as a smaller lvm volume, and grow it as needed ? That'd lower the initial overhead anyhow. – Sirex Aug 01 '12 at 05:21
  • 1
    or use XFS if filesystem creation time really bothers you - mkfs.xfs is a lot faster than mkfs.ext4 (because it doesn't do - or need to do - as much). but mkfs is something you usually only have to do once per filesystem - there are probably better things to optimise. – cas Aug 01 '12 at 10:14
  • or use ZFS. Creating file systems with ZFS is is near to instantaneous, whatever their size. – jlliagre Sep 18 '12 at 22:13
  • Only use ZFS if you have enterprise class hardware, in particular if you have ECC RAM and you have UPS. It's *not recommended* to use ZFS without these requirements. You may be not able to recover after a memory parity error and/or a power failure. All your volume will be lost in this case. You have been warned. – Richard Gomes Oct 23 '17 at 10:48

4 Answers4

21

Add the flag -E lazy_itable_init

Here's what the man page says:

If enabled and the uninit_bg feature is enabled, the inode table will not be fully initialized by mke2fs. This speeds up filesystem initialization noticeably, but it requires the kernel to finish initializing the filesystem in the background when the filesystem is first mounted. If the option value is omitted, it defaults to 1 to enable lazy inode table initialization.

Scott Pack
  • 14,717
  • 10
  • 51
  • 83
Alex Wheeler
  • 211
  • 1
  • 2
  • 3
    Technically this doesn't actually make the format go faster, but it lets you mount and use the filesystem while the format finishes in the background. – Wyzard Sep 18 '12 at 22:37
  • thanks so much! this wo4ked really well, the docs/man weren't clear to me until knew what to look for! thx so much! – stuck Dec 25 '12 at 07:20
  • I've seen references about minimum kernel version, which is the minimum necessary to use this? – Xarses Aug 13 '13 at 18:05
18

Strict answer

Solutions like -E lazy_itable_init don't change the result, only speed up the process. This is what was explicitly asked yet in many cases people need more.

Extra bonus

In most case you actually want some options that match your usage patterns and not only speed up filesystem creation but also allow faster usage and more usable space.

I just did a test. Even without using -E lazy_itable_init, the options below speed up creation time of a 2TB filesystem from 16 minutes 2 second to 1 minute 21 second (kernel 3.5.0 64bit on Intel i7 2.2GHz, 2TB disk on USB2 connection -- SATA would probably be faster).

For a filesystem that will hold large files, I use this combination:

mkfs.ext3 /dev/sdXX -O sparse_super,large_file -m 0 -T largefile4

where -T largefile4 picks options in /etc/mke2fs.conf which generally contain something like:

    inode_ratio = 4194304
    blocksize = -1

Do a man mke2fs for details on each of these options.

Here are relevant extracts:

               sparse_super
                      Create a filesystem with fewer superblock backup copies (saves space on large filesystems).

               large_file
                      Filesystem can contain files that are greater than 2GB.  (Modern kernels set this feature  automatically
                      when a file > 2GB is created.)

   -i bytes-per-inode
          Specify  the  bytes/inode ratio.  mke2fs creates an inode for every bytes-per-inode bytes of space on the disk.  The
          larger the bytes-per-inode ratio, the fewer inodes will be created.  This value generally shouldn't be smaller  than
          the blocksize of the filesystem, since in that case more inodes would be made than can ever be used.  Be warned that
          it is not possible to expand the number of inodes on a filesystem after it is created, so be  careful  deciding  the
          correct value for this parameter.

-m 0 only says not to reserve 5% for root, which is okay for a data (not boot/root) filesystem. 5% of a 2TB disk means 100Gb. That's a pretty significant difference.

Stéphane Gourichon
  • 714
  • 6
  • 13
  • 1
    Note that allowing filesystem to get more than 95% full will lead to extreme fragmentation, slowing down performance substantially. If anything, if one cares about performance of his data partition, it should be increased to 10% – Matija Nalis Aug 08 '13 at 13:41
  • If your disk will hold **not only** large files but **small files also**, read this: reducing inode ratio will save you at most about 1.5% of storage, at the cost of versatility. `inode_ratio = x` means that if the **average** file size on the volume is less than x bytes, then you're headed towards *running out of inodes* before *running out of storage space*. In that scenario you may waste literally thousands more than you expected to save. – Stéphane Gourichon Nov 04 '16 at 09:13
  • In other words, if the **average** file size is smaller than 4MB, don't use "-T largefile4". For huge drives I now end up using `-T huge` which does `inode_ratio = 65536`. If in doubt, just don't set `-i` or `-T` and let the wise people who made ext2/3/4 choose for you. You'll reserve a known small part of space for housekeeping and (unless you store millions of very small files) you are guaranteed to not waste an arbitrary percentage of actual storage space. As people write, fancy software is nice, but a good filesystem must be boring and just work. – Stéphane Gourichon Nov 04 '16 at 09:16
  • on my case, `mkfs.ext4` is more faster (time: `real 0m16.001s`) than `mkfs.ext3` (time: `real 2m10.336s`). – coanor Apr 19 '17 at 01:44
8

The default is a quick format; setting up the structures for a ext* volume takes much longer than for a NTFS volume, since there's more of them. You can reduce the number of superblocks, but even that only goes so far.

Ignacio Vazquez-Abrams
  • 45,019
  • 5
  • 78
  • 84
  • thanks for the info, my device started formatting a 2TB USB2 disk about 20 hours ago, it's still going - the device is low powered, think Raspberry Pi with 64MB of RAM. in order to start the format I had to setup paging to the same USB disk being formatted is there anything I can do to speed this up? i'd be great if I could get the performance to be under an hour or two? – stuck Aug 01 '12 at 22:30
  • 2
    lazy_itable_init did the trick for me, this isnt the default and does make formatting much quicker – stuck Jan 07 '13 at 01:29
4

If you'll be storing mostly larger files you can increase the number of bytes per inode, thus decreasing the number of inodes created. This can substantially speed up creation time.

rnxrx
  • 8,103
  • 3
  • 20
  • 30