Ext4

Related articles

From Ext4 - Linux Kernel Newbies:

Ext4 is the evolution of the most used Linux filesystem, Ext3. In many ways, Ext4 is a deeper improvement over Ext3 than Ext3 was over Ext2. Ext3 was mostly about adding journaling to Ext2, but Ext4 modifies important data structures of the filesystem such as the ones destined to store the file data. The result is a filesystem with an improved design, better performance, reliability, and features.

Create a new ext4 filesystem

Install e2fsprogs.

To format a partition do:

# mkfs.ext4 /dev/partition
Tip:
  • See mke2fs(8) for more options; edit /etc/mke2fs.conf to view/configure default options.
  • If supported, you may want to enable metadata checksums.

Bytes-per-inode ratio

From mke2fs(8):

mke2fs creates an inode for every bytes-per-inode bytes of space on the disk. The larger the bytes-per-inode ratio, the fewer inodes will be created.

Creating a new file, directory, symlink etc. requires at least one free inode. If the inode count is too low, no file can be created on the filesystem even though there is still space left on it.

Because it is not possible to change either the bytes-per-inode ratio or the inode count after the filesystem is created, mkfs.ext4 uses by default a rather low ratio of one inode every 16384 bytes (16 KiB) to avoid this situation.

However, for partitions with size in the hundreds or thousands of GB and average file size in the megabyte range, this usually results in a much too large inode number because the number of files created never reaches the number of inodes.

This results in a waste of disk space, because all those unused inodes each take up 256 bytes on the filesystem (this is also set in /etc/mke2fs.conf but should not be changed). 256 * several millions = quite a few gigabytes wasted in unused inodes.

This situation can be evaluated by comparing the Use% and IUse% figures provided by and :

To specify a different bytes-per-inode ratio, you can use the -T usage-type option which hints at the expected usage of the filesystem using types defined in /etc/mke2fs.conf. Among those types are the bigger and largefile4 which offer more relevant ratios of one inode every 1 MiB and 4 MiB respectively. It can be used as such:

# mkfs.ext4 -T largefile /dev/device

The bytes-per-inode ratio can also be set directly via the option: e.g. use for a 2 MiB ratio and for a 6 MiB ratio.

Reserved blocks

By default, 5% of the filesystem blocks will be reserved for the super-user, to avoid fragmentation and "allow root-owned daemons to continue to function correctly after non-privileged processes are prevented from writing to the filesystem" (from mke2fs(8)).

For modern high-capacity disks, this is higher than necessary if the partition is used as a long-term archive or not crucial to system operations (like ). See this email for the opinion of ext4 developer Ted Ts'o on reserved blocks and this superuser answer for general background on this topic.

It is generally safe to reduce the percentage of reserved blocks to free up disk space when the partition is either:

  • Very large (for example > 50G)
  • Used as long-term archive, i.e., where files will not be deleted and created very often

The option of ext4-related utilities allows to specify the percentage of reserved blocks.

To totally prevent reserving blocks upon filesystem creation, use:

# mkfs.ext4 -m 0 /dev/device

To change it to 1% afterwards, use:

# tune2fs -m 1 /dev/device

To set the number of reserved block space to an absolute size in gigabytes, use :

# tune2fs -r $((ngigs * 1024**3 / blocksize)) /dev/device

blocksize is the block size of the filesystem in bytes. This is almost always 4096, but you can check to be sure:

The syntax is for math expansion. This syntax works in and , but it will not work in fish. For fish, this is the syntax:

# tune2fs -r (math 'ngigs * 1024^3 / blocksize') /dev/device

These commands can be applied to currently-mounted filesystems, the changes taking effect immediately. You can use to find the device name:

# tune2fs -m 1 "$(findmnt -no SOURCE /the/mount/point)"

To query the current number of reserved blocks:

This is the number of blocks, so this has to be multiplied by the filesystem's block size to get the number of bytes or gigabytes: .

Migrating from ext2/ext3 to ext4

Rationale

A compromise between fully converting to ext4 and simply remaining with ext2/ext3 is to mount the partitions as ext4.

Pros:

  • Compatibility (the filesystem can continue to be mounted as ext3) – This allows users to still read the filesystem from other operating systems without ext4 support (e.g. Windows with ext2/ext3 drivers)
  • Improved performance (though not as much as a fully-converted ext4 partition).

Cons:

  • Fewer features of ext4 are used (only those that do not change the disk format such as multiblock allocation and delayed allocation)

Procedure
  1. Edit and change the 'type' from ext2/ext3 to ext4 for any partitions you would like to mount as ext4.
  2. Re-mount the affected partitions.

Rationale

To experience the benefits of ext4, an irreversible conversion process must be completed.

Pros:

  • Improved performance and new features.

Cons:

  • Partitions that contain mostly static files, such as a /boot partition, may not benefit from the new features. Also, adding a journal (which is implied by moving a ext2 partition to ext3/4) always incurs performance overhead.
  • Irreversible (ext4 partitions cannot be 'downgraded' to ext2/ext3. It is, however, backwards compatible until extent and other unique options are enabled)

Procedure

These instructions were adapted from Kernel documentation and an BBS thread.

In the following steps denotes the path to the partition to be converted, such as .

  1. Back up all data on any ext3 partitions that are to be converted to ext4. A useful package, especially for root partitions, is .
  2. Edit and change the 'type' from ext3 to ext4 for any partitions that are to be converted to ext4.
  3. Boot the live medium (if necessary). The conversion process with e2fsprogs must be done when the drive is not mounted. If converting a root partition, the simplest way to achieve this is to boot from some other live medium.
  4. Ensure the partition is not mounted
  5. If you want to convert a ext2 partition, the first conversion step is to add a journal by running as root; making it a ext3 partition.
  6. Run as root. This command converts the ext3 filesystem to ext4 (irreversibly).
  7. Run as root.
    • This step is necessary, otherwise the filesystem will be unreadable. This fsck run is needed to return the filesystem to a consistent state. It will find checksum errors in the group descriptors - this is expected. The -f option asks fsck to force checking even if the file system seems clean. The option may be used on top to "automatically repair" (otherwise, the user will be asked for input for each error).
  8. Recommended: mount the partition and run as root.
    • Even though the filesystem is now converted to ext4, all files that have been written before the conversion do not yet take advantage of the extent option of ext4, which will improve large file performance and reduce fragmentation and filesystem check time. In order to fully take advantage of ext4, all files would have to be rewritten on disk. Use e4defrag to take care of this problem.
  9. Reboot

Improving performance

E4rat

E4rat is a preload application designed for the ext4 filesystem. It monitors files opened during boot, optimizes their placement on the partition to improve access time, and preloads them at the very beginning of the boot process. E4rat does not offer improvements with SSDs, whose access time is negligible compared to hard disks.

Disabling access time update

The ext4 file system records information about when a file was last accessed and there is a cost associated with recording it. With the noatime option, the access timestamps on the filesystem are not updated.

Doing so breaks applications that rely on access time, see fstab#atime options for possible solutions.

Increasing commit interval

The sync interval for data and metadata can be increased by providing a higher time delay to the option.

The default 5 sec means that if the power is lost, one will lose as much as the latest 5 seconds of work. It forces a full sync of all data/journal to physical media every 5 seconds. The filesystem will not be damaged though, thanks to the journaling. The following fstab illustrates the use of :

Turning barriers off

Ext4 enables write barriers by default. It ensures that file system metadata is correctly written and ordered on disk, even when write caches lose power. This goes with a performance cost especially for applications that use fsync heavily or create and delete many small files. For disks that have a write cache that is battery-backed in one way or another, disabling barriers may safely improve performance.

To turn barriers off, add the option to the desired filesystem. For example:

Disabling journaling

Disabling the journal with ext4 can be done with the following command on an unmounted disk:

# tune2fs -O "^has_journal" /dev/sdXN

Use external journal to optimize performance

For those with concerns about both data integrity and performance, the journaling can be significantly sped up with the mount option. Note that it does not work with the balanced default of data=ordered, so this is only recommended when the filesystem is already cautiously using .

You can then format a dedicated device to journal to with . Use to assign the journal to an existing device, or replace with mkfs.ext4 if you are making a new filesystem.

Tips and tricks

Using file-based encryption

Since Linux 4.1, ext4 natively supports file encryption, see the fscrypt article. Encryption is applied at the directory level, and different directories can use different encryption keys. This is different from both dm-crypt, which is block-device level encryption, and from eCryptfs, which is a stacked cryptographic filesystem.

Enabling metadata checksums in existing filesystems

When a filesystem has been created with e2fsprogs 1.43 (2016) or later, metadata checksums are enabled by default. Existing filesystems may be converted to enable metadata checksum support.

If the CPU supports SSE 4.2, make sure the kernel module is loaded in order to enable the hardware accelerated CRC32C algorithm . If not, load the module instead.

To read more about metadata checksums, see the ext4 wiki.

Tip: Use dumpe2fs to check the features that are enabled on the filesystem: 
# dumpe2fs -h /dev/path/to/disk

First the partition needs to be checked and optimized using :

# e2fsck -Df /dev/path/to/disk

Convert the filesystem to 64-bit:

# resize2fs -b /dev/path/to/disk

Finally enable checksums support:

# tune2fs -O metadata_csum /dev/path/to/disk

To verify:

Enabling fast_commit in existing filesystems

Starting from the 5.10 kernel, ext4 may have a performance boost by enabling the fast_commit option: 

# tune2fs -O fast_commit /dev/drivepartition

To clarify the current configuration or changes: 

# tune2fs -l /dev/drivepartition | grep features
gollark: Just keep brightness low maybe?
gollark: Some other teleporting equipment does, and so does the staff of levitation.
gollark: I don't remember it running on that in 1.12.2.
gollark: With enough power in portable batteries, it also allows flight.
gollark: You can loot one from desert temples quite early, and conveniently ignore stuff like walls forever.

See also
This article is issued from Archlinux. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.