20

We use RAID1+0 with md on Linux (currently 2.6.37) to create an md device, then use LVM to provide volume management on top of the device, and then use ext4 as our filesystem on the LVM volume groups.

With SSDs as the drives, we'd like to see the TRIM commands propagate through the layers (ext4 -> LVM -> md -> SSD) to the devices.

It looks like recent 2.6.3x kernels have had a lot of new SSD-related TRIM support added, including lots more coverage of Device Mapper scenarios, but we still can't seem to get it to cascade down properly.

Is this possible yet? If so, how? If not, is any progress being made?

Don MacAskill
  • 1,808
  • 3
  • 16
  • 22
  • See updated http://serverfault.com/a/229486/67675 :) – poige Apr 07 '12 at 03:30
  • and http://serverfault.com/questions/227918/possible-to-get-ssd-trim-discard-working-on-ext4-lvm-software-raid-in-linu/229486#comment426326_229486 – poige Jul 11 '12 at 01:13

6 Answers6

14

As of 2.6.37, it should be present (source). The kernel doesn't do it in the background, the block discard process is currently designed to be run on demand (cron script!). Dm-crypt support doesn't exist yet.

On January 13th, 2011 a patch was merged into dm-raid1.c that reads: 

dm raid1: support discard

Enable discard support in the DM mirror target.
Also change an existing use of 'bvec' to 'addr' in the union.

I'm not 100%, but I think that's the merge-window for 2.6.38.

EXT4 added support a while ago, as did LVM. RAID is the one key that doesn't have support. As of 1/13/2011, it appears support has been added. Look for it in 2.6.38 or maybe 2.6.39.

Time has passed and TRIM support is definitely included in the 3.7 kernel. The commit for RAID10 reads:

This makes md raid 10 support TRIM. If one disk supports discard and another not, or one has discard_zero_data and another not, there could be inconsistent between data from such disks. But this should not matter, discarded data is useless. This will add extra copy in rebuild though.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
  • Saw that thread, and the related commits, but like I said in the question - does this mean it'll properly pass through not only LVM (Device Mapper) but also md (Software RAID)? – Don MacAskill Jan 30 '11 at 07:01
  • 1
    This sounds promising, but my understanding is that dmraid is primarily used for so-called 'fakeRAID' hardware RAID controllers. What I'm talking about is a more typical software-only mdadm RAID-1 (+0) array(s) with LVM on top. AFAIK, that setup doesn't benefit from dmraid's new-found ability to handle discards. Right? – Don MacAskill Feb 01 '11 at 21:59
  • 1
    @DonMacAskill The 'raid1.c' file doesn't have any commits referencing TRIM, FITRIM, or 'dispose'. So, looks like mdadm --create RAID support isn't there yet after all. – sysadmin1138 Feb 01 '11 at 22:25
  • @sysadmin1138 I think you mean `discard`, which is what it's called within the Linux kernel. And that patch does seem to refer to discard. – Michael Hampton Aug 30 '12 at 20:10
  • "dm-raid1.c that reads" — and dm's RAID isn't about having LVM-2 built on top of Linux Software RAID as originally given in the subject. I mean it's totally unrelated to the q-n altogether. – poige Jun 17 '20 at 03:58
8

UPD. 2020-06-17

Looking back through commits history from 2020:

2 years later there're commits in regards for md (LSR), the one among them:

— Basically in a few months since I edited my answer previously, Linux kernel became able to support block discards in the setup.

Previous versions of the answer:

UPD. 2012-07-17 UPD.: Thanks to Wodin for letting me know — according to lkml.org/lkml/2012/3/11/261 this functional has been addedproposed recently. proposed != accepted, though.

UPD. 2011-02-01 Not possible, cause Linux Soft RAID doesn't support this (yet?).

poige
  • 9,171
  • 2
  • 24
  • 50
1

As a summary, here are current recommended settings for performance and data safety on Linux.

Because of some bugs/risks, Debian does not enable LVM discard by default: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=717313

Gentoo Wiki proposes a one-time sequence to discard blocks in VG free space (as root):

lvcreate -l100%FREE -n trim yourvg
blkdiscard /dev/your/trim
lvremove yourvg/trim

About filesystems, it is recommended not to enable discard mount option in /etc/fstab but to only enable fstrim.timer generally provided in systemd-based distributions.

sudo systemctl enable fstrim.timer

Underlying layers like software raid should handle/forward discard/trim events from FS or LVM.

Yves Martin
  • 879
  • 3
  • 7
  • 21
1

Mdtrim may need more working:

Cyberax-mdtrim-0a40e8d# ./mdtrim.py -m /dev/md4 -s /home
Scratch directory is /home, trimmer file size is 0 GB 790 MB
Found slave sdc2 on /dev/sdc with MD offset 0 and partition offset 249856
Creating trimmer file
252,2: device not found in /dev
Traceback (most recent call last):
  File "./mdtrim.py", line 120, in <module>
    if lines[2].find("assuming %d byte sectors" % sector_size) == -1:
IndexError: list index out of range
Kenny Rasschaert
  • 8,925
  • 3
  • 41
  • 58
ERos
  • 11
  • 1
0

You can use my MDTRIM script ( https://github.com/Cyberax/mdtrim/ ) to TRIM empty space on ext4/3 level-1 RAIDs. We start it periodically from cron and it works great for us.

Adding support for other RAID levels is possible, but I don't have time (or need) for that.

Cyberax
  • 249
  • 1
  • 5
0

As suggested here You can use

lsblk -D

in order to check if your blockdevices pass through the discard commands.

Also note that the section devices in lvm.conf contains an option issue_discards. See

man 5 lvm.conf

for more info.

sebastianwagner
  • 306
  • 2
  • 2