4

It is known that certain AMD64 Linuxes are subject of being unresponsive under heavy disk I/O (see Gentoo forums: AMD64 system slow/unresponsive during disk access (Part 2)), unfortunately have such one.

I want to put /var/tmp/portage and /usr/portage trees to a separate partition, but what FS to choose for it?

Requirements:

* for journaling, performance is preffered over safe data read/write operations
* optimized to read/write 10000 of small files

Candidates:

* ext2 without any journaling
* BtrFS

In Phoronix tests, BtrFS had demonstrated a good random access performance (fat better than XFS thereby it may be less CPU-aggressive). However, unpacking operation seems to be faster with XFS there, but it was tested that unpacking kernel tree to XFS makes my system to react slower for 51% disregard of any renice'd processes and/or schedulers.

Why no ReiserFS? Google'd this (q: reiserfs ext2 cpu): 1 Apr 2006 ... Surprisingly, the ReiserFS and the XFS used significantly more CPU to remove file tree (86% and 65%) when other FS used about 15% (Ext3 and ... Is it same now?

kagali-san
  • 1,921
  • 5
  • 17
  • 20
  • 1
    why no raiserfs? btrfs is still not stable... – Hubert Kario Feb 12 '11 at 17:05
  • @Hubert Kario, will try to explain in post, wait 2m pls. – kagali-san Feb 12 '11 at 17:09
  • Does that case (deletions) happen often? It's not in your requirements. If not, give resierfs a try. It's all well and good to look for guidance on the web (in the docs and mailing lists, and here too), but none of us are running your system. It sounds like you have a specific application profile, so our advice might not fit your environment best. You might just need to test. What is your application(s), and why does splitting the portage directories have anything to do with it? – mfinni Feb 12 '11 at 17:36
  • 1
    Are you aware that btrfs doesnt even have a fsck hook? IMHO, it's not even worth looking at yet if you plan to keep your data. From my own experience XFS pretty much outperforms all others in real world usage (databases, heavy file servers). Do you really need to delete files that often (and in so high numbers)? – pauska Feb 12 '11 at 17:58
  • @mfinni, @pauska, Gentoo really can delete 100K files under a hour while updating something like KDE4/kernel, and nowadays got back on a limited connection, so using emerge-webrsync which actually reinstalls whole /usr/portage kills me with a large XFS stone. – kagali-san Feb 12 '11 at 18:25
  • also, it's fine for brtfs being unstable (hope it just would not transform to grey goo and get out of its partition eating all my data), one can reinstall both /var/tmp/portage and /usr/portage within 10 minutes (mkdir -p and emerge-webrsync) – kagali-san Feb 12 '11 at 18:28
  • Raiser may use more CPU to remove files, thing is, it can do it *faster* than other filesystems. – Hubert Kario Feb 12 '11 at 22:57

3 Answers3

2

Ext4, Btrfs, Reiser4, Reiser3 (with notail option). XFS sucks on metadata intensive operations (like rm'ing lots of files).

poige
  • 9,171
  • 2
  • 24
  • 50
1

You don't really need journalling for those directories (since they can easily be restored), so no matter which filesystem you choose, you can disable journalling.

Back when i used Gentoo, i used ReiserFS 3 for /usr/portage. The reason is simple: it was pretty fast, and it saved a lot of space compared to other filesystems when you are dealing with a huge number of very small files. (using the "notail" option removes this benefit of course) I did notice that over time performance on that filesystem would decrease. This is due to the fact that there are very frequent writes/deletes on /usr/portage and no matter what filesystem you use, after a while fragmentation is gonna make it slower. So sometimes i just took a targz from /usr/portage, reformatted the partition, and unpacked the targz on it again... this gives you a new /usr/portage with no fragmentation.

I don't know if i would still use ReiserFS now.. i would probably go for ext4 (with journalling disabled) or ext2.

About /var/tmp/portage... if you have enough ram, you might want to map that on tmpfs. That way, it will use ram as much as possible, but go to swap when the data does not fit in ram.

rvdginste
  • 376
  • 2
  • 8
  • "using the `notail` option removes this benefit of course" But `notail` gives benefit in throughput. Who cares ± 100 MiB disk space now on desktop systems? And better throughput is always welcome. – poige Feb 13 '11 at 07:05
  • Does notail give a benefit in throughput? That depends on where the bottlenecks are in the system: is it I/O bound, or CPU bound? When updating the portage tree, is it cheaper to have 500MB disk IO, or is it cheaper to have 700MB disk IO with notail option. Is the system gonna be cpu starved, or io starved? In my case, specifically for the /usr/portage tree (huge number of tiny files), it was io starved, and leaving out the notail option performed better. (500MB and 700MB are just number, don't remember the exact difference, but the difference was bigger than i expected.) – rvdginste Feb 13 '11 at 08:19
  • "unpacked the targz on it again" - there's `emerge-webrsync` – kagali-san Feb 16 '11 at 01:50
  • so, for a CPU-bound system (rare -- if ever -- disk write), I should add the `notail` option? – pepoluan Mar 11 '11 at 03:18
  • @pepoluan yes, for a cpu-bound system, you should use the notail option – rvdginste Mar 13 '11 at 00:29
0

Ext2 is rather slow compared with the other OS, It may also have problems with very large numbers of files. I'd strongly recommend NOT using it - ext4 with journalling off is a different kettle of fish altogether.

Personally, I'd lean towards Reiserfs for a filesystem with lots of small files - it's specifically optimized for this kind of operation. But I mainly work with webservers - where the access is predominantly read/only (I use ext4 for db). The rate at which it uses CPU is not such a big deal - indeed, higher CPU would be a pretty good indicator that less actual file I/O is required to complete an operation!

Looking up the original article - the comment you quoted specifically relates to the behaviour for large files - I'm no expert on Gentoo, but I believe this is irrelevant to portage.

symcbean
  • 19,931
  • 1
  • 29
  • 49
  • ....and you probably want to mount the filesystem with noatime - which will change its behaviour greatly. – symcbean Feb 13 '11 at 13:22