Use all processor cores, 1 core/per 1 thread in *nix

According to xz documentation the "-T 4" means I should be using 4 threads, each compressing a different part of the file simultaneously.

Before using nice -19 I would only get %50 usage of 1 core, Now how do I use the other 3 cores?(these aren't virtual cores either, they're real)

nice -19 xz -v9ekC sha256 -T 4 /path/to/file.tar

only 1 processor over %50 at time!!

GlassGhost

Posted 2013-09-28T21:37:01.977

Reputation: 919

Which version of xz are you using? – jjlin – 2013-09-28T22:20:01.677

1How fast is your IO? If you can not read fast enough from disk (or disk cache) then throwing more cores at it will not help. Are you testing with a huge file ? – Hennes – 2013-09-28T22:21:39.747

@jjlin 5.1.0alpha – GlassGhost – 2013-09-30T20:41:18.910

@Hennes xz compression is very much CPU-bound, unless you have a pretty unusual hardware configuration. – jjlin – 2013-09-30T21:16:45.440

Answers

Multithreaded coding wasn't implemented until 5.1.1alpha:

http://git.tukaani.org/?p=xz.git;a=blob;f=NEWS;hb=HEAD

This is easy to verify if you compare src/xz/coder.c for the 5.1.0alpha and 5.1.1alpha (just search for "thread").

jjlin

Posted 2013-09-28T21:37:01.977

Reputation: 12 964

Before I mark this as accepted, I'd like to test it 1st. I guess I'll have to wait til 5.1.1alpha or higher is released for ubuntu; I've been looking for an excuse to install funtoo linux. – GlassGhost – 2013-10-25T17:38:12.790

It's actually pretty straightforward to just download the source and compile it yourself. I don't recall xz having any uncommon external dependencies. – jjlin – 2013-10-25T17:45:07.513