3
1
I tested the speed of md5sum on a few Ubuntu 8.04 servers
- Pentium III 700 MHz: 52 MB/s
- Atom 1.6 GHz, 32-bit: 119 MB/s
- Core 2 (Yorkfield) 2.5GHz, 32-bit: 194 MB/s
- Core 2 (Yorkfield) 2.5GHz, 64-bit: 222 MB/s
Then I downloaded a tool (by apt-get install) called md5deep and found that it's roughly 20% faster (as tested on the 32-bit Core 2 server). This makes me feel that the "vanilla" md5sum included in Ubuntu isn't the fastest one.
Questions:
- Other than
md5deep, are you aware of any MD5 calculators that are potentially faster thanmd5sum? (Answers for software from other OS are also welcomed.) - If I want to compile
md5summyself, what compiler options would you suggest for the Core 2 server? (note:gcc4.2.4 in Ubuntu 8.04 does not seem to support-march=core2)
Edit: I downloaded and compiled coreutils-8.4 on the 64-bit Core 2 server and the speed increased to 320 MB/s... Unbelievable!
FYI, the one included in Ubuntu 8.04 is coreutils-6.10.
Is it safe to replace coreutils with a newer version?
netvope
Posted 2010-04-01T22:19:31.737
Reputation: 4 505
Usually md5sum is faster than my disk...eg I don't have a HDD that can read at 200MB/sec, so a faster than that md5sum doesn't help any. Do you have a really fast SSD RAID array? Or are you running md5sum on files stored on a ramdisk (tmpfs) or something? – davr – 2010-04-01T22:26:35.340
At first I used a 1GB file of random bytes, loaded completely in the memory cache. Then I found that the
md5sumspeed is the same if I pipe null bytes intomd5sumon the fly. I decided to use the latter method so that servers without 1GB RAM can also be tested. – netvope – 2010-04-01T22:35:04.883I'd be curious to know how OpenSSL's md5 implementation compares to the others you've benchmarked. – Spiff – 2010-04-02T04:33:07.270