Poor write performance on ecryptfs

Question

I did a bit of benchmarking with ecryptfs and dm-crypt, and got some interesting results. All the following were done with a Btrfs filesystem, using dd to copy a ~700MB file to/from a ramdisk with the conv=fdatasync option to force data syncing. Disk caches were cleared before each test.

No encryption:
 read - 165MB/s
 write - 120MB/s
ecryptfs:
 read - 125MB/s
 write - 15MB/s
dm-crypt:
 read - 150MB/s
 write - 115MB/s
dm-crypt + ecryptfs:
 read - 120MB/s
 write - 15MB/s

Now I understand that encryption is slower than a raw filesystem, however I didn't expect the massive write performance drop with ecryptfs. Does the fact that I am forcing data syncing make this test unrealistic? Or are there any options I can pass to ecryptfs to get writes working faster?

I was using filename encryption on ecryptfs, but other than that everything was set to defaults.

Answer 1

Man page of dd about fdatasync reads: physically write output file data before finishing, so it only writes data physically "once" (read it as "not forcing a flush every X blocks or bytes, but a single flush at the end"). If you are using dd to do your tests, it is the best way to get the most accurate results. On the contrary, not using that specific flag, would render your results unrealistic: omiting it would probably miss the time for the encryption itself since dd is just copying data around.

Nevertheless, I also thought there were something going on as for your results, but I found this article that shows almost the same: ecryptfs is painfully slow. And your test (a single file being copied) is the best-case scenario for ecryptfs!

As ecryptfs writes an encrypted file (with a custom header with metadata inside) for every clear-text version, having lots of little files implies even a greater performance drop.

However, ecryptfs has its benefits: you can send an encrypted file right away without loosing the encryption. Your backups (assuming you are backing up your encrypted data) would be faster since you would only copy files as big as your data (and even faster if they are incremental, as you would only copy modified files).

dm-crypt, on the other hand, may be way faster but you would need to send the whole container (a whole filesystem) to keep the encryption as it is. And the backups would also consist of the whole container, not being able to do incremental backups in most cases.

I have used (and still use) both methods (not the same tools, though) to hold encrypted data: file-based (ecryptfs) is easier to keep synchronized via online hosting services such as dropbox between PCs, but it is pretty slow when doing changes and has caused me some issues with the underlaying filesystem (it assumes it can write the files and issues related to limits on the filesystem tend to break the whole thing up); I prefer the block device encryption: I treat them as simple partitions, so the limits and issues do not break so badly. The only drawback is copying the container, which can take way longer.