Get the total size of all files of a given extension

1

I'm trying to get the total size of all the JPEGs in a particular folder, but no amount of combinations of options while piping to du seems to make this work:

find -type f -iname "*.jpg" | du -sch
166G    .
166G    total

All of them print the total of every file in the directory.

How do I get only the total file size of the files that I'm passing to du?

Hashim

Posted 2020-01-15T21:10:21.270

Reputation: 6 967

Answers

1

If you're only in one directory as it appears from your question, the simplest way would be to avoid piping altogether and instead do

du -sch *.jpg

Christian Seitz

Posted 2020-01-15T21:10:21.270

Reputation: 28

1Wow, I didn't even know this was possible, and didn't see it referenced anywhere else while looking. Thank you. – Hashim – 2020-01-16T19:06:59.950

1

If you call du with the files as arguments you are limited by the length of the argument string (a few thousands files)(*). If you expect a large set of files, you have to provide the files in a list file using the --files0-from=F argument (you can use - for standard input):

find . -name '*.JPG' -print0 | du -msc --files0-from -

(*) And using ... | xargs du -c won't help because xargs will chop the file list in batches and call du several times so you will have several sums...

xenoid

Posted 2020-01-15T21:10:21.270

Reputation: 7 552

0

I'm unsure why simply piping to du doesn't work here (feel free to post answers elaborating on why), but I figured out that using -exec to execute du does work, and is also very fast with the + syntax.

find -type f -iname "*.mov" -exec du -ch {} +

This makes it much more efficient than piping would ever have been, especially since I was already using find to enumerate the files. It will also match the JPEG file extension no matter what case it's in.

Note that the above will print each file with its individual size before eventually printing the total itself. To print only the total file size for all the files, use:

find -type f -iname "*.mov" -exec du -ch {} + | tail -n1

Hashim

Posted 2020-01-15T21:10:21.270

Reputation: 6 967

Note: If there are many, many matching files, then -exec du -ch {} + will run more than one du. And then tail -n1 will show you the result from the last one only and you may never know it's not what you wanted. – Kamil Maciorowski – 2020-01-16T19:12:20.330

So are you saying it would be safer to use \;? – Hashim – 2020-01-16T19:32:29.020

Not at all. Then you will run many, many dus and there will be no way to get a total, other than to add up what they all say. This is possible but note a single du is able to count hardlinked files only once. If each file is processed by its own du then (after adding up) you will get an inflated result in comparison to what a single du would say.

– Kamil Maciorowski – 2020-01-16T19:44:07.497