7

I have list of files in a file, cache_temp.

In file cache_temp:

/home/maildir/mydomain.com/een/new/1491397868.M395935P76076.nm1.mydomain.com,S=1740,W=1777
/home/maildir/mydomain.com/een/new/1485873821.M199286P14170.nm1.mydomain.com,S=440734,W=446889
/home/maildir/mydomain.com/td.pr/cur/1491397869.M704928P76257.nm1.mydomain.com,S=1742,W=1779:2,Sb
/home/maildir/mydomain.com/td.pr/cur/1501571359.M552218P73116.nm1.mydomain.com,S=1687,W=1719:2,Sa
/home/maildir/mydomain.com/td.pr/cur/1498562257.M153946P22434.nm1.mydomain.com,S=1684,W=1717:2,Sb

I have a simple script for getting the size of files from cache_temp:

#!/bin/bash

for i in `grep -v ^# ~/cache_temp | grep -v "dovecot.index.cache"`; do
    if [ -f "$i" ]; then
        size=$(du -sh "$i" | awk '{print $1}')
        echo $size
    fi
done

I have a list of sizes of files:

4,0K
4,0K
4,0K
432K
4,0K

How can I calculate the sum of them?

Peter Mortensen
  • 2,319
  • 5
  • 23
  • 24
Piduna
  • 501
  • 3
  • 10
  • 23
  • 6
    Don't use the `-h` switch for basic size calculations, taking `k` `M` or `G`'s into account is going to be horribly complex for a simple shell script. Simply adding numbers is trivial http://tldp.org/LDP/abs/html/arithexp.html – HBruijn Dec 12 '17 at 14:27

5 Answers5

6

Use stat instead of du:

#!/bin/bash

for i in `grep -v ^# ~/cache_temp | grep -v "dovecot.index.cache"`; do
     [ -f "$i" ] && totalsize=$[totalsize + $(stat -c "%s" "$i")]
done
echo totalsize: $totalsize bytes
glenn jackman
  • 4,320
  • 16
  • 19
Ipor Sircer
  • 1,230
  • 7
  • 8
6

If you need to use the file this snippet is hopefully efficient.

xargs -a cache_file stat --format="%s" | paste -sd+ | bc -l

The xargs is to prevent overflowing the argument limit but getting the max number of files into one invocation of stat each time.

Matthew Ife
  • 22,927
  • 2
  • 54
  • 71
6

According to du(1), there is a -c option whose purpose is to produce the grand total.

% du -chs * /etc/passwd
92K ABOUT-NLS
196K    NEWS
12K README
48K THANKS
8,0K    TODO
4,0K    /etc/passwd
360K    total
Laurent
  • 61
  • 1
  • 1
    And to just get the human-readable total size: `du -chs * | tail -1 | cut -f1` – glenn jackman Dec 12 '17 at 17:44
  • Good option, and so close to a full answer... combined with reading files from `cache_temp` and maybe xargs in case of large lines & add them... I guess you'd have shearn89's answer... – Xen2050 Dec 12 '17 at 17:47
4

If you remove the "-h" flag from your "du" command, you'll get the raw byte sizes. You can then add them with the ((a += b)) syntax:

a=0
for i in $(find . -type f -print0 | xargs -0 du -s | awk {'print $1'})
do
  ((a += i))
done
echo $a

The -print0 and -0 flags to find/xargs use null-terminated strings to preserve whitespace.

EDIT: turns out I type slower than @HBruijn comments!

shearn89
  • 3,143
  • 2
  • 14
  • 39
2

Well... For better or worse, here's my implementation of this. I've always preferred using "while" to read lines from files.

#!/bin/bash

SUM=0
while read file; do
    SUM=$(( $SUM + $(stat $file | awk '/Size:/ { print $2 }') ))
done < cache_temp
echo $SUM

Per janos' recommendation below:

#!/bin/bash

while read file; do
    stat $file
done < cache_temp | awk 'BEGIN { s=0 } $1 == "Size:" { s=s+$2 } END  { print s; }'
Erik
  • 160
  • 4
  • 1
    With good reason: http://mywiki.wooledge.org/BashFAQ/001 – glenn jackman Dec 12 '17 at 17:38
  • 1
    @glennjackman Actually the "reason" link is here https://mywiki.wooledge.org/DontReadLinesWithFor but both links are useful – Xen2050 Dec 12 '17 at 17:44
  • I see what you're saying.. no sense running stat AND awk "wc -l cache_temp" times. Use awk one to roll everything up at the end – Erik Dec 18 '17 at 13:56