Why can't I redirect output from awk?

Question

I've written this one liner to show MB read/written every 5 seconds.

#!/bin/bash
#iosum.sh
iostat -dmz 5 |\
    awk 'BEGIN{rx=wx=0}{if($0 == ""){printf"%.1f %.1f\n",rx,wx}else if($0~/^Device:/){rx=wx=0}else{rx+=$3;wx+=$4}}'

When run from the command line it works perfectly

./iosum.sh
# wait
1.1 0.0
0.0 0.1
0.0 0.0
3.0 3.0
0.0 0.0
# ctrl-c

However, when I try to put the output into a file I get nothing

./iosum.sh > out.txt
# wait
# ctrl-c
cat out.txt
# nothing!

What gives?

Answer 1

This is a buffering issue. Awk behaves differently depending on whether it is interactive or not. \n flushes the output buffer in the latter case only, otherwise, as you experienced, buffering happens until the output buffer is full or is explicitly flushed with the fflush command.

Eventually, there will be something in the output file if you wait for long enough time.

Otherwise, output should be flushed if you kill iostat, not the script itself.

Alternatively, here is a simple workaround:

script -c iosum.sh out.txt

Note that disabling buffering (with script or an explicit fflush) is affecting performance so killing the iostat process (pkill -f "iostat -dmz 5") or setting it to terminate after a given period of time is a better approach if you don't need to monitor the file content, e.g. with a two minute capture period:

iostat -dmz 5 120 |\
    awk 'BEGIN{rx=wx=0}{if($0 == ""){printf"%.1f %.1f\n",rx,wx}else if($0~/^Device:/){rx=wx=0}else{rx+=$3;wx+=$4}}'

Answer 2

Awk buffers all output and only writes it out when it finishes reading input or when the buffer is full. Use fflush(stdout) (see awk man page) in your awk script to make sure the output is properly flushed to disk after every line printed. Like this:

iostat -dmz 5 |\
       awk 'BEGIN{rx=wx=0}{if($0 == ""){printf"%.1f %.1f\n",rx,wx;fflush(stdout)}else if($0~/^Device:/){rx=wx=0}else{rx+=$3;wx+=$4}}' > x