Multiple standard input? How?

Question

From this commandlinefu post:

$ diff <(wget -q -O - URL1) <(wget -q -O - URL2)

Seems like multiple standard inputs but---if I remember my Linux Porgramming course correctly---this can't be it. I thought that, by definition, standard input was one stream.

Perhaps this has zilch to do with stdin? Can someone explain this to me; maybe provide links to documentation.

Bonus: How would a script handle these multiple streams? If someone could provide a Python or Perl example, that would be very helpful.

Answer 1

This isn't multiple standard input. This is a bash'ism that called 'Process Substitution' http://tldp.org/LDP/abs/html/process-sub.html

It creates a pseudo file (/dev/fd/something) for each substitution. It's pretty useful. The command can only read as a stream, meaning it can not go back and forth with fseek. It needs to read it as a stream of bytes, like a pipe.

BONUS Answer

You don't need to do too much to use this. As far as your script is concerned, it gets a valid filename on the command line, that can be open()ed like anything else. As others have said, you'd see diff /dev/fd/XX /dev/fd/YY. If you do a stat() on any of these pseudo-files, you'll see it's a named pipe, and you should treat it with pipe semantics - namely no fseek() or ftell(). If you do a stat() test to explicitly see if it's a file (e.g. [ -f $1 ]) this will break - this is implemented as a named pipe after all.

Answer 2

<(...) does process substitution in bash. The output of the process in the parens is sent to an additional file descriptor beyond the normal 3, and a filename is returned corresponding to that file descriptor. In this way the output of a command can be treated as a filename to be passed to another command.

Answer 3

There is one stdin and one stdout for each process. They are usually connected to the terminal, but they can be redirected separately from one another.

In the example, there are two wget processes involved, each of which gets its own stdin and stdout. Each wget process writes to -, which is its stdout. Then bash's process substitution <(...) connects the stdout of the process to a unique pseudo-file, from which diff can read. Note that the two process substitutions yield two different pseudo-files! Thus, diff sees something like:

diff /dev/fd/XX /dev/fd/YY

where the stdout of wget -q -O - URL1 is connected to /dev/fd/XX, and the stdout of wget -q -O - URL2 to /dev/fd/YY.