1

Is there a way to use pipes or FIFO in combination to continually monitor several (same format) log files whilst sorting on the first field and then being able to effectively do a tail -f on that continual, sorted output?

I can do stuff like :-

mkfifo /tmp/logfile

tail -F -q *.op > /tmp/logfile &

tail -f < /tmp/logfile

but I have tried piping that somehow through sort -k 1 but cannot get it to work.

Jon Scobie
  • 11
  • 1

2 Answers2

2

Why wouldn't this work?

tail -qf *.op | sort -k 1

I guess multitail might work for you, unless the log files you speak about are huge. Something like

multitail -R 5 -l "sort -k1 *.op"

would execute sort -k1 *.op command every five seconds and put it nicely to multitails output view. Of course, if only X last lines from log is enough, then tail -n somenumber *.op | sort -k1 would do at the command part.

Janne Pikkarainen
  • 31,454
  • 4
  • 56
  • 78
0

The problem here is probably that sort is waiting for the end of the stream, and can't start until it reaches it. Typically sort is used on a finite set of data, so it waits for all of it, and then sorts. If it didn't wait, it would have to keep resorting as new data came it, which doesn't really fit into the unix pipes architecture.

Perhaps what you could do is use tail -100l to take the last 100 lines of each log, sort by time - and slice off as many lines as you can display, and then sort in the way you want. Then just loop to update regularly.

Peter Bagnall
  • 196
  • 4
  • I did try a loop scenario and the first field of each log is a timestamp to microseconds. I might have to bite the bullet and write a bit of code to do it as each log file actually represents a threads' output and new threads can come along over time - meaning new log files all the time. All output used to come in one file but locking contentions within the app meant I had to split it. From a monitoring purpose though, it would be very useful to somehow have them all amalgamated back in order with the ability to then pipe this info through the standard unix tool set. – Jon Scobie Jun 27 '12 at 12:45