7

I come across quite a few use cases where it would be very useful to take input from a (usually newline-delimited) stream and summarise it in a top-like fashion (see top, iotop, etc). A kind of on-the-fly pivot table.

e.g. Take the log-esque input:

I heard A from unit 1 and it said "Great!" 56
I heard A from unit 2 and it said "Oh no!" 42
I heard C from unit 1 and it said "Waiting for input." 33
I heard B from unit 3 and it said "Stopped." -1
...

From this, we could run a tool with a regex and group indicators:

topify [lineout] [regex] [name #1] [group #1] [name #2] [group #2] [All other columns name position]
     where:
         lineout is the number of lines before removing it from the display
         regex is a regex of the lines to match, complete with group indicators
         name #n is a string for the title of column n
         group #n is the number of the group in the regex

e.g.

topify '/^I heard ([A-Z]) from unit ([1-9]) and it said "(.*)" ([-0-9]*)$/' Unit 2 Status 1 Message 3 RetVal 4

This would display in an interactive way, such that the columns could be selected/reordered, etc.:

Unit Status Message            Retval
1    C      Waiting for input. 33
2    A      Oh no!             42
3    B      Stopped.           -1

I understand the brittleness of it, but I'd be really surprised if it hadn't been built before and wanted to check before I go into building it. I also appreciate it isn't particularly complicated to write so maybe everyone has just implemented their own solution...

Has anyone seen a tool like this?

(Please excuse the tags I've used here. I know I may be pushing/breaking the rules of some tags, but this is very generalised. Suggestions welcome.)

tudor -Reinstate Monica-
  • 1,805
  • 18
  • 35
  • This really is something that you ought to write yourself… the closest thing I can think of is the `watch` command. – MikeyB Aug 20 '13 at 01:11
  • That was my first thought, too, but watch doesn't group things together or filter with smarts. I thought about using an awk/watch/shell combination but it's one step more complex than that. – tudor -Reinstate Monica- Aug 20 '13 at 01:28
  • I think that `awk` is the way to go... using a keyed array. – Floris Aug 20 '13 at 04:01

1 Answers1

1

You don't need to write a tool, the standard unix toolset can accommodate you just fine.

#!/bin/bash
echo -e 'Unit\tStatus\tMessage\t\t\tRetval'
cat /var/log/filename | awk '{match($0,"\".*\"",a)}{print $6 "\t" $3 "\t" a[0] "\t\t" $NF}' |sort -k<fieldnum>

Put that in a .sh file and run watch on it.

Aaron Tate
  • 1,222
  • 7
  • 9
  • 1
    I think you just wrote a tool. – Michael Hampton Sep 11 '13 at 02:04
  • Good effort, but I think you missed a few criteria "(see top/iotop, etc.)...on-the-fly....in an interactive way such that the columns could be selected/reordered..." It's possible that you could rewrite the command, but this isn't very flexible once you start the script and you could miss information by doing so. – tudor -Reinstate Monica- Sep 11 '13 at 05:09
  • Fair enough, it doesn't match criteria, but then not much other than specifically designed custom software will, if you know want to write it yourself look at the ncurses development libraries, other than that i'm fairly certain you'll find nothing to meet your exact requirements. – Aaron Tate Sep 11 '13 at 06:57