I'm searching for a command line tool working with a stream of lines (tail -f typically) and counting them like : tail -f /var/log/apache2/access.log | cut -d' ' -f1 | SOME_COMMAND and displaying a top-like view as :
52 xxx.xxx.xxx.xxx 12 xxx.xxx.xxx.xxx 6 xxx.xxx.xxx.xxx 2 xxx.xxx.xxx.xxx
It could be so handy, associated for example to this sh :
#!/bin/sh
# NCSA structure :
#IP - - [DATE] "METHOD URL HTTP/VERSION" STATUS LENGTH "REFERER" "USER AGENT"
QUERY=""
while [ "$1" ] ; do
case "$1" in
ip) QUERY="$QUERY"'\1' ;;
date) QUERY="$QUERY"'\4' ;;
method) QUERY="$QUERY"'\5' ;;
url) QUERY="$QUERY"'\6' ;;
version) QUERY="$QUERY"'\7' ;;
status) QUERY="$QUERY"'\8' ;;
length) QUERY="$QUERY"'\9' ;;
referer) QUERY="$QUERY"'\10' ;; # Does not work...
useragent) QUERY="$QUERY"'\11' ;; # Does not work
*) QUERY="$QUERY""$1" ;;
esac
shift
done
sed -r 's/^([^ ]+) ([^ ]+) ([^ ]+) \[([^]]+)] "([^ ]+) ([^"]+) HTTP\/([^"]+)" ([^ ]+) ([^ ]+) "([^"]+)" "([^"]+)"$/'"$QUERY"'/g'
With this command i'm searching and my script you could do : cat somelog | ncsa.sh url | SOME_COMMAND and get an top of your viewed url'z, or referer, or what you want
(and if someone can fix the bug of \10 interpreted as \1 followed by a 0 ... :p )
Have a great day !