Plot custom log data on GNU/Linux

Question

I'm logging some data relative to clients NFS usage over a network. Now I want to plot it on a lines diagram, and I'm overwhelmed by the possibilities I'm finding out there. No one seems to be easy to understand for me.

Here's a simplified log:

1356112995  192.168.1.46    766
1356112995  192.168.1.12    14
1356112995  192.168.1.141   5
1356112995  192.168.1.11    38
1356114790  192.168.1.46    760
1356114790  192.168.1.12    10
1356114790  192.168.1.11    18
1356116586  192.168.1.46    758
1356116586  192.168.1.12    9
1356118387  192.168.1.46    783
1356120187  192.168.1.46    687
1356121987  192.168.1.46    699
1356123787  192.168.1.46    371
1356125587  192.168.1.46    717
1356127386  192.168.1.46    0

First column is a timestamp, it should be on the X axis. Second column is the client IP. Third column is the number of operations, it should be on the Y axis. The graph should be a line graph.

I want this to be automatic, so plotting from the command line is the way to go. No idea on how to do that, though.

It should generate a png file that will be subsequently uploaded to some webserver for visualization.

Answer 1

There are probably some ways to do this without installing software (With NFS, I assume you're using a Linux box). But, my favorite solution involves R which is free and available for Linux/Windows/Mac, etc. and will present you with myriad opportunities to customize your plot or manipulate your data.

Step 1
Make a script with code that looks something like this:

logfile <- read.table(file="client.log") #Adjust the logfile name as needed.
#This assumes tab separated columns. 
#If needed, column delimiters can be adjusted.
names(logfile) <- c("time.stamp","client","operations") 
#Rename the columns of the input data (if they are named at all)
require(lattice) #This package is needed for the xyplot() function below.  
#The main benefit of this function for your purposes is the ability to color 
#your plot circles by client name.
png(filename="logfile.png") #Designate the plot file type and save location
xyplot(operations~time.stamp,group=client,data=logfile,jitter.x=T,jitter.y=T)
#I've added jitter here which helps prevent plotted points from overlapping.
dev.off() #Close and save the plot

I called my script log.to.png.R

Step 2
From a Unix or Mac command line you can call the script in this manner:
$R CMD BATCH log.to.png.R

Answer 2

This is really late, but as I happened across this post, I'll put in my two pennies.

Without installing R, we can use call awk within gnuplot:

plot "<awk '{print $1, $3}' logfile" u 1:2

where the output of the awk command in the double quotes is used by gnuplot to plot your data. The awk command simply prints out the first and third column.

You can place the above command into a gnuplot file, such as plot_log.gp and then call it on the command line like so

gnuplot -e "logfile='your_actual_log_file.log'" plot_log.gp

which is a command that is easily placed into a cron job or some script you have. Note the single quotes around the text.

I just wanted to demonstrate that gnuplot is actually really flexible as it can take command line arguments and assign them as variables within a script. No heavy dependencies (looking at you, R) as awk is pretty pervasive.