measure amount of content downloaded from internet - SQUID

Question

I have set up a squid proxy and now want to monitor the amount of data transfer that is taking place. It would be nice to get numbers per domain and also about how much of the content is coming from cache. Is there any way to do this?

Answer 1

Lightsquid should do the trick. http://lightsquid.sourceforge.net/ I often use it on pfSense installations where I have Squid installed, I've found sometimes it doesn't work for the first day - I'm guessing it needs time to gather stats, never actually got to the bottom of that.

Answer 2

Calamaris is a nice perl script that parses Squid's log and outputs daily/weekly/monthly stats in mail/html/csv (pick one) format.

If you don't need active bandwidth monitoring, you should be able to do with. You get stats such as number of requests per domain, total traffic per domain, request size distribution per hour, top 10 requesters, top 10 visited domains, traffic distribution by content type, and so on...

You also get request status data, such as

# Incoming TCP-requests by status
status                          request      %  sec/req   Byte       %  kB/sec 
HIT                                12684   5.66    0.05      72M   1.68  115.92 
 TCP_IMS_HIT                        6682   2.98    0.02       2M   0.05   16.63 
 TCP_HIT                            2146   0.96    0.10      50M   1.16  235.88 
 TCP_NEGATIVE_HIT                   1338   0.60    0.04       1M   0.01   11.05 
 TCP_MEM_HIT                        1263   0.56    0.03       3M   0.07   83.50 
 TCP_REFRESH_HIT                    1255   0.56    0.15      17M   0.39   89.12 
MISS                              189807  84.67    4.02    4187M  97.33    5.62 
 TCP_MISS                         187967  83.85    4.05    4176M  97.06    5.62 
 TCP_CLIENT_REFRESH_MISS            1547   0.69    0.88       9M   0.20    6.52 
 TCP_REFRESH_MISS                    293   0.13    0.20       3M   0.07   50.48 
ERROR                              21683   9.67    0.10      42M   0.98   20.83 
 TCP_DENIED                        21573   9.62    0.05      42M   0.98   39.75 
 TCP_MISS                            110   0.05    9.02       0M   0.00    0.04 

Which is exactly what you're looking for (HIT = from cache, MISS = from Web)

Have a look at http://Calamaris.Cord.de/ if you're interested. It's free, of course.

Hope that helps.