I was curious on how I would go about calculating a running percentage for specific URLs... For example, when varnishlog is executed you get a realtime output for information of incoming http requests.
Below an excerpt with some information cut out.
46 ReqStart c
46 RxRequest c GET
46 RxURL c /cat?getParam
46 RxProtocol c HTTP/1.1
46 RxHeader c User-Agent: **************
46 RxHeader c Accept-Encoding: gzip
46 RxHeader c Host: myServer.org
46 RxHeader c Connection: Keep-Alive
46 RxHeader c Cookie: ********
46 RxHeader c Cookie2: ************
46 VCL_call c recv lookup
46 VCL_call c hash
46 Hash c /cat?getParam
46 Hash c myServer.org
46 VCL_return c hash
46 VCL_call c miss fetch
46 Backend c 90 default default
...
46 ObjHeader c Server: Apache-Coyote/1.1
46 ObjHeader c Content-Type: text/xml
46 ObjHeader c Content-Encoding: gzip
46 ObjHeader c Vary: Accept-Encoding
46 ObjHeader c Date: Thu, 05 Sep 2013 12:56:04 GMT
46 Gzip c u F - 1596 8098 80 80 12697
46 VCL_call c deliver deliver
46 TxProtocol c HTTP/1.1
46 TxStatus c 200
46 TxResponse c OK
...
46 ReqEnd c ...
What I am interested is RxURL
. Specifically, I have requests starting with:
- /cat?
- /template?
How would you go about running a command that can count:
- total number of requests
- number of requests starting with /cat
- number of requests starting with /temp
and output these numbers at the end?
Currently, I am executing varnishlog | grep "Hash" > requests.txt
,
And then utilizing an excel spreadsheet to clean up, and make my counts and calculate percentage.
I was curious though if AWK or a script could be utilized to do this directly on the server.
Update
varnishtop -i txurl
is even more suited as it displays the URLS directly.