Lightweight tool for viewing raw HTTP messages?

I'm investigating differences in behaviour between a couple of Web servers. I need to see raw response data from the servers (i.e. before the response is de-chunked if it has "Transfer-Encoding:chunked" and before it is decompressed if it has "Content-Encoding:gzip").

I can find plenty of simple HTTP client that nearly do what I need (e.g. Poster, RESTClient), but they tend to decode the response one step too far.

Network analysers like Wireshark give me what I need but are a bit heavyweight. Telnet is my best bet so far, but is a bit too simplistic (actions like capturing data or entering requests are a bit laborious).

Can anyone recommend a good, lightweight tool for sending / viewing the raw data that constitute HTTP messages?

Edit: I should add that I'm on Windows. Also, the tool would need to work both with remote and local servers.

rewbs

Posted 2010-02-28T15:01:17.063

Reputation: 295

Interesting question – at. – 2010-03-01T19:28:30.220

Answers

Fiddler is an "HTTP debugging proxy". It allows you to inspect HTTP messages, and also compose new HTTP messages. Each message can be viewed in multiple ways, such as raw text, and in a hex editor. A possible downside is that it requires the .NET Framework.

sblair

Posted 2010-02-28T15:01:17.063

Reputation: 12 231

i vote for curl

$ curl -I http://www.amazon.com
HTTP/1.1 405 MethodNotAllowed
Date: Tue, 16 Mar 2010 01:21:45 GMT
Server: Server
Set-Cookie: skin=noskin; path=/; domain=.amazon.com; expires=Tue, 16-Mar-2010 01:21:45 GMT
x-amz-id-1: 02Q7DN8FGW708892524E
allow: POST, GET
x-amz-id-2: POduQxVVwgUXSZRQEM5nWw+9DDvV22s7
Vary: Accept-Encoding,User-Agent
Content-Type: text/html; charset=ISO-8859-1

ZaphodB

Posted 2010-02-28T15:01:17.063

Reputation: 151

Tamper Data is a great plugin for Firefox that shows all HTTP requests that the browser makes. It shows the headers, content and everything you would wanna know. If you have Firefox available to you, give it a squeeze.

alt text

Sam152

Posted 2010-02-28T15:01:17.063

Reputation: 2 052

Tamper data is a great tool - I use it a lot. But I think it doesn't quite cover the requirements in this case. For example, I can't see the raw bytes composing a chunked or a compressed response with tamper data. I'll upvote anyway when I have enough rep, because it is a very nice tool. – rewbs – 2010-02-28T15:34:58.710

How about NirSoft SmartSniff?

SmartSniff is a network monitoring utility that allows you to capture TCP/IP packets that pass through your network adapter, and view the captured data as sequence of conversations between clients and servers. You can view the TCP/IP conversations in Ascii mode (for text-based protocols, like HTTP, SMTP, POP3 and FTP.) or as hex dump. (for non-text base protocols, like DNS)

alt text

SmartSniff is freeware. The zip file containing the tool as well as documentation is only 65kb.

John T

Posted 2010-02-28T15:01:17.063

Reputation: 149 037

Very nearly what I need! I'll upvote as soon as I have enough rep. As far as I can see, the features that are missing are:

I can't use it to compose my own requests (I'd still to use another tool like telnet for that, and then monitor the results in SmartSniff).
Out of the box, it doesn't seem to detect communication with a local server. I expect I need to set up a loopback device.

2010-02-28T15:38:04.257

@rewbs It should be able to detect LAN communication. I just gave it a test run by pinging some local devices: http://i47.tinypic.com/ftn33n.png But you are correct, it does not allow you to compose requests unfortunately.

– John T – 2010-02-28T16:10:47.843

Yes, LAN communication is fine. The problem is when the server and client are both on localhost (e.g. in a webserver development environment setup). – rewbs – 2010-02-28T17:18:38.930

@rewbs I see what you mean. You can probably trick the program by mapping a second address to your NIC. Use the second address to access the web server. – John T – 2010-02-28T17:26:36.057

I just saw your question, I guess you can do it by a workload generator like httperf. It is a linux based tool that can be used in windows using Cygwin. You can change its code to see complete message, as far as i know, it shows the reply text when you use the command --print-reply body and the header when you use --print-reply header I hope it works works for you

user31270

Posted 2010-02-28T15:01:17.063

Reputation:

Tcpdump is a simple plain text interface for a network sniffer. You can run it with simple filter rules to allow log or display the connection(s) you're interested in. It is also multi-platform, available for MS Windows (alternative version) as well as most Unix platforms. You can also save a log of packets with tcpdump, and then decode with wireshark later.

Sample usage:

tcpdump dst port 80

mctylr

Posted 2010-02-28T15:01:17.063

Reputation: 1 290