Different response between curl and openssl s

I'm having a headache with some results.

Running the following command results in an HTTP 200 response:

$ curl -v -L www.example.com >/dev/null

while running this, and using the exact headers that curl uses results in an HTTP 403 response:

$ openssl s_client -connect www.example.com:443

There is no problem with certificate, as curl and openssl verify it. I really can't think of anything to start digging this problem, so even just a wild guess is more than welcome!

The 403 response comes from Fortigaurd servers, reading "Web filter service error: all Fortiguard servers failed to respond".

I can provide more information, such as curl and openssl outputs, if needed.

vfsoraki

Posted 2019-01-09T01:19:31.250

Reputation: 1 927

1Are you sure you got the line endings right? The HTTP specification requires a carriage return followed by a newline after each line. Just using a newline is not correct. I also noticed that you don't specify -quiet or -ign_eof. This means that s_client will parse your text for commands like R which might appear in your headers! – David Schwartz – 2019-01-09T01:22:11.583

@DavidSchwartz Thanks, I didn't know about those parameters. Adding -quiet and also -crlf didn't help. Still getting 403. – vfsoraki – 2019-01-09T02:07:54.173

@DavidSchwartz I'm almost sure. I used emacs to change all endings to crlf, then copying that into my ssh session inside terminal. Unless copy/paste doesn't change line endings, all things should be fine. – vfsoraki – 2019-01-09T02:21:45.600

1(1) Copy/paste to a terminal very well can change line endings; put your request in a file (and use <file) so you can edit exactly what you want. Be sure to include the empty line after the last header (that is a vital part of HTTP format). (2) The server (or its frontend) may want SNI, which s_client doesn't do by default; try adding -servername $hostname. – dave_thompson_085 – 2019-01-09T05:37:03.943

1Your curl command is requesting on port 80 (HTTP), while your openssl command is requesting on port 443 (hopefully configured for HTTPS). Check your server config. – garethTheRed – 2019-01-09T07:17:57.437

@garethTheRed Yes you are correct. The server returns a redirect to https when using curl, which is handled by -L flag. Anyway, this does not change anything. – vfsoraki – 2019-01-09T12:52:04.733

@dave_thompson_085 Thanks! SNI was the problem. Adding mentioned flag solved issue. I will accept if you write your comment as an answer. – vfsoraki – 2019-01-09T12:53:32.370

Different response between curl and openssl s_client

No answers