3

I have a ProxyPass configured to reach the following: On my server I start a service which provides a Rest-API listening on port 7777. From the client side, I want to be able to call this API like this: http://example.org/servicename/PARAMETER

A full call to this API should look like this: HTTP PUT @ http://example.org/servicename/PARAMETER (where PARAMETER is some string). Internally this should translate to the following url: http://server.ip:7777/servicename/PARAMETER

Everything works as expected as long as the PARAMETER is not (!) like this: http://parameter.org (actually I need to URL encode it: http%3A%2F%2Fparameter.org). So all in all, the call is http://example.org/servicename/http%3A%2F%2Fparameter.org

The http:// in the parameter confuses apache leading to the following error message in the reply to the call: 

!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /servicename/http://parameter.org was not found on this server.</p>
<hr>
<address>Apache/2.2.22 (Debian) Server at example.org Port 80</address>
</body></html>

If I replace http%3A%2F%2Fparameter.org with, for instance, test, everything works as it should. Somehow the http:// in the parameter confuses apache. Is there a way to let apache ignore it?

My current configuration of this vhost looks like this:

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/example
        ServerName example.org
        ErrorLog /var/log/apache2/example_error.log
        LogLevel warn
        CustomLog /var/log/apache2/example_access.log combined

    <IfModule mod_proxy.c>
        ProxyRequests Off

        ProxyPass / http://localhost:7777/
        ProxyPassReverse / http://localhost:7777/
    </IfModule>
</VirtualHost>

Prerequisites:

  • I am not able to change the behavior of the API. it's third party.
  • I need to be able to provide URLs as parameters.

Edit 1:

tail -f /var/log/apache2/example_access.log yields

128.xxx.xxx.xxx - - [19/Aug/2015:16:53:17 +0200] "PUT /servicename/http%3A%2F%2Fparameter.org HTTP/1.1" 404 521 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36"
beta
  • 175
  • 1
  • 2
  • 7
  • Can you check, in the access.log of the proxied web-server which URL got request, when using this parameter: http%253A%252F%252Fparameter.org ? As a side note, I don't think that mod-proxy gets confused. It's only that the receiving web server is receiving (by mod-proxy) a not-properly encoded request, resulting in a 404 error. – Damiano Verzulli Aug 19 '15 at 23:25
  • 1
    Also, as for this: http://stackoverflow.com/questions/12895674/apache-mod-proxy-url-encoding I suggest to investigate both the AllowEncodedSlashes and nocanon option. – Damiano Verzulli Aug 19 '15 at 23:39
  • I added "Edit 1" to the question, showing `example_access.log` output after the HTTP PUT request. – beta Aug 20 '15 at 07:53
  • 1
    Sorry. You added the log-line of your FRONT-END apache instance, while I'm wondering which is the HTTP/POST request that mod-proxy SEND to your BACK-END web-server (the one listening on the 7777 port). In other words, from your log-line, we saw that the front-end received the "_PUT /servicename/http%3A%2F%2Fparameter.org_" but we still don't know which PUT/request have been sent (by mod-proxy) to your 7777 port (encoded or not? And if yes, how?). If you don't have access to related log file, please try some "tcpdump -n -i lo port 7777" packet capture and let us know. – Damiano Verzulli Aug 20 '15 at 08:09
  • okay, sorry I got you wrong. the issue is: the HTTP PUT is not arriving at the backend web-server. so mod-proxy doesn't even forward it there. if I make a valid call (i.e. without `http://` in the parameter, e.g., `http://example.org/servicename/test`), then the log in the back-end says: `2015-08-20 10:34:06 0:0:0:0:0:0:0:1 - - 7777 PUT /servicename/test - 200 36 0 347 http://localhost:7777 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36 -` – beta Aug 20 '15 at 08:35
  • As you suggested, I also did "tcpdump -n -i lo port 7777". After submitting the request, no packets were captured, indicating that mod-proxy doesn't even forward the request to the internal back-end server. – beta Aug 20 '15 at 09:00
  • `AllowEncodedSlashes On` + `nocanon` did the trick! Thanks. Would have never come up with that one on my own ! – beta Aug 20 '15 at 09:12
  • Would you mind to accept this as an answer, here, should I report it as such (a proper answer, instead of a comment)? – Damiano Verzulli Aug 20 '15 at 19:05
  • answers cannot be accepted, when they are in comments. so in order to "close this question" we would need to accept an answer. maybe just post your comment above as an answer, then i can accept it. – beta Aug 21 '15 at 11:45

2 Answers2

9

In the default Apache configuration, the AllowEncodedSlashes directive is set to off. This means that: "...The AllowEncodedSlashes directive allows URLs which contain encoded path separators [...] to be used in the path info. With the default value, Off, such URLs are refused with a 404 (Not found) error. ..."

So the problem is that mod_proxy is not proxying your URL-based POST requests, as Apache is refusing them (with a 404) before mod_proxy taking action.

Another possible problem relates to the URL-encoding process: your apache (front-end) will surely receive a properly URL-encoded string (the one you're sending to it: http://example.org/servicename/http%3A%2F%2Fparameter.org ) and I expect it (apache) will URL-decode it while internally processing related POST request. So I expect that mod_proxy, inside Apache, will receive a real URL (not encoded) and I wonder if, while proxying, it will perform an URL-encoded cycle. On the official ProxyPass documentation I see: "Normally, mod_proxy will canonicalise ProxyPassed URLs. But this may be incompatible with some backends, particularly those that make use of PATH_INFO. The optional nocanon keyword suppresses this and passes the URL path "raw" to the backend. Note that this keyword may affect the security of your backend, as it removes the normal limited protection against URL-based attacks provided by the proxy", so you should evaluate also the usage of the "nocanon" option.

Both issues (AllowEncodedSlashes and nocanon) have been mentioned in this StackOverflow question

Community
  • 1
Damiano Verzulli
  • 3,948
  • 1
  • 20
  • 30
0

I think it has this behaviour because in http:// the / is still seen as directory separator. Thus you're looking for the resource paramater.org under folder http: (by folder, I don't really mean a real folder as it could just be an access path but you can get the point).

I don't think you can juste type / for a resource in an URL, so you have to use %2F

axellink
  • 83
  • 1
  • 7
  • Thanks for your help. I think you are right. However, I am doing already, what you suggest (given, that I understand correctly, what you suggest). As explained, the actual http call is `http://example.org/servicename/http%3A%2F%2Fparameter.org`. So I URL encode the `http://parameter.org` in the call already. Still, it doesn't work. – beta Aug 19 '15 at 13:51
  • oh ok I didn't understand that even with %2F it does not work. Hm in this case I'm afraid I won't be able to help you much. I would have have suggested that the parameter should be an url parameter (like `http://example.org/servicename?url=http%3A%2F%2Fparameter.org`) but as you ddon't have access to the service this won't work either. – axellink Aug 19 '15 at 13:59
  • yes, unfortunately you are right. so right now i am stuck.. we will see if there's a solution or workaround, hopefully. – beta Aug 19 '15 at 14:09
  • 1
    give a look to url rewriting maybe but I can't guarantee a success – axellink Aug 19 '15 at 14:12