We use an explicit proxy to access the Internet.
This is what the HTTP header for "407 Proxy-Authentication required" looks like:
Proxy-Authenticate: BASIC realm="xxx"\r\n
Cache-Control: no-cache\r\n
Pragma: no-cache\r\n
Content-Type: text/html; charset=utf-8\r\n
Proxy-Connection: close\r\n
Connection: close\r\n
Content-Length: 813\r\n
Now when transmitting the Proxy-Authorize header (the following step when the browser transmits credentials), behavior seems to be different between browsers (here password=passwörd):
Using FF or IE (here IE):
Credentials: user:passw\366rd <----------- ISO-8859
Accept: text/html, application/xhtml+xml, image/jxr, */*\r\n
Accept-Language: en-US,en;q=0.8,fr-LU;q=0.5,fr;q=0.3\r\n
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko\r\n
Accept-Encoding: gzip, deflate\r\n
Using Chrome
Credentials: user:passw\303\266rd <----------- UTF-8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3\r\n
Accept-Encoding: gzip, deflate\r\n
Accept-Language: en-US,en;q=0.9\r\n
Using FF or IE the proxy complains that the encoding is not what it expects. This means that it expects UTF-8 since auth using Chrome is working.
I found this online:
Previously, Firefox was using the ISO-8859-1 character encoding for the username and password in Basic HTTP authentication requests. Firefox 59 and later will use UTF-8 instead for the Authorization header to make sure non-ASCII characters like French accented letters are encoded properly.
However we are using FF 60.7.2ESR and it still seems to be sending ISO-8859 just like IE11 shown above.
There are several relevant RFCs: 7617, 7235, 7615.
https://www.rfc-editor.org/rfc/rfc7617#section-2 states that
For backwards compatibility reasons, this specification continues to leave the default encoding undefined, as long as it is compatible with US-ASCII (mapping any US-ASCII character to a single octet matching the US-ASCII character code).
Then, https://www.rfc-editor.org/rfc/rfc7617#section-2.1 states
In challenges, servers can use the 'charset' authentication parameter to indicate the character encoding scheme they expect the user agent to use when generating "user-pass" (a sequence of octets). This information is purely advisory. The only allowed value is "UTF-8" ...
It gives an example: WWW-Authenticate: Basic realm="foo", charset="UTF-8"
And suggests the same applies to "Proxy-Authenticate: ".
What does that mean for challenges when the password is not US-ASCII and no charset authentication parameter is sent? It must not be undefined but it doesn't state what it should be then.
We could force the proxy to interpret everything as ISO-8859 but that would break everything where the client sends UTF-8.
