Impact of setting allowUrlEncodedPercent to true in StrictHttpFirewall

Question

I need to implement a public REST API that manipulates usernames.

So I have an endpoint that looks like GET http://.../api/users/<username> where username can contain special characters (slashes, percents...) that are URL encoded (e.g. if my username is en/johndoe, my URL becomes http://.../api/users/en%2Fjohndoe).
By consistency with other endpoints in my API, I don't want to use a query param to pass the username.

When a username contains a slash (%2F), my GET request is rejected by Tomcat because of the default value (reject) of parameter encodedSolidusHandling (see https://tomcat.apache.org/tomcat-9.0-doc/config/http.html).
So I thought of double encoding the username (which would result in en%252Fjohndoe).
But in this case, my request is rejected by Spring Security because StrictHttpFirewall is configured this way by default (allowUrlEncodedPercent is false by default).

My understanding is that this is to avoid double URL encoding attacks.

But is double encoding a real threat? If it were, Tomcat would probably reject such requests as well, wouldn't it?
Is there any other risk in allowing URL encoding of percents?

Is there another solution (if possible not relying on a custom encoding) to deal with these special characters in paths?

Any feedback would be greatly appreciated.

Answer 1

Double-encoding is absolutely a real problem. I've used it myself in pentests, and found sites whose XSS protections were based on the raw input or a single round of decoding but then performed a second round before emitting the user content and thus were vulnerable. It is of course usually a bug to double-decode (and always a bug to validate or re-encode input at the wrong level of encoding), but then, it's always a bug to double-free (in e.g. C) or skip authorization checks on a secured endpoint, and those happen too. It's common for the final step before handling content (e.g. returning it in a response) to be decoding it; it's also common for the first step (immediately when extracting from the request) to be decoding it, and as such it's unsurprising to me when double-decoding happens even though it's almost never intended. After all, most of the time it has no effect, not even visible non-security bugs.

A significant part of the field of defensive software security is deciding what errors are worth spending resources / risking other bugs to mitigate, and there are rarely clear answers; it's not surprising that Tomcat and Spring came down on different sides of that line for the particular case of default handling for %25 in URL paths. As for making that decision yourself, consider both when you perform decoding and what state your codebase expects user content, and in particular try to judge the risk that you will have some code path that performs double decoding but only validates/escapes the single-decoded or raw content.