I'm working on a REST API endpoint where we only accept requests from certain domain names. Whitelisting. A dev I'm working with recommended that we return HTTP 400 instead of HTTP 403 if the incoming IP address is not whitelisted. They said it was because we don't want to disclose any unnecessary information.
Is this a common security practice? If so, what is the point of the other HTTP error codes (4xx)? Is there ever a scenario where it's safe to return specific error codes?
There is a trade-off between two requirements: what to reveal to help the user and as aid in debugging and what to hide from the user as another layer in defense.
Returning 400 "bad request" instead of a more specific error code is definitely misleading since this error code is related to malformed requests. It will confuse attackers but it will also confuse intended users of your system, thus possibly decrease customer satisfaction and increase your support costs. The problems of too generic error messages can be seen with TLS, where one often only gets a generic "handshake failure" which makes it very hard to figure out what the underlying problem is.
Returning 403 "forbidden" with a very specific reason like "not on source IP whitelist" might reveal too much information about this layer of protection though. A generic 403 instead just means that there is access control and does not reveal anything about its details. This might be a good trade-off between not revealing too much internals and allowing focused debugging of problems.
Apart from that there are specific error codes which trigger actions in clients: 401 and 407 will (in interactive use) lead to prompting for authentication credentials, so they should not be simply replaced with a plain 400 when authentication is requested.
It can make sense to return a misleading error code, provided the error code is consistent with the request. 400 is not a generic error code but is only intended for malformed requests. IMHO returning it for a perfectly correct request is a bad idea.
403 only means that the request is not allowed, but does not disclose the exact reason so I do not think it can be a problem to consistently return it for request coming from unauthorized domains. Alternatively, 404 (nonexistent URL) can also be used as a generic error.
Honestly, I would stick to 403. Imagine that a valid user takes it machine elsewhere and tries to connect from there with an IP coming from an unauthorized domain. Security reasons require that you reject the request. Telling them that the request is unauthorized does make sense, and they could guess the reason. But giving a different error could have them spending time in trying to fix a network (local proxy) problem and in the end lead to a bad user experience for little if any added security.
If it is a common use in your organization, I cannot imagine a strong reason not to follow it anyway, but if it is just a why not idea from a co-worker, IMHO you can safely forget about it...
I agree with the other contributors. As said by @MechMK1 telling a potential attacker that their IP address is not whitelisted does not provide them any real advantage.
What your dev is proposing will only make testing and debugging more difficult for the developers, because they will always have to keep in mind that they can't trust the error codes they are receiving. Not to mention unit testing.
And since this is a REST API using proper HTTP status codes is even more crucial because they are examined to determine the outcome of the requests.
But this one has me confused:
we only accept requests from certain domain names
Do you mean IP addresses or you are performing reverse DNS against the remote host ? Be careful, PTR records can be spoofed.
If there is a dedicated domain name or IP address for the API you could as well perform whitelisting at the firewall level, so that unauthorized IP addresses will not even have an opportunity to interact with the website hosting the API.
Note that before a potential attacker even gets a 403 response they should figure out how to send valid requests to your API (unless your API is publicly documented). Until then, they should actually get 400 errors, precisely because their queries are malformed and not in the expected format.
What your colleague is suggesting is Security by Obscurity.
It arguably has zero real security benefits. The thinking goes like this: if an attacker is not very good, then obscurity by security may work, but in this case any other kind of security (i.e., the "good" kind of security) will work as well. If, on the other hand, the attacker is very good (or has more knowledge, for example the source code of the application under attack, which is generally not considered part of security these days), then obscurity will simply not help.
(There is a valid reason for obscurity, and that is if you're living in an oppressive regime, using something like Tor or other schemes to communicate without that being visible. But even then - this kind of obscurity is not there to keep the content of your transactions secret, TLS would be enough for that. It is there to hide that you are communicating at all. Not the use-case you're currently asking about.)
So you need the "good" variant of security in any case. In your case this means that your whitelist processing must be tight - for example, the way your whitelist is stored must be such that an attacker cannot easily add their identity to it; it must also be hard to spoof the identity, and so on. The simple fact that there is a whitelist is part of the algorithm, and thus should not be relied upon for security. You might think about ditching the whitelist approach at all, and switch over to client certificates. But all of that is not here nor there...
To answer your question: the HTTP codes should be used as intended, and the code in itself provides no particular benefit to an attacker. If it really would make a difference, you have a whole other problem to solve. At the same time, the additional info you can give along in a plaintext response message should not reveal any details helpful to an attacker. If you want to make it easy for you, just return the standard meaning of the HTTP code as message body, most libraries should do that automatically if you leave the response message empty.
Do not add any kind of debug info, JSON output, stack traces etc. into the message body of your 4xx or 5xx responses. Those go into your server-side logfile.
If a 4xx is for human consumption, keep in mind that only throwing a bare 4xx response is very user unfriendly - make it a proper "normal" HTML page within your UI design (i.e. with the regular menu bars, side panels, a link to your help desk or documentation or whatever it is you use) that's as beautiful as you want for your customers. But still do not give any particular information that tells a possible attacker anything they can use to refine their attacks.
This will look quite similar to the other requests in that I agree as well that it's better to show a clear message.
Note that you should apply the IP whitelist as the first check. I.e. you don't parse the json parameters, and error a 400 if they are wrong, then issue a 403. Or, a more extreme case, check if the password is wrong and only if it is right then verify the source IP. No, it's better to check the IP first, and fail early if the IP address is not authorized, without parsing any of their data.†
At this point, the only information provided by a detailed error is that you are checking the source IP. Something which is hard to guess anyway [if not directly (almost) spelled out in the documentation for end users], and that they cannot "bypass" anyway (hopefully).‡
I would go even further and suggest not only to say "Your IP address is not whitelisted", but also the detected IP address such as "198.51.100.85 is not allowed here".
Anonymous mentions that it will make testing and debugging more difficult for the developers, but I add that it's not only testing that will be hampered but production as well.
If it's only used internally, the more verbose error may help the sysadmins or developers deploying the systems connecting to the API when setting it up (or when suddenly discovering that it no longer works after some non-announced routing changes).
If it's also used by end users, it will be even more useful, for both end users ("D'oh!, I needed to be connected from X") and your own support having to help users when "It doesn't work". If you are lucky, when asking "Are you accessing from a whitelisted IP?" you could receive a clear "No, we changed it 2 months ago, our connecting IP address is X now". Other people wouldn't even know what their public IP address is, claim that they are connecting (to your system in the internet) from IP 192.168.1.3 ...
† It's common to whitelist at the firewall level as well, in which case it wouldn't even reach the application layer.
‡ In order to bypass an IP filter options would basically be to perform BGP hijacking, control an intermediate router between the server and a whitelisted client or get a client to open a malicious web page to send the evil request on your behalf (assuming the API will not reject what comes that way). If the API needs to cater to proxys forwarding the original IP address (such as X-Forwarded-For header), there may be vulnerabilities on its handling that could allow additional bypasses.
