This seems like an easy question, but I've failed to find an answer.
One of the uses of CAPTCHA is to cope mitigate Denial of Service attacks. Suppose an adversary performs excessive login attempts, leaving other users unable to log in; the service can require solving a CAPTCHA before logging in. But why, in this case, does the CAPTCHA service continue to run while the login service fails under the same traffic load? Why does it work?
Snark aside, CAPTCHA is a very poor solution for D/DoS protection. While it does have some effect, this is minimal, and easily compensated for by the attackers.
CAPTCHA solves the wrong problem for this, and solves it badly.
CAPTCHA does not try to rate-limit the connections; it is not intended to protect the login mechanism from attacks; it is not possible for CAPTCHA to differentiate between attackers and legitimate users.
The only thing it does do - and not very well, at that - is, not surprisingly: Tell Computers and Humans Apart. This might in fact be useful to you, if you were in a Matrix-like reality, where all Computers are the enemy, and all Humans are allies.
In our reality, that is a pointless differentiation. Ostensibly, this might help with preventing scripted attacks - but even if this were true (it's not), there are plenty of ways to bypass that requirement - e.g. CAPTCHA proxies (where you need to solve a CAPTCHA before we show you dem pr0n) and CAPTCHA farms (where you hire some fareasterners to "solve" CAPTCHA for you, at 4$ per 1000 pops).
Moreover, in the case of DDoS - often this attack is mounted by "political movement" - i.e. large masses of humans decide to bring down a certain site. So yeah, CAPTCHA would be irrelevant here.
Besides all that, state-of-the-art CAPTCHA is far behind state-of-the-art OCR. If you want your users to easily decipher those squiggly images - the computer can do this too, pretty well. The best CAPTCHAs were measured at 20% OCR success rate - which effectively means that for every successful request, the attacker would simply need to send 5 requests. Not quite the order of magnitude that would make a substantial difference, to an attacker that is already planning on mounting a DDoS.
Some of my other posts here and on SO regarding this:
TL;DR:
The question is based on a faulty assumption:
How does CAPTCHA defend from DDoS attacks?
It doesn't.
At most it might require a bit more effort from the attacker, but not much.
It is easier/faster to check if a CAPTCHA is correct than to lookup a user in the database (this may include new connections, hashing and more). A server first check the CAPTCHA, if it is correct then process the rest of the login, if not return an error.
It is important to note that making of a CAPTCHA image requires some processing, but this can be done quite effective pre rendered images or even outsourcing to other services (like reCAPTCHA).
A captcha prevents an attacker from performing more database-intensive operations that may cause a DoS via CPU or memory exhaustion. However, this is only the case when the CPU and memory consumption caused by generating the captcha image must be less than that of the normal page request. One way to ensure this is to use an off-site captcha service, such as reCAPTCHA.
Ordinarily, the DoS prevention aspect of the captcha is a by-product of using it as a login security measure, to prevent automated login attempts.
I don't think that captchas could help against a Denial Of Service attack. I think they could even pose a thread of DoSing the site if the captcha algorithm is CPU intensive.
In order to execute a successful DoS attack on a server the costs of the attacker should be lower than the cost of the attacked server. The server has high costs even if he uses captchas. The server needs to generate the captchas and transmit the whole page to the clients.
Edit: In short: A Captcha can be contraproductive if the generation of the captcha costs too much.
It doesn't, although you sometimes see claims to that effect. CAPTCHAs are intended to prevent automated submission of data on a website. DDoS attacks are almost always at the network level, well before data is submitted over HTTP. If an HTML form is vulnerable to DoS (not DDoS) attacks, a CAPTCHA would make exploitation difficult, but the proper solution is to fix the form, not slap a band-aid (CAPTCHA) on it.
One of the uses of CAPTCHA is to cope mitigate Denial of Service attacks.
Is that a use of CAPTCHA? I doubt it could help very much at all with such an attack.
Whether the CAPTCHA field is correct or not is largely irrelevant, if the request is simply sitting in a queue waiting to be processed.
If the attacking request is accepted by the HTTP service, then it has done is job.
I'm not sure how a remote CAPTCHA service can improve the situation much. It cannot prevent the hacker from making the request. The only thing it potentially does, is reduce the load on the main server, as it doesn't then need to generate the images.
DDoS attacks are better handled by the firewall layer, where requests can be dropped before they ever hit the web server.
Depending on your method of implementation simply just being able to differentiate robots from real users is completely adequate enough to mitigate a Layer 7 type attack, so long as you are hosting the CAPTCHA on a different machine/ip, you can then forward only the traffic that passed the captcha.
I would also suggest setting up multiple of these captcha proxies in a DNS round-robin for redundancy, then you're increasing the pipe size so that the chances of your servers going down due to pipe saturation drops drastically.
My company's site was impacted twice this week by a DDoS attack ... not on us, but on another site hosted in the same facility as ours. So how does this answer questions about CAPTCHA? It's that most DDoS attacks are simple floods-of-requests, and may have nothing to do with your login page or a form, where (presumably) a CAPTCHA exists.
DDoS is remarkably easy to do, and hard to detect and prevent. So sure, you could make a server do lots of work to generate and serve a CAPTCHA and take it down that way. But a bot-net or cluster of cloud computers can quickly generate enough traffic to flood the queues of most web servers, or even before that, the routers and firewalls.
This might have been true in some olden day, like 2013 or something.
