How to protect a web application from IPv6 bots?

Question

I'm interested in this in the context of protecting a web application from bots, but I guess it applies to all kind of attacks that can be done over IPv6 by bots.

In a web application, you have some pages you want to protect from bots.

It could be a login page: you want to avoid million of requests trying username/password combinations.

It could be a registration page. You don't want thousands of bot accounts created on your website.

Or maybe you don't want the whole content of your database to be downloaded at once. You think it's ok for an anonymous user to make 100 or 200 requests a day to your database to get some infos, but you're not ok with bots downloading all the content you're offering by doing 1000 requests a minute.

Or just for statistical purpose. You're logging visitors activity on your site and you don't want bots to completely bias your data, by say, artificially clicking a link thousands of time so that the article it's linking to becomes the "most popular article" on your news website.

I've often handled this using throttling/rate limiting based on IP addresses, using Nginx rate limiting capabilities:

limit_req_zone $binary_remote_addr zone=mylimit:10m rate=10r/m;    

server {
    location /sensitiveUrl {
        limit_req zone=mylimit;

        proxy_pass http://my_upstream;
    }
}

But with IPv6, I'm not sure how effective this is. I mean, with IPv4, it's relatively effective because it's costly for attackers to create and use thousands of ip addresses, but it seems that with IPv6 anyone can create millions of IPs. How to handle this? Should I apply restrictions on IPv6 networks instead of addresses? How effective would that be in practice if I did that?

score 5 · Accepted Answer · answered Jul 03 '18 at 14:49

The flexibility of IPv6 with addressing is great, but it does indeed make things like this harder. An algorithm that I would recommend:

Start by blocking separate IPv6 addresses (a /128). It might be a single user on a network with multiple users, and you want to avoid blocking innocent users (which happens with IPv4 all the time because of NAT, let's not repeat that)
If you have more than x blocks inside the same /64, assume that the whole /64 is tainted and block all of it. You can now remove the individual /128 records from your blacklist because they are now covered by the /64. This prevents your blacklisting system from overflowing memory/storage.
It might be that an attacker has more than a single /64. The default size is a /48, but /56 and even /60 (which is way too little for IPv6, but some ISPs never learn) occur. I would scale up per 4 bits: if multiple /64s from the same /60 are blocked, scale up to block the whole /60. Same with multiple /60 in a /56 etc.
I would also recommend using different blacklist timeouts for different prefix lengths. It's easier for a single user to accidentally block a /64 than it is for someone to "accidentally" get a whole /48 blocked. Larger blocks deserve a longer time on the blacklist IMHO.
This algorithm can be abused by an attacker spreading their attack over a whole /48 from the start, thereby not quickly triggering the scaling algorithm. You might therefore want several conditions in parallel for scaling up quickly.

An example of such a scaling mechanism could be:

+---------------+----------------------------------------+-----------+
|               |   Block when containing >= of these:   | Blacklist |
| Prefix length |  /128  |  /64  |  /60  |  /56  |  /52  |   time    |
+---------------+--------+-------+-------+-------+-------+-----------+
|     /128      |   N/A  |  N/A  |  N/A  |  N/A  |  N/A  |    5 min  |
|      /64      |     5  |  N/A  |  N/A  |  N/A  |  N/A  |   15 min  |
|      /60      |    15  |    2  |  N/A  |  N/A  |  N/A  |   30 min  |
|      /56      |    50  |    4  |    2  |  N/A  |  N/A  |   60 min  |
|      /52      |    75  |    8  |    4  |    2  |  N/A  |  120 min  |
|      /48      |   100  |   16  |    8  |    4  |    2  |  240 min  |
+---------------+--------+-------+-------+-------+-------+-----------+

Thanks, that looks like a good strategy. Do you implement this on the web application level or does Nginx or another web server can be configured to act like this? — John Smith Optional, Jul 04 '18 at 13:06
As far as I know no web server has something like this built in, so you'd have to implement it in your own app. It would be good to have a library to make this easier, but I haven't gotten around to writing it yet. — Sander Steffann, Jul 05 '18 at 14:20

score 2 · Answer 2 · answered Jun 01 '22 at 01:10

You can do some really primitive IP subnetting via regex. Here, I am passing IPv4 addresses through unchanged but grabbing the first half of IPv6 addresses, which corresponds to the /64 prefix of the address. As a result nginx will track rate limits across the entire /64 at once.

map $binary_remote_addr $masked_ip_addr {
        # Extract the first half (the /64 prefix) of an ipv6 address
        "~^(?P<a>........)........$" "$a";
        # Extract the entire ipv4 address
        "~^(?P<a>....)$" "$a";
}

limit_req_zone $masked_ip_addr zone=myzone:10m rate=1r/s;

server {
        limit_req zone=myzone;
}

How to protect a web application from IPv6 bots?

2 Answers2