the iptables chain DOCKER-USER does not seems to work

Question

I use ipset to block IP ranges from many countries on my server (almost the entire world at the exception of France and french speaking countries).

I have two rulesets in ipset : badbadworld1 and badbadworld2

My host is running docker : one Nginx docker, and other dockers for differents services.

# docker -v
Docker version 20.10.17, build 100c701

I want to add my ipsets to netfilter (on my host) in order to access Nginx (and by consequence, other services behind it) only from countries i have choosen.

When i add DROP rules to DOCKER-USER chain, i dont see any packet dropped.

When i add the same rules to DOCKER chain, i see dropped packets.

I both case, i can test by using an online proxy from blocked countries.

I can add DROP rules to both chains, from the documentation DOCKER-USER is evaluated before DOCKER, so in this case, packets must be dropped by DOCKER-USER and not by DOCKER chain, right ?

I test this with :

iptables -I DOCKER-USER -m set --match-set badbadworld2 src -j DROP
iptables -I DOCKER-USER -m set --match-set badbadworld1 src -j DROP
iptables -I DOCKER -m set --match-set badbadworld2 src -j DROP
iptables -I DOCKER -m set --match-set badbadworld1 src -j DROP

Here is my iptables -v -n -L after a few minutes :

[...]

    Chain FORWARD (policy DROP 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination         
     1596  253K DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
     1188  162K ACCEPT     all  --  *      br-a4cbc882767a  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
       56  3324 DOCKER     all  --  *      br-a4cbc882767a  0.0.0.0/0            0.0.0.0/0           
      352 87455 ACCEPT     all  --  br-a4cbc882767a !br-a4cbc882767a  0.0.0.0/0            0.0.0.0/0           
       43  2580 ACCEPT     all  --  br-a4cbc882767a br-a4cbc882767a  0.0.0.0/0            0.0.0.0/0           
        0     0 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
        0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
        0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
        0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
        0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited
        0     0 DROP       all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0 
    
    [...]
    
    Chain DOCKER (2 references)
     pkts bytes target     prot opt in     out     source               destination         
       10   584 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set badbadworld1 src
        1    40 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set badbadworld2 src
        2   120 ACCEPT     tcp  --  !br-a4cbc882767a br-a4cbc882767a  0.0.0.0/0            172.18.0.6           tcp dpt:443
        0     0 ACCEPT     tcp  --  !br-a4cbc882767a br-a4cbc882767a  0.0.0.0/0            172.18.0.6           tcp dpt:80
    
    [...]
    
    Chain DOCKER-USER (1 references)
     pkts bytes target     prot opt in     out     source               destination         
        0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set badbadworld1 src
        0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set badbadworld2 src
        0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

[...]

You can see, for the same rules, 11 packets dropped in DOCKER chain, and 0 in DOCKER-USER.

Note : I have first tested to add only to DOCKER-USER chain, but i was suprised to not see any dropped packet (to be sure, i have tried to connect with an online proxy from a blocked country, and i can connect), adding rules to both chain is useless but it is only to demonstrate the problem.

Thanks !

gapsf · Accepted Answer · 2022-08-17T15:04:07.927

0

All packets already accepted or dropped before jump to DOCKER-USER chain.
Packet checks goes sequentially from first rule in the chain until some rule matched. If target of matched rule is final, like ACCEPT, DROP, REJECT, packet will be accepted or dropped and checks don't goes further against next rules. You have several ACCEPT and DROP matched against all of your packets before DOCKER-USER chain. That's why counters of -j DOCKER-USER rule are zero.

    0     0 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Matched rule with ACCEPT, DROP, REJECT target is final rule - it stops futher checks.
In your case all packets "eated" by previous matched ACCEPT and DROP rules in FORWARD chain, and never reach your DOCKER-USER chain.

If all rules in FORWARD are generated automatically and your can't change their order it's likely a docker's bug or documentation error.

Otherwise you should change FORWARD rules order so jump to DOCKER-USER goes before ACCEPT ESTABLISHED rule (#2 rule) or insert your DROP rules directly into FORWARD before ESTABLISHED.

If DROP goes after ESTABLISHED rule, DROP will not work immediately for already established connections until they closed explicitly or by timeout.

Also you should review in/out interfaces of rules since it's not obvious what interfaces you have and for what interfaces you want to drop packets.

edited Aug 17 '22 at 15:04

answered Aug 17 '22 at 09:25

gapsf

641
1
5
12

Thanks for your answer. I see dropped packets in DOCKER (by my DROP rules). But following documentation, DOCKER-USER must be evaluated first (see https://docs.docker.com/network/iptables/), in the FORWARD chain, we see that packets are send to DOCKER-USER before DOCKER, my example was just to demonstrate that is not the behaviour i see. Because DOCKER-USER is before DOCKER, i expect to see dropped packets in this chain (and not in DOCKER). And even if i put DROP rules only in DOCKER-USER chain, packet are not DROPPED at all. – Endoril Aug 17 '22 at 12:55
As you can see **in fact DOCKER goes first** and **DOCKER-USER checks goes after DOCKER** chain rules. It's a bug in rules organization or in documentation. – gapsf Aug 17 '22 at 12:59
Oh i see i suspect the 3rd rule of FORWARD to send send my packets to DOCKER (who "eat" them) before DOCKER-USER. It seems is not a correct behaviour if we trust the documentation. Thanks ! – Endoril Aug 17 '22 at 13:00
What if you remove your DROP rules from DOCKER chain end add them to DOCKER-USER? – gapsf Aug 17 '22 at 13:03
Also it is possible that 4 and 5 ACCEPT rules also match against all packets and DOCKER-USER never reached. Are they automatically generated? It's not obvious what interfaces you have. At list DOCKER and DOCKER-USER chains order doesn't match documentation – gapsf Aug 17 '22 at 13:08
As stated, in the post, if i put DROPS only in DOCKER-USER, nothing is dropped :( – Endoril Aug 17 '22 at 19:36
I have not touched interfaces or iptables rules (except the one with badbadworld). All interfaces are legacy or added by docker (the br ones). – Endoril Aug 17 '22 at 19:37

the iptables chain DOCKER-USER does not seems to work

1 Answers1