1

I have a Debian 11 server that is running several Qemu/KVM virtual machines ( not using libvirtd created purely with Qemu commands ), I've created a network bridge and each VM has its own TAP device connected to the bridge. Please consider that I know avoiding MAC spoofing is easily done in libvirtd via its network filters but I aim to configure it myself for my Qemu-created VMs. My real interface is called eth0, my bridge is called br0, and one of my VMs TAP devices connected to this bridge is called vm0.

What I want to do is that use nftables to drop all OUTPUT packets except for eth0 and br0 in the first place, and then allow OUTPUT packets of my TAP devices only if the source MAC address is the same as the actual MAC address I see in my ip -c a output that I've assigned myself while creating the VM, in other words, I want to avoid MAC spoofing on the VMs via nftables. In addition, I want to block SMTP ports.

I have already created some iptables and ebtables rules to achieve my desired outcome however after learning that nftables is a framework to replace both of them and that achieving such results is more efficient with it, I want to migrate these rules to nftables, however, I can't wrap my head around how to write these rules.

My rules are as follows:

iptables rules, originally learned from this post:

iptables -t filter -A FORWARD -m physdev --physdev-in vm1 --physdev-is-bridged -j 0-out
iptables -t filter -A 0-out -m mac ! --mac-source <SOME_MAC_ADDRESS> -j DROP
iptables -t filter -A 0-out -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m udp --sport 68 --dport 67 -j ACCEPT
iptables -t filter -A 0-out ! -s <SOME_IP_ADDRESS> -j DROP
iptables -t filter -A 0-out -j RETURN

ebtables rules:

# OUTPUT rules

ebtables -A OUTPUT -p IPv4 -o vm0 --ip-protocol tcp --ip-sport 25 -j DROP
ebtables -A OUTPUT -p IPv4 -o vm0 --ip-protocol tcp --ip-sport 587 -j DROP
ebtables -A OUTPUT -p IPv4 -o vm0 --ip-protocol tcp --ip-sport 465 -j DROP
ebtables -A OUTPUT -p IPv4 -o vm0 --ip-protocol udp --ip-sport 25 -j DROP
ebtables -A OUTPUT -p IPv4 -o vm0 --ip-protocol udp --ip-sport 587 -j DROP
ebtables -A OUTPUT -p IPv4 -o vm0 --ip-protocol udp --ip-sport 465 -j DROP

# INPUT rules

ebtables -A INPUT -p IPv4 -i vm0 --ip-protocol tcp --ip-dport 25 -j DROP
ebtables -A INPUT -p IPv4 -i vm0 --ip-protocol tcp --ip-dport 587 -j DROP
ebtables -A INPUT -p IPv4 -i vm0 --ip-protocol tcp --ip-dport 465 -j DROP
ebtables -A INPUT -p IPv4 -i vm0 --ip-protocol udp --ip-dport 25 -j DROP
ebtables -A INPUT -p IPv4 -i vm0 --ip-protocol udp --ip-dport 587 -j DROP
ebtables -A INPUT -p IPv4 -i vm0 --ip-protocol udp --ip-dport 465 -j DROP

I know the combination of using both ebtables and iptables for the basically same reason is a bit unorthodox, this is because I am new to the whole thing and I wrote these rules learning from several sources, however, this is another reason to use nftables to unify them.

Any help is appreciated, either the ruleset syntax itself or hints and guides.

Many thanks in advance.

Skipper
  • 63
  • 9

1 Answers1

0

After much research and documentation reading I've finally found the right approach and answer:

Using ebtables-nft

There is support to use the iptables/ip6tables/arptables/ebtables old syntax with the nf_tables kernel backend.

IP & MAC Spoofing + Port Blocking

My rules for SMTP port blocking work fine, as a matter of fact, my approach to use ebtables for port blocking on virtualization host is correct, however, after research, I found out the MAC/IP spoofing rules written via iptables isn't really a best-practice approach, thanks to this post I found out I should be using ebtables for MAC/IP spoofing too.

Rules

Since I'm using these rules on the virtualization host to avoid SMTP abuse and MAC/IP spoofing, these rules must be applied per VM in order to be dynamic, - again thanks to the post mentioned earlier - we create a custom chain and append it to the FORWARD chain of ebtables.

The rules that I created are as follows:

Creating a custom chain appended to the FORWARD chain per VM for MAC/IP spoofing:

ebtables -A FORWARD -p ip -i $VM_INTERFACE_NAME -j VM-MAC # just a name

Creating a custom chain appended to the FORWARD chain for port blocking

ebtables -A FORWARD -p ip -o $VM_INTERFACE_NAME -j VM-PORT # again just a name
# note that the target chain for these rules are custom chains.

Dropping all the INPUT traffic in the first place

ebtables -P VM-MAC DROP

Now create a rule and appending to the VM-MAC chain to match VM IP and VM's interface MAC address inside the VM.

ebtables -A VM-MAC -p ip --ip-src IP_ADDR_ALLOWED_FOR_VM -s VM_MAC_ADDR -j ACCEPT

Finally, we block desired ports ( in my case, SMTP ) by creating the following rules and appending them to the VM-PORT chain.

ebtables -A VM-PORT -p ip --ip-protocol tcp --ip-sport 25 -j DROP
ebtables -A VM-PORT -p ip --ip-protocol tcp --ip-sport 587 -j DROP
ebtables -A VM-PORT -p ip --ip-protocol tcp --ip-sport 465 -j DROP
ebtables -A VM-PORT -p ip --ip-protocol udp --ip-sport 25 -j DROP
ebtables -A VM-PORT -p ip --ip-protocol udp --ip-sport 587 -j DROP
ebtables -A VM-PORT -p ip --ip-protocol udp --ip-sport 465 -j DROP

Saving the rules.

ebtables-save

Converting to nftables

Now for the converting part, according to the nftables howto wiki

Since June 2018, the old xtables/setsockopt tools are considered legacy. However, there is support to use the iptables/ip6tables/arptables/ebtables old syntax with the nf_tables kernel backend.

Meaning we can simply use the ebtables-nft utility to write nft rulesets through legacy ebtables syntax, therefore we can just re-write the previous commands as below:

ebtables-nft -A FORWARD -p ip -i $VM_INTERFACE_NAME -j VM-MAC
ebtables-nft -A FORWARD -p ip -o $VM_INTERFACE_NAME -j VM-PORT
ebtables-nft -P VM-MAC DROP
ebtables-nft -A VM-MAC -p ip --ip-src IP_ADDR_ALLOWED_FOR_VM -s VM_MAC_ADDR -j ACCEPT
ebtables-nft -A VM-PORT -p ip --ip-protocol tcp --ip-sport 25 -j DROP
ebtables-nft -A VM-PORT -p ip --ip-protocol tcp --ip-sport 587 -j DROP
ebtables-nft -A VM-PORT -p ip --ip-protocol tcp --ip-sport 465 -j DROP
ebtables-nft -A VM-PORT -p ip --ip-protocol udp --ip-sport 25 -j DROP
ebtables-nft -A VM-PORT -p ip --ip-protocol udp --ip-sport 587 -j DROP
ebtables-nft -A VM-PORT -p ip --ip-protocol udp --ip-sport 465 -j DROP
ebtables-nft-save

The above rules will be automatically converted to nft ruleset, you can even verify it via the following command:

nft list ruleset

The output looks like the below:

nft list ruleset
table bridge filter {
      
        chain FORWARD {
                type filter hook forward priority filter; policy accept;
                iifname "<INTERFACE_NAME>" ether type ip counter packets 139 bytes 24158 jump VM-MAC
                oifname "<INTERFACE_NAME>" ether type ip counter packets 279 bytes 23784 jump VM-PORT
        }

        chain VM-MAC {
                ether saddr <MAC_ADDR> ether type ip ip saddr <IP_ADDR>  counter packets 75 bytes 7646 accept
                counter packets 64 bytes 16512 drop
        }

        chain VM-PORT {
                ether type ip tcp sport 25  counter packets 14 bytes 840 drop
                ether type ip tcp sport 587  counter packets 0 bytes 0 drop
                ether type ip tcp sport 465  counter packets 0 bytes 0 drop
                ether type ip udp sport 25  counter packets 0 bytes 0 drop
                ether type ip udp sport 587  counter packets 0 bytes 0 drop
                ether type ip udp sport 465  counter packets 0 bytes 0 drop
                counter packets 265 bytes 22944 accept
        }
}

There you go!

Script

In case anyone was interested I've created a script too:

#!/usr/bin/env bash

# set -x for debugging
set +x
readonly SCRIPT_NAME=$(basename $0)

function help {
    echo "Usage: $0 options"
    echo "  -n <ifname> VM TAP interface name on the host"
    echo "  -m <mac>    VM MAC address inside the virtual machine"
    echo "  -i <id>     VM id"
    echo "  -h <ip>     Allowed IP address for the VM"
}

function err {
    logger -p user.crit -t $SCRIPT_NAME "$@"
}

optstring=":n:m:i:h:"
while getopts $optstring opt
do
    case $opt in
    n)
    VM_IF=$OPTARG
    ;;

    m)
    VM_MAC=$OPTARG
    ;;

    i)
    VM_ID=$OPTARG
    ;;

    h)
    VM_IP=$OPTARG
    ;;
    esac
done

if [[ -z $VM_IF || 
      -z $VM_MAC  ||
      -z $VM_ID ||
      -z $VM_IP ]]
then
    help
    exit 1
fi

ebtables-nft -N $VM_ID-MAC
ebtables-nft -N $VM_ID-PORT
ebtables-nft -A FORWARD -p ip -i $VM_IF -j $VM_ID-MAC
ebtables-nft -A FORWARD -p ip -o $VM_IF -j $VM_ID-PORT
ebtables-nft -P $VM_ID-MAC DROP
ebtables-nft -A $VM_ID-MAC -p ip --ip-src $VM_IP -s $VM_MAC -j ACCEPT
ebtables-nft -A $VM_ID-PORT -p ip --ip-protocol tcp --ip-sport 25 -j DROP
ebtables-nft -A $VM_ID-PORT -p ip --ip-protocol tcp --ip-sport 587 -j DROP
ebtables-nft -A $VM_ID-PORT -p ip --ip-protocol tcp --ip-sport 465 -j DROP
ebtables-nft -A $VM_ID-PORT -p ip --ip-protocol udp --ip-sport 25 -j DROP
ebtables-nft -A $VM_ID-PORT -p ip --ip-protocol udp --ip-sport 587 -j DROP
ebtables-nft -A $VM_ID-PORT -p ip --ip-protocol udp --ip-sport 465 -j DROP
ebtables-nft-save

ret=$?

if [[ $ret -ne 0 ]]
then
    err "Could not create nft ruleset for instance ${VM_ID}"
    exit $ret
fi

echo -e "nft ruleset for instance ${VM_ID} created."
exit $ret

Link to gist

Using nft

We can achieve the same goal via the nft utility itself and add maps to it which will result in much less code and cleaner ruleset, I won't discuss the approach here since I did in the ebtables-nft section:

Since with nft there is not default chain or table present in the ruleset we have to create our table first and use the bridge address family from nftables official wiki:

bridge Tables of this family see traffic/packets traversing bridges (i.e. switching). No assumptions are made about L3 protocols. The ebtables tool is the legacy x_tables equivalent. Some old x_tables modules such as physdev will also eventually be served from the nftables bridge family. Note that there is no nf_conntrack integration for the nftables bridge family.

Therefore we create our table like below:

nft add table bridge filter

Now we create a forward chain using the forward hook:

nft add chain bridge filter forward '{ type filter hook forward priority filter; policy accept ;}'

As you can see there are several keywords, hook, filter, priority, and policy. Each of these must be understood first before creating the rules, as a matter of fact understanding these took much time for me. In order to fully understand what these mean kindly refer to the nftables official wiki since explaining them is out of the scope of this question.

Now we create two chains, one for MAC/IP spoofing and one for SMTP port blocking:

nft add chain bridge filter mac
nft add chain bridge filter ports

As you can see there is no hook, priority, or policy set for these, again refer to the wiki for more information.

Now we implement the nft feature map in order to have a cleaner code:

We create two maps, one for mapping ifname to mac address to ip address finishing with a verdict, and one for mapping ifname to verdict for port blocking:

nft add map bridge filter allowed '{ type ifname . ether_addr . ipv4_addr: verdict ;}'
nft add map bridge filter smpt-stat '{ type ifname: verdict; }'

For more information on maps and verdict maps refer to the official wiki.

Now we add our rules which is kind of similar to the ebtables-nft rules:

nft add rule bridge filter forward iifname "vm*" ether type ip jump macs
nft add rule bridge filter forward oifname "vm*" ether type ip jump ports

# clean rules using maps and vmaps:

nft add rule bridge filter macs ether type ip iifname . ether saddr . ip saddr vmap @allowed
nft add rule bridge filter macs drop
nft add rule bridge filter ports ether type ip tcp sport '{ 25, 587, 465 }' oifname vmap @smtp-stat

All we need to do now is to add elements to our maps:

nft add element bridge filter allowed '{ vm0 . <ALLOWED_MAC_ADDR> . <ALLOWED_IP_ADDR> : accept }'
nft add element bridge filter smtp-stat '{ vm0 : drop }'

Later on if we needed to open the SMTP ports for the vm0 interface we'd only have to replace the drop verdict element in that map.

As you can see the nft list ruleset output is much cleaner than ebtables-nft:

table bridge filter {
        map allowed {
                type ifname . ether_addr . ipv4_addr : verdict
                elements = { "vm0" . <ALLOED_MAC_ADDR> . <ALLOWED_IP_ADDR> : accept }
        }

        map smtp-stat {
                type ifname : verdict
                elements = { "vm0" : drop }
        }

        chain forward {
                type filter hook forward priority filter; policy accept;
                iifname "vm*" ether type ip jump macs
                oifname "vm*" ether type ip jump ports
        }

        chain macs {
                iifname . ether saddr . ip saddr vmap @allowed
                drop
        }

        chain ports {
                ether type ip tcp sport { 25, 465, 587 } oifname vmap @smtp-stat
        }
}

Hope this all is helpful for someone :)

Skipper
  • 63
  • 9