While trying to setup a small server with LXC containers, I encountered a problem:
I want to assign 4 different IP addresses: 1 address for the host and 3 addresses in a different net for the containers.
Setting up 4 IP addresses using virtual interfaces (eth0:1, eth0:2, etc) is no problem at all - but LXC doesn't work with those virtual interfaces.
The host's IP address is 2.2.2.2 (net mask 255.255.252.0). The 3 container IP addresses are 33.33.33.33, 33.33.33.44 und 33.33.33.55. Net mask for all of them: 255.255.255.255.
I'm struggling while setting up the network for hosts and containers. Other posts explain similar settings, with different, but controllable subnets (bridging LXC containers to host network with different IP range) or suggest using virtual interfaces (Several IP address within the same subnet on the same host) which we cannot do here.
This is my configuration:
Host: interfaces file
source /etc/network/interfaces.d/*
auto lo
iface lo inet loopback
auto br0
iface br0 inet static
address 2.2.2.2
netmask 255.255.252.0
broadcast 2.2.2.255
gateway 2.2.2.1
bridge_ports eth0
bridge_fd 0
bridge_maxwait 0
auto br0:1
iface br0:1 inet static
address 33.33.33.33
netmask 255.255.255.255
auto br0:2
iface br0:2 inet static
address 33.33.33.44
netmask 255.255.255.255
auto br0:3
iface br0:3 inet static
address 33.33.33.55
netmask 255.255.255.255
This is the only way I know to bind several IP addresses to an interface. The bridge needs to listen to the container's IP addresses to retrieve packets, doesn't it?
Container: interfaces file
Small file, since all parameters are kept in the LXC configuration.
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
LXC: network configuration
This is the network part of my LXC configuration file:
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.name = eth0
lxc.network.ipv4 = 33.33.33.33/32
lxc.network.ipv4.gateway = 2.2.2.2
However, the connection just doesn't work. An address is assigned to the container, but IP packets sent from the container do not reach the host.
Here is the output of ip
and route
:
root@container:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 2.2.2.2 0.0.0.0 UG 0 0 0 eth0
2.2.2.2 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
root@container:~# ip
1: lo: <LOOPBACK,UP,LOWER_UPmtu 65536 qdisc noqueue state UNKNOWN group
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:
inet 127.0.0.1/8 scope host
valid_lft forever preferred_lft
inet6 ::1/128 scope
valid_lft forever preferred_lft
15: eth0: <BROADCAST,MULTICAST,UP,LOWER_UPmtu 1500 qdisc pfifo_fast state UP
group default qlen 1000
link/ether ae:af:7c:e4:b5:4d brd ff:ff:ff:ff:ff:
inet 33.33.33.33/32 brd 255.255.255.255 scope global
valid_lft forever preferred_lft
inet6 fe80::acaf:7cff:fee4:b54d/64 scope
valid_lft forever preferred_lft forever
And the same details given by the host:
root@host:~# route -n
Routing Table
Destination Router Genmask Flags Metric Ref Use Iface
2.2.2.0 0.0.0.0 255.255.252.0 U 0 0 0 br0
0.0.0.0 2.2.2.1 0.0.0.0 UG 0 0 0 br0
root@host:~# ip
1: lo: <LOOPBACK,UP,LOWER_UPmtu 65536 qdisc noqueue state UNKNOWN group
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:
inet 127.0.0.1/8 scope host
valid_lft forever preferred_lft
inet6 ::1/128 scope
valid_lft forever preferred_lft
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UPmtu 1500 qdisc pfifo_fast master br0
state UP group default qlen 1000
link/ether 2e:cf:22:12:dd:e2 brd ff:ff:ff:ff:ff:
16: veth93SMLW: <BROADCAST,MULTICAST,UP,LOWER_UPmtu 1500 qdisc pfifo_fast state
UP group default qlen 1000
link/ether dd:c1:1e:68:90:47 brd ff:ff:ff:ff:ff:
inet6 fe80::fcc1:1eff:fe68:9047/64 scope
valid_lft forever preferred_lft
17: br0: <BROADCAST,MULTICAST,UP,LOWER_UPmtu 1500 qdisc noqueue state UP group
default
link/ether 4d:cf:22:12:dd:e2 brd ff:ff:ff:ff:ff:
inet 2.2.2.2/22 brd 1.1.1.255 scope global
valid_lft forever preferred_lft
inet 33.33.33.33/32 brd 2.2.2.2 scope global br0:
valid_lft forever preferred_lft
inet 33.33.33.44/32 brd 3.3.3.3 scope global br0:
valid_lft forever preferred_lft
inet 33.33.33.55/32 brd 4.4.4.4 scope global br0:
valid_lft forever preferred_lft
inet6 fe80::829:caff:fece:bbd7/64 scope
valid_lft forever preferred_lft forever
Have you got hints which part of the configuration I messed up?
Thanks for your help!