Access to Guest VM via additional public IP

Question

I have a CentOS dedicated server (Hetzner):
IP: aa.bb.cc.16
Gateway: aa.bb.cc.1
Netmask: 255.255.255.255

and additional IP (guest VM (libvirt) with Debian installed):
IP: aa.bb.cc.61
Gateway: aa.bb.cc.1
Netmask: 255.255.255.192

I have setup a routed configuration with Hetzner documentation (https://wiki.hetzner.de/index.php/Netzkonfiguration_CentOS/en#Routed_.28brouter.29).

Host config:
cat /etc/sysctl.conf

net.ipv4.conf.all.rp_filter=1  
net.ipv4.icmp_echo_ignore_broadcasts=1  
net.ipv4.ip_forward=1  
net.ipv6.conf.all.forwarding=1  
net.ipv4.conf.default.proxy_arp=1  
net.ipv4.conf.all.send_redirects=0  
net.ipv4.conf.eth0.send_redirects=0  
net.ipv4.conf.br0.send_redirects=0  
net.ipv4.conf.default.send_redirects=0  
net.bridge.bridge-nf-call-ip6tables = 0  
net.bridge.bridge-nf-call-iptables = 0  
net.bridge.bridge-nf-call-arptables = 0

cat /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
ONBOOT=yes
HWADDR=XX:XX:XX:XX:XX:XX
BOOTPROTO=none
IPADDR=aa.bb.cc.16
NETMASK=255.255.255.255
SCOPE="peer aa.bb.cc.1"
IPV6INIT=yes
IPV6ADDR=2a01:4f8:yyy:yyyy::2/128
IPV6_DEFAULTGW=fe80::1
IPV6_DEFAULTDEV=eth0

cat /etc/sysconfig/network-scripts/ifcfg-br0

DEVICE=br0
ONBOOT=yes
TYPE="Bridge"
BOOTPROTO=static
IPADDR=aa.bb.cc.16
NETMASK=255.255.255.255
IPV6INIT=yes
IPV6ADDR=2a01:4f8:yyy:yyyy::2/64
STP=off
DELAY=0

cat /etc/sysconfig/network-scripts/route-br0

ADDRESS0=aa.bb.cc.61
NETMASK0=255.255.255.255

Guest config:
cat /etc/network/interfaces

auto lo
iface lo inet loopback

auto eth0
    iface eth0 inet static
    address aa.bb.cc.61
    netmask 255.255.255.255
    pointopoint aa.bb.cc.16
    gateway aa.bb.cc.16
    dns-nameservers 212.133.98.98 212.133.99.99 212.133.100.100

iface eth0 inet6 static
    address 2a01:4f8:yyy:yyyy::4
    netmask 64
    address 2a01:4f8:yyy:yyyy::2

For internet access from guest VM to internet I have added a rule:

iptables -t nat -A POSTROUTING -o eth0 -s aa.bb.cc.61 -j SNAT --to-source aa.bb.cc.16

Question: It is possible to access from internet to guest VM via guest IP aa.bb.cc.61 in the routed mode? When I pinging from outside (internet) to aa.bb.cc.61 I get 'Request timed out'. But aa.bb.cc.16 (host) pinging ok. What rule I have to add?

Thank you.

Are you attempting to access this VM from the internet directly? — Spooler, Oct 08 '16 at 13:38
You have to create a network bridge on the host. Search for a guide. — Broco, Oct 08 '16 at 14:20
@SmallLoanOf1M Yes. I would like to install an http server (nginx, for example) and access via http://aaa.bb.cc.61 (by VM ip). — Alexander Dyuzhev, Oct 08 '16 at 15:51
Hetzner's network is kind of ridiculous and setting up VM networking that way is just not going to be reliable. Use the bridged method if you can. — Michael Hampton, Oct 08 '16 at 18:05
@AlexanderDyuzhev yes, the router mode is like the NAT mode on virtualbox. I'm not saying it can't be done with a lot of tweaking, but according to the KVM documentation it's not possible, also it affects the performance pretty badly. Try bridging, it's really not that hard to configure and also pretty straightforward. — Broco, Oct 11 '16 at 07:01
Please add the solution as an answer, not into the question. — Tero Kilkanen, Oct 12 '16 at 15:08

score 0 · Answer 1 · answered Oct 08 '16 at 16:26

First, a few things need to be made clear.

The guide that you're using refers to a "router mode" and "bridged mode".

The router mode described is almost entirely pointless. It involves creating an empty "host only" bridge, connecting your VM to that, and then routing IP level traffic from that bridge to your outbound interface. This ignores the fact that libvirt can create its own NAT virtual interface (with optional routing, DHCP, and DNS) and connect VM interfaces to that. In fact, libvirt has this network created and enabled by default. It's referred to as the "default" network.

In your case, you have two subnets (which I'm assuming are on separate layer 2 domains - as would be the case in nearly all situations). One address will be used to access your hypervisor "aa.bb.cc.16/32" (This is an invalid subnet - more on that later) This address can be placed directly on an interface.

And then you have a second IP on another subnet (aa.bb.cc.61/26). This address needs to go on the VM interface within the guest. This can be done with a bridge directly, and while the guide you posted originally offers bad solutions for the "routed mode", a proposed configuration for a directly used bridge is actually the correct way to do things. However, it also offers an entirely incorrect configuration.

The following network configuration will work for said bridge, provided you update values to reflect your scenario:

# /etc/sysconfig/network-scripts/ifcfg-eth0
# device: eth0
DEVICE=eth0
BOOTPROTO=static
ONBOOT=yes
BRIDGE=br0

# /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE=br0
TYPE="Bridge"
BOOTPROTO=static
ONBOOT=yes

Once this is done, use this bridge as a device to connect your VM NIC to. If need be, you may put an IP address on the bridge itself, allowing communication between the host and guest. However, that would be on your subnet that you're suing for public access. If you do need host to guest communication, I would recommend a private layer 2 domain either via the "default" NAT interface, or via an empty bridge with a private IP space on both the host and guest (which would create two NICs total on the VM).

As a side note, the netmask that you've posted for your hypervisor IP (255.255.255.255) is 32 bits. That will not allow a single address on its network, and I'm assuming it is merely a typo. If it isn't a typo, you need to use an address space that will actually allow a host, even if it's only a 31 bit netmask.

That covers things in a general sense. Hopefully, that's enough information for you to make a decision and configure things properly.

Thank you for your comment. It very helpful. I will add full solution above. — Alexander Dyuzhev, Oct 11 '16 at 16:37

score 0 · Accepted Answer · answered Oct 12 '16 at 16:50

Here is working solution:

cat /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
ONBOOT=yes
HWADDR=XX:XX:XX:XX:XX:XX
BOOTPROTO=none
IPADDR=aa.bb.cc.16
NETMASK=255.255.255.255
SCOPE="peer aa.bb.cc.1"
BRIDGE=br0

cat /etc/sysconfig/network-scripts/ifcfg-br0

DEVICE=br0
TYPE=Bridge
BOOTPROTO=static
IPADDR=aa.bb.cc.16
NETMASK=255.255.255.255
SCOPE="peer  aa.bb.cc.1"
ONBOOT=yes
DELAY=0

mv /etc/sysconfig/network-scripts/route-eth0 /etc/sysconfig/network-scripts/route-br0 cat cat /etc/sysconfig/network-scripts/route-br0

### Hetzner Online GmbH - installimage
# routing for eth0
ADDRESS0=0.0.0.0
NETMASK0=0.0.0.0
GATEWAY0=aa.bb.cc.1

Guest config:
cat /etc/network/interfaces

auto lo
iface lo inet loopback

auto eth0
    iface eth0 inet static
    address aa.bb.cc.61
    netmask 255.255.255.255
    pointopoint aa.bb.cc.16
    gateway aa.bb.cc.16
    dns-nameservers 212.133.98.98 212.133.99.99 212.133.100.100
    hwaddress ether 00:01:02:ab:cd:de # MAC Address for additional IP

Access to Guest VM via additional public IP

2 Answers2