Why do I lose network connectivity for a virtual machine outside /24?

Question

I have a bunch of virtual machines (Ubuntu Server 14.04) hosted on a single real server (Ubuntu Server 14.04 with KVM). The number of virtual machines grow, and I'll soon have more than 254 of them.

To test,

1. I changed in /etc/network/interfaces of the real server the netmask from 255.255.255.0 to 255.255.254.0. The IP address of the real server is 192.168.1.30.

2. I did the similar operation with virsh net-edit default, which changed the netmask of virbr0 to 255.255.254.0.

3. I changed the netmask of the switch to 255.255.254.0 as well.

4. I changed the configuration of the test virtual machine like this:

   auto eth0
   iface eth0 inet static
   address 192.168.2.35
   netmask 255.255.254.0
   network 192.168.1.0
   broadcast 192.168.2.255
   gateway 192.168.1.1
   dns-nameservers 192.168.1.13
   dns-search example.com
   

This is the current state:

1. I can't ping 192.168.2.35 from the real server: it is just stuck at "PING 192.168.2.35 (192.168.2.35) 56(84) bytes of data." When stopped (Ctrl+C), it complains: "143 packets transmitted, 0 received, 100% packet loss, time 142134ms".

2. The virtual machine 192.168.2.35 can't ping anything (neither the switch, nor the real server, nor google.com), throwing "connect: Network is unreachable" error for the first two, or "ping: unknown host google.com" for the last one.

3. The virtual machine can ping itself.

What's wrong with my configuration?

Answer 1

I still don't understand why /22 instead of /23

Because you have calculated your subnets wrong, and you are trying to use IPs from two different subnets.

Or to put it a different way, if you use a /23 bit mask your starting address for the range, must be the first address for that range. 192.168.1.1 is not a starting address for any network described by a 23 bit mask, because that is just how the math works out.

With a 23 bit mask the valid networks are the following.

$ ipcalc 192.168.0.0/23
Address:   192.168.0.0          11000000.10101000.0000000 0.00000000
Netmask:   255.255.254.0 = 23   11111111.11111111.1111111 0.00000000
Wildcard:  0.0.1.255            00000000.00000000.0000000 1.11111111
=>
Network:   192.168.0.0/23       11000000.10101000.0000000 0.00000000
HostMin:   192.168.0.1          11000000.10101000.0000000 0.00000001
HostMax:   192.168.1.254        11000000.10101000.0000000 1.11111110
Broadcast: 192.168.1.255        11000000.10101000.0000000 1.11111111
Hosts/Net: 510

$ ipcalc 192.168.2.0/23
Address:   192.168.2.0          11000000.10101000.0000001 0.00000000
Netmask:   255.255.254.0 = 23   11111111.11111111.1111111 0.00000000
Wildcard:  0.0.1.255            00000000.00000000.0000000 1.11111111
=>
Network:   192.168.2.0/23       11000000.10101000.0000001 0.00000000
HostMin:   192.168.2.1          11000000.10101000.0000001 0.00000001
HostMax:   192.168.3.254        11000000.10101000.0000001 1.11111110
Broadcast: 192.168.3.255        11000000.10101000.0000001 1.11111111
Hosts/Net: 510

In your interface configuration though you seem to have address 192.168.2.35, and gateway 192.168.1.1. Given that mask your gateway and address are on completely separate subnets. Of course your broadcast and network address settings are also incorrect.

If you use a 22 bit mask you will get a range that includes 192.168.1.x and 192.168.2.x. Be warned that your broadcast, and network values are still wrong, they also need to be fixed.

$ ipcalc 192.168.2.0/22
Address:   192.168.2.0          11000000.10101000.000000 10.00000000
Netmask:   255.255.252.0 = 22   11111111.11111111.111111 00.00000000
Wildcard:  0.0.3.255            00000000.00000000.000000 11.11111111
=>
Network:   192.168.0.0/22       11000000.10101000.000000 00.00000000
HostMin:   192.168.0.1          11000000.10101000.000000 00.00000001
HostMax:   192.168.3.254        11000000.10101000.000000 11.11111110
Broadcast: 192.168.3.255        11000000.10101000.000000 11.11111111
Hosts/Net: 1022