Why do ICMP Redirect Host happen?

Question

I'm setting up a Debian box as a router for 4 subnets. For that I have defined 4 virtual interfaces on the NIC where the LAN is connected (eth1).

eth1      Link encap:Ethernet  HWaddr 94:0c:6d:82:0d:98  
          inet addr:10.1.1.1  Bcast:10.1.1.255  Mask:255.255.255.0
          inet6 addr: fe80::960c:6dff:fe82:d98/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6026521 errors:0 dropped:0 overruns:0 frame:0
          TX packets:35331299 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:673201397 (642.0 MiB)  TX bytes:177276932 (169.0 MiB)
          Interrupt:19 Base address:0x6000 

eth1:0    Link encap:Ethernet  HWaddr 94:0c:6d:82:0d:98  
          inet addr:10.1.2.1  Bcast:10.1.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:19 Base address:0x6000 

eth1:1    Link encap:Ethernet  HWaddr 94:0c:6d:82:0d:98  
          inet addr:10.1.3.1  Bcast:10.1.3.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:19 Base address:0x6000 

eth1:2    Link encap:Ethernet  HWaddr 94:0c:6d:82:0d:98  
          inet addr:10.1.4.1  Bcast:10.1.4.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:19 Base address:0x6000 

eth2      Link encap:Ethernet  HWaddr 6c:f0:49:a4:47:38  
          inet addr:192.168.1.10  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::6ef0:49ff:fea4:4738/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:199809345 errors:0 dropped:0 overruns:0 frame:0
          TX packets:158362936 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:1000 
          RX bytes:3656983762 (3.4 GiB)  TX bytes:1715848473 (1.5 GiB)
          Interrupt:27 

eth3      Link encap:Ethernet  HWaddr 94:0c:6d:82:c8:72  
          inet addr:192.168.2.5  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::960c:6dff:fe82:c872/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:110814 errors:0 dropped:0 overruns:0 frame:0
          TX packets:73386 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:16044901 (15.3 MiB)  TX bytes:42125647 (40.1 MiB)
          Interrupt:20 Base address:0x2000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:22351 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22351 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:2625143 (2.5 MiB)  TX bytes:2625143 (2.5 MiB)

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.8.0.1  P-t-P:10.8.0.2  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:41358924 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23116350 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:3065505744 (2.8 GiB)  TX bytes:1324358330 (1.2 GiB)

I have two other computers connected to this network. One has IP 10.1.1.12 (subnet mask 255.255.255.0) and the other one 10.1.2.20 (subnet mask 255.255.255.0). I want to be able to reach 10.1.1.12 from 10.1.2.20.

Since packet forwarding is enabled in the router and the policy of the FORWARD chain is ACCEPT (and there are no other rules), I understand that there should be no problem to ping from 10.1.2.20 to 10.1.1.12 going through the router.

However, this is what I get:

$ ping -c15 10.1.1.12
PING 10.1.1.12 (10.1.1.12): 56 data bytes
Request timeout for icmp_seq 0
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 81d4   0 0000  3f  01 e2b3 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 1
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 899b   0 0000  3f  01 daec 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 2
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 78fe   0 0000  3f  01 eb89 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 3
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 14b8   0 0000  3f  01 4fd0 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 4
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 8ef7   0 0000  3f  01 d590 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 5
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 ec9d   0 0000  3f  01 77ea 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 6
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 70e6   0 0000  3f  01 f3a1 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 7
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 b0d2   0 0000  3f  01 b3b5 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 8
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 f8b4   0 0000  3f  01 6bd3 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 9
Request timeout for icmp_seq 10
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 1c95   0 0000  3f  01 47f3 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 11
Request timeout for icmp_seq 12
Request timeout for icmp_seq 13
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 62bc   0 0000  3f  01 01cc 10.1.2.20  10.1.1.12

Why does this happen?

From what I've read the Redirect Host response has something to do with the fact that the two hosts are in the same network and there being a shorter route (or so I understood). They are in fact in the same physical network, but why would there be a better route if they are not on the same subnet (they can't see each other)?

What am I missing?

Some extra info you might want to see:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.8.0.2        0.0.0.0         255.255.255.255 UH    0      0        0 tun0
127.0.0.1       0.0.0.0         255.255.255.255 UH    0      0        0 lo
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth3
10.8.0.0        10.8.0.2        255.255.255.0   UG    0      0        0 tun0
192.168.1.0     0.0.0.0         255.255.255.0   U     1      0        0 eth2
10.1.4.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
10.1.1.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
10.1.2.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
10.1.3.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth2
0.0.0.0         192.168.2.1     0.0.0.0         UG    100    0        0 eth3

# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination   

# iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  -- !10.0.0.0/8           10.0.0.0/8          
MASQUERADE  all  --  10.0.0.0/8          !10.0.0.0/8          

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

score 23 · Accepted Answer · edited Oct 07 '21 at 06:47

At first blush, it looks like Debian is stretching the boundaries for sending an ICMP redirect; quoting RFC 792 (Internet Protocol).

  The gateway sends a redirect message to a host in the following

  situation.  A gateway, G1, receives an internet datagram from a
  host on a network to which the gateway is attached.  The gateway,
  G1, checks its routing table and obtains the address of the next
  gateway, G2, on the route to the datagram's internet destination
  network, X.  If G2 and the host identified by the internet source
  address of the datagram are on the same network, a redirect
  message is sent to the host.  The redirect message advises the
  host to send its traffic for network X directly to gateway G2 as
  this is a shorter path to the destination.  The gateway forwards
  the original datagram's data to its internet destination.

In this case, G1 is 10.1.2.1 (eth1:0 above), X is 10.1.1.0/24 and G2 is 10.1.1.12, and the source is 10.1.2.20 (i.e. G2 and the host identified by the internet source address of the datagram are **NOT** on the same network). Maybe this has been historically interpreted differently in the case of interface aliases (or secondary addresses) on the same interface, but strictly speaking I'm not sure we should see Debian send that redirect.

Depending on your requirements, you might be able to solve this by making the subnet for eth1 something like 10.1.0.0/22 (host addresses from 10.1.0.1 - 10.1.3.254) instead of using interface aliases for individual /24 blocks (eth1, eth1:0, eth1:1, eth1:2); if you did this, you'll need to change the netmask of all hosts attached and you wouldn't be able to use 10.1.4.x unless you expanded to a /21.

EDIT

We're venturing a bit outside the scope of the original question, but I'll help work through the design/security issues mentioned in your comment.

If you want to isolate users in your office from each other, let's step back for a second and look at some security issues with what you have now:

You currently have four subnets in one ethernet broadcast domain. All users in one broadcast domain doesn't meet the security requirements you articulated in the comments (all machines will see broadcasts from other machines and could spontaneously send traffic to each other at Layer2, regardless of their default gateway being eth1, eth1:0, eth1:1 or eth1:2). There is nothing your Debian firewall can do to change this (or maybe I should say there is nothing your Debian firewall should do to change this :-).

You need to assign users into Vlans, based on security policy stated in the comments. A properly-configured Vlan will go a long way to fixing the issues mentioned above. If your ethernet switch doesn't support Vlans, you should get one that does.
With respect to multiple security domains accessing 10.1.1.12, you have a couple of options:
- Option 1: Given the requirement for all users to access services on 10.1.1.12, you could put all users in one IP subnet and implement security policies with Private Vlans (RFC 5517), assuming your ethernet switch supports this. This option will not require iptables rules to limit intra-office traffic from crossing security boundaries (that is accomplished with private Vlans).
- Option 2: You could put users into different subnets (corresponding to Vlans) and implement iptables rules to deploy your security policies
After you have secured your network at the Vlan level, set up source-based routing policies to send different users out your multiple uplinks.

FYI, if you have a router that supports VRFs, some of this gets even easier; IIRC, you have a Cisco IOS machine onsite. Depending on the model and software image you already have, that Cisco could do a fantastic job isolating your users from each other and implement source-based routing policies.

Basically what I need is to have 4 subnets for different areas of an office. Some subnets will go out to the internet using one ISP and others will use a different one. Machines from different subnets shouldn't be able to see or connect with each other. EXCEPT for host 10.1.1.12 which offers some services which should be available for all. Currently I haven't setup the appropriate FORWARD rules for this. However, since all forwards are accepted I thought I should be able to ping from 10.1.2.20 to 10.1.1.12. — El Barto, Jun 25 '12 at 15:37
Hmm... ok, thanks Mike. I'll look into VLANs more deeply. I had thought about it before starting all this, and thought I wouldn't need it. The switches we have do support VLANs, though they are unmanaged switches so, if I'm not wrong, I guess I will have to do the tagging on the Debian router, right? The isolation of subnets it's actually not a critical issue in this office, but it's something I think it would be good to have if it doesn't require too much extra work. I'll look into it and see what I can do :) — El Barto, Jun 25 '12 at 18:28
@ElBarto, if your switches don't support Vlan tagging (and that's unlikely if they are unmanaged), then only tagging on Debian wouldn't help. If intra-office subnet isolation isn't a critical issue, then put everyone in two different subnets and make things easy (two subnets ensures you can policy route on the Debian). I would say that the current scheme with four Debian interface aliases offers no real subnet isolation, and it adds much more complication. — Mike Pennington, Jun 25 '12 at 18:38
That's right, from what I understand from the user manual the switches support "keeping the tag" but not "doing the actual tagging". Thanks for the clarification regarding Debian. The thing is that even if I keep two subnets, I'll still need machines from the subnet 10.1.2.0/24 to access to 10.1.1.12. — El Barto, Jun 25 '12 at 18:46
The machines in a different subnet should still be able to access `10.1.1.12`. If you block linux from sending ICMP unreachables with `iptables`, then you will still burn CPU from it sending the ICMP messages, but at least they won't get installed in your host tables. That said, if you add another ethernet interface on the Debian (i.e. dedicate one interface per user 'class'), Debian *should* not send ICMP unreachables anymore; that would imply that you have two different ethernet switches: one for each user 'class'. Your cabling technicians would dislike it, but it gets the job done — Mike Pennington, Jun 25 '12 at 18:50
You mean doing something like this? `iptables -A OUTPUT -o eth1 -p icmp --icmp-type redirect -j DROP` — El Barto, Jun 25 '12 at 18:56
Exactly, however, I haven't got time to validate your syntax until I get home tonight — Mike Pennington, Jun 25 '12 at 19:27
Sorry for the delay, I was able to test this just now and adding that iptables rule worked like a charm. Thanks again, Mike. — El Barto, Jul 10 '12 at 19:18

score 3 · Answer 2 · answered Jun 25 '12 at 14:33

3

It is not really clear what you are trying to do, but I can say the following.

These subnets are connected to the same physical interface. The Linux router will return ICMP redirect message when the received packet should be forwarded over the same physical interface.

answered Jun 25 '12 at 14:33

Khaled

35,688
8
69
98

I need to handle these 4 subnets which are all connected through the same NIC. The idea is that hosts from the different subnets shouldn't be able to connect to each other, except for host 10.1.1.12 which should be available for all. I haven't defined the forward rules for this yet, so I thought any host from any of those subnets should be able to reach 10.1.1.12. Is there any way to avoid the ICMP redirect? – El Barto Jun 25 '12 at 15:32
1

@ElBarto, one method is to add an `iptables` rule that drops redirects going out `eth1` – Mike Pennington Jun 25 '12 at 15:39

score 1 · Answer 3 · edited Apr 29 '16 at 11:20

I agree with Khaled's comments and would also add to end of his phrase:

"These subnets are connected to the same physical interface. The Linux router will return ICMP redirect message when the received packet should be forwarded over the same physical interface" to the same destination subnet then redirecting the request to the next hop. That happen to me today using a Mikrotik linux router and an F5 bigip LTM device.

root@(primaryadc)(cfg-sync In Sync)(Standby)(/Common)(tmos)# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
From 192.168.153.20: icmp_seq=1 Redirect Host(New nexthop: 192.168.153.2)
64 bytes from 8.8.8.8: icmp_seq=1 ttl=128 time=82.8 ms
From 192.168.153.20: icmp_seq=2 Redirect Host(New nexthop: 192.168.153.2)
64 bytes from 8.8.8.8: icmp_seq=2 ttl=128 time=123 ms
**routing table**
0.0.0.0  192.168.153.20  0.0.0.0         UG        0 0          0 external

score 0 · Answer 4 · answered Mar 15 '20 at 23:47

0

Doing this physically sounds like an easier solution - put each LAN on their own switch, and install a 4-port NIC into your one box that wants to serve all of them.

answered Mar 15 '20 at 23:47

cnd

220
2
5

And if he has 16 'boxes' he wants to serve? Possibly if he is experimenting with Kubernetes? – Drakes Nov 09 '20 at 00:48

Why do ICMP Redirect Host happen?

4 Answers4

Linked