No route to host elasticsearch exception

Question

When turning on elasticsearch, I am getting the following configuration error:

[2015-04-21 20:49:45,635][INFO ][discovery.zen            ] [Blackwulf] failed to send join request to master [[Conquistador][HzGNMjroRCuoHsoTyI3zag][elastic02][inet[/10.70.121.114:9300]]], reason [RemoteTransportException[[Conquistador][inet[/10.70.121.114:9300]][internal:discovery/zen/join]]; nested: NotSerializableTransportException[[org.elasticsearch.transport.ConnectTransportException] [Blackwulf][inet[/10.70.112.23:9300]] connect_timeout[30s]; No route to host; ]; ]

Even though the host is reachable via telnet:

[root@elastic05 ~]# telnet 10.70.121.114 9300
Trying 10.70.121.114...
Connected to 10.70.121.114.
Escape character is '^]'.
^]
telnet> quit
Connection closed.
[root@elastic05 ~]# telnet 10.70.121.114 9200
Trying 10.70.121.114...
Connected to 10.70.121.114.
Escape character is '^]'.
^]
telnet> quit
Connection closed.

I'm also getting even the localhost can't find a route out upon booting:

[2015-04-21 20:24:18,291][INFO ][node                     ] [Ezekiel Stane] version[1.4.1], pid[1347], build[89d3241/2014-11-26T15:49:29Z]
[2015-04-21 20:24:18,292][INFO ][node                     ] [Ezekiel Stane] initializing ...
[2015-04-21 20:24:18,306][INFO ][plugins                  ] [Ezekiel Stane] loaded [cloud-aws], sites []
[2015-04-21 20:24:21,558][INFO ][node                     ] [Ezekiel Stane] initialized
[2015-04-21 20:24:21,558][INFO ][node                     ] [Ezekiel Stane] starting ...
[2015-04-21 20:24:21,684][INFO ][transport                ] [Ezekiel Stane] bound_address {inet[/0.0.0.0:9300]}, publish_address {inet[/10.70.112.23:9300]}
[2015-04-21 20:24:21,696][INFO ][discovery                ] [Ezekiel Stane] elasticsearch/f5p3lsxyTfukHZ2E8_G4vQ
[2015-04-21 20:24:21,737][WARN ][transport.netty          ] [Ezekiel Stane] exception caught on transport layer [[id: 0x3d524f37]], closing connection
java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
        at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
        at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
        at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)

I have the hosts listed within elasticsearch.yml to connect via unicast and not multicast (which I have disabled).

Is multicast disabled on the remote host as well (10.70.121.114)? — Chad Smith, Apr 21 '15 at 22:18

score 4 · Answer 1 · answered Jun 03 '15 at 10:07

4

Give a name to your cluster and the issue will be solved.

Edit config/elasticsearch.yml in your elasticsearch directory.

Uncomment cluster.name: elasticsearch and change it to something like cluster.name: your_hostname

Then try to start elasticsearch again.

answered Jun 03 '15 at 10:07

Harikrishnan

1,057
2
14
31

1

This worked for me, but I do not want to have to do it every time I build elastic search - can and will you elaborate on why this fixes the problem? – nephiw Oct 15 '15 at 16:49

score 3 · Answer 2 · edited Jan 20 '17 at 13:19

No route to host these days often can mean that you are getting an ICMP Administratively Prohibited message -- ie. you are being denied by firewall.

They are particularly common on Red Hat systems, from what I have seen, but I would hope that you see them elsewhere too.

This makes it easier to differentiate from Connection Refused (which would just mean that there is nothing listening, or that the connection was reset -- possibly by firewall).

You can verify this with tcpdump -p -nn icmp or similar, and look for ICMP Administratively Prohibited messages when you connect.

For what its worth, the reason you get the very obtuse No route to host instead of something more understandable are twofold:

The ICMP Administratively Prohibited message is part of (IIRC) the unreachable codespace in ICMP;
The error codes are locked into the Berkeley Sockets API, which doesn't have any other useful error (ie. errno) to map it to.

As a result, I find myself having to educate every consultant that comes our way.

No route to host elasticsearch exception

2 Answers2