I have 2 kvm virtualised servers. Each has a internet-facing ens3 and a ens6 interface for a VLAN bet ween the hosts.
I'm running Ubuntu 18.04.
I have configured the interfaces via netplan.
network:
version: 2
renderer: networkd
ethernets:
ens3:
dhcp4: yes
ens6:
dhcp4: no
addresses: [10.0.0.10/24]
The other machine has 10.0.0.20/24.
One machine runs multiple services and the other run runs Prometheus.
I have setup node_exporter on both machines and listen on the privates networks ip address.
I can ping from one machine to the other also connect via ssh between them. It seems connectivity works in general.
I can even curl the node_exporter from the second machine
curl http://10.0.0.10:9100 -v
* Rebuilt URL to: http://10.0.0.10:9100/
* Trying 10.0.0.10...
* TCP_NODELAY set
* Connected to 10.0.0.10 (10.0.0.10) port 9100 (#0)
> GET / HTTP/1.1
> Host: 10.0.0.10:9100
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Mon, 31 Dec 2018 09:00:42 GMT
< Content-Length: 150
< Content-Type: text/html; charset=utf-8
<
<html>
<head><title>Node Exporter</title></head>
<body>
<h1>Node Exporter</h1>
<p><a href="/metrics">Metrics</a></p>
</body>
* Connection #0 to host 10.0.0.10 left intact
The issue now comes when actually trying to pull the metrics:
curl http://10.0.0.10:9100/metrics -v
* Trying 10.0.0.10...
* TCP_NODELAY set
* Connected to 10.0.0.10 (10.0.0.10) port 9100 (#0)
> GET /metrics HTTP/1.1
> Host: 10.0.0.10:9100
> User-Agent: curl/7.58.0
> Accept: */*
>
^C
The connection simply hangs.
When doing the same curl command from the other machine itself, it works as expected. I also tried binding the node_exporter to 0.0.0.0. This also works, I can fetch the metrics through the servers public ip address.
I'm confused that a call to http://10.0.0.10:9100 works but not to http://10.0.0.10:9100/metrics. No errors or anything in my logs. Any idea what could be the reason for this?
