2

I have tried to run the guestbook example in Kubernetes Github repository but I can't reach this service from my local host. My test enviroment consists of two virtual machines (with CentOS7) provisioned by CloudStack, with OpenShift Origin installed on it. Here it's the services list:

    [root@openshift-master amd64]# ./oc get svc
NAME              CLUSTER-IP       EXTERNAL-IP   PORT(S)                   AGE
docker-registry   172.30.39.251    <none>        5000/TCP                  1d
guestbook         172.30.55.125    nodes         3000/TCP                  56m
kubernetes        172.30.0.1       <none>        443/TCP,53/UDP,53/TCP     1d
redis-master      172.30.24.94     <none>        6379/TCP                  1h
redis-slave       172.30.132.250   <none>        6379/TCP                  1h
router            172.30.33.117    <none>        80/TCP,443/TCP,1936/TCP   1d

The service exposed is guestbook. Here is the service guestbook description:

[root@openshift-master amd64]# ./oc describe svc guestbook
Name:           guestbook
Namespace:      default
Labels:         app=guestbook
Selector:       app=guestbook
Type:           NodePort
IP:         172.30.55.125
Port:           <unset> 3000/TCP
NodePort:       <unset> 30642/TCP
Endpoints:      172.17.0.6:3000,172.17.0.7:3000,172.17.0.8:3000
Session Affinity:   None
No events.

If I do:

curl 172.30.55.125:3000

It works only from the node who host the guestbook pod, from others node in the cluster and my host machine (192.168.1.2) It doesn't work.

I opened all ports in CloudStack, otherwise I can't ssh the nodes and in the node I set this firewall rule:

firewall-cmd --permanent --zone=public --add-port=30642/tcp

30642 is the NodePort, that is mandatory to reach it from out of the cluster. Have you any idea on how to resolve? Thanks in advance.

DarkSkull
  • 161
  • 1
  • 9
  • 1
    (I would have posted this as a comment, if I had the required reputation) It is surprising that you cant reach the service from other nodes in your cluster. Just to make sure that your cluster is setup properly, can you try accessing some other service and see how that works? Like can you access the kubernetes service from your nodes in the cluster? – Nikhil Jindal Jun 08 '16 at 20:10
  • yes, I can access Docker container from the node in which the pod stays. – DarkSkull Jun 08 '16 at 20:12
  • 1
    Sorry, I meant can you access any other service (apart from the guestbook service) from a node which does not have a pod for that service running? For ex: you can try accessing kubernetes service from a node where the kube-apiserver pod is not running. That will help us isolate the problem with general cluster setup or the guestbook example. – Nikhil Jindal Jun 08 '16 at 20:17
  • I can only access a pod if that pod is running on that node. Anyway, I installed another cluster with only Kubernetes without OpenShift and in that case I can access any pod from any node, also the host. There should be something wrong in my OpenShift configuration. – DarkSkull Jun 08 '16 at 20:22
  • 1
    hmm ok. fwiw, its the kube-proxy (http://kubernetes.io/docs/admin/kube-proxy/) which does the magic for you to be able to access a service with pod running on a different node. If you run into problems again, you can probably look at kube-proxy logs. kube-proxy runs on every node in the cluster. – Nikhil Jindal Jun 08 '16 at 20:53

1 Answers1

1

curl 172.17.0.6:3000 (i.e. each of the Endpoints addresses) should be usable directly from every cluster node. If it doesn't work, then the cluster network is not set up correctly. This could include any firewall or SDN that filters packets sent from one node to another.

172.30.55.125:3000 should have an entry in the iptables list on every cluster node, maintained by the local kube-proxy daemon on each cluster node. If curling a remote endpoint works but using the service virtual ip and port fails, then it's possible that kube-proxy is not working. Check its iptables entries, its process status, and its log file.

Finally, it's possible that the guestbook app is indeed receiving the connection, but it's then aborting or blocking while it tries a doomed reverse DNS lookup.

aecolley
  • 943
  • 4
  • 15