1

I'm trying to setup Jenkins in a Digital Ocean Kubernetes cluster. I'm using a NGINX ingress controller as I want to access my server from a subdomain (jenkins.example.com). Everything is working fine so far, I have my UI on said domain secured with a custom certificate. I started encountering problems when trying to connect my agents (or slaves I read elsewhere?) to the server.

What I've tried

I tried to setup a load balancer but couldn't make it work as I am using Digital Ocean and most of the docs I found were for GKE / EKS and others, which have their own internal LB. I also tried mapping a route /agents on the service's port name then on another service with a "ClusterIP" type, without success (not found from Jenkins).

My current config

jenkins-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins-dep
  labels:
    app: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      containers:
      - name: jenkins
        image: jenkins/jenkins:lts-jdk11
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
        - containerPort: 50000

jenkins-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: jenkins-svc
  labels:
    app: jenkins
spec:
  type: NodePort
  selector:
    app: jenkins
  ports:
  - name: ui
    protocol: TCP
    port: 8080
    targetPort: 8080
    nodePort: 32500
  - name: agents
    protocol: TCP
    port: 50000
    targetPort: 50000
    nodePort: 32501

jenkins-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: jenkins-ingress
spec:
  tls:
  - hosts:
      - jenkins.example.com
    secretName: tls-secret
  rules:
  - host: jenkins.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: jenkins-svc
            port:
              name: ui
      #- path: /agents
      #  pathType: Prefix
      #  backend:
      #    service:
      #      name: jenkins-svc
      #      port:
      #        name: agents

The bottom line is: how do I make available the Jenkins port 50000 for the agents to connect on my server? Or how could I change Jenkins settings to accommodate such a config?

greg_wss
  • 13
  • 3

1 Answers1

2

To the best of my knowledge, Jenkins uses a custom protocol for communicating with its build agents that is a binary Java RMI, and not HTTP based. Since the Ingress resource is only for host: based virtual dispatching of HTTP requests, you cannot use any kind: Ingress declaration for doing that

But, with the nginx-ingress controller specifically, they do allow exposing TCP services which will likely do what you want

mdaniel
  • 2,338
  • 1
  • 8
  • 13
  • Thank you for this. I'm currently following their doc, I'll update this post if I manage to achieve anything. – greg_wss Jun 09 '21 at 20:11
  • Effectively, the solution is to expose the TCP service. To avoid errors, I recommend using the helm release from ingress-nginx (I tested 4.0.12) like described here (https://docs.cloudbees.com/docs/cloudbees-ci/latest/cloud-setup-guide/configure-ports-jnlp-agents#values-yaml-configure-jnlp) – andolsi zied Dec 03 '21 at 13:34