2

Im running a kubernetes (kubeflow + k8s) pod with a jupyter notebook and a docker service outside of the kubernetes server, im currently trying to connect to a sql service but it keeps getting ConnectionResetError, both firewall and docker are exposing the port needed but the k8s keeps not being able to connect, what could be the problem? (tell me if you need more details).

Thanks.

tommyduarte
  • 31
  • 1
  • How is your setup configured? You have minikube with kubernetes,kubeflow and istio and sql on your local pc deployed with docker? From the istio perspective you would have to add [ServiceEntry](https://istio.io/latest/docs/reference/config/networking/service-entry/) so istio injected pods could talk with external database. There is an example in istio [documentation](https://istio.io/latest/blog/2018/egress-mongo/). Note you might also need to [disable mtls](https://istio.io/latest/faq/security/#mysql-with-mtls) for it. – Jakub Dec 10 '20 at 11:26
  • Both are in seperated GCP Compute Instances, the kubeflow is deployed with minikf, the docker is running a docker-compose with a presto image and a bunch of another ones from the project. Gonna check that ServiceEntry and MTLS ty – tommyduarte Dec 10 '20 at 12:03
  • @tommyduarte as you already mention under answer, could you elaborate which firewall you needed to add? – PjoterS Mar 10 '21 at 12:27

1 Answers1

0

As mentioned in comments

From the istio perspective to make this work you would have to add ServiceEntry so istio injected pods could talk with external database.

ServiceEntry enables adding additional entries into Istio’s internal service registry, so that auto-discovered services in the mesh can access/route to these manually specified services. A service entry describes the properties of a service (DNS name, VIPs, ports, protocols, endpoints). These services could be external to the mesh (e.g., web APIs) or mesh-internal services that are not part of the platform’s service registry (e.g., a set of VMs talking to services in Kubernetes). In addition, the endpoints of a service entry can also be dynamically selected by using the workloadSelector field. These endpoints can be VM workloads declared using the WorkloadEntry object or Kubernetes pods. The ability to select both pods and VMs under a single service allows for migration of services from VMs to Kubernetes without having to change the existing DNS names associated with the services.

There is an example in istio documentation.

Note that You may find MySQL can’t connect after installing Istio. This is because of PERMISSIVE mode, which does not work with MySQL. You may see error messages such as ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0.

There have two options to solve the problem.

1.Disable Mutual TLS.

Choose this option if you don’t want Istio mutual TLS. You achieve this by disabling mutual TLS on the MySQL service explicitly.

$ kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: mysql-nomtls-peerauthn
spec:
  selector:
    matchLabels:
      app: <YOUR-MYSQL-SERVICE>     # The label of *your* K8s Service
  mtls:
    mode: DISABLE
EOF

2.Enable mutual TLS in STRICT mode.

If you want mutual TLS protection for MySQL, enable mutual TLS using a destination rule and an authentication policy.

$ kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: mysql-mtls-peerauthn
spec:
  selector:
    matchLabels:
      app: <YOUR-MYSQL-SERVICE>     # The label of *your* K8s Service
  mtls:
    mode: STRICT
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: mysql-mtls-dr
spec:
  host: YOUR-MYSQL-SERVICE     # The name of *your* K8s Service
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
EOF
Jakub
  • 365
  • 1
  • 9