Using environment variables in Kubernetes deployment spec

Question

I currently use a Kubernetes spec Deployment.yaml for deploying a service. The spec includes a verbatim reference to a specific IP address (marked as <static-ip-address> below):

spec:
  type: LoadBalancer
  loadBalancerIP: <static-ip-address>

I am concerned about pushing information such as passwords or IP addresses into remote Git repositories. Can I avoid this e.g. by making use of environment variables, e.g. with a deployment spec and actual deployment roughly as follows:

spec:
   type: LoadBalancer
   loadBalancerIP: ${SERVICE_ADDRESS}

and

export SERVICE_ADDRESS=<static-ip-address>
kubectl create -f Deployment.yaml

Obviously this specific syntax does not work yet. But is something like this possible and if so how?

I'd prefer not relying on a separate provisioning tool. Secrets and ConfigMaps seem promising, but apparently they cannot be consumed in a way that suits this purpose. If I could directly reference a static IP address that was defined with gcloud compute addresses create service-address that would be best.

Answer 1

A much easier/cleaner solution: envsubst

In deploy.yml:

LoadbalancerIP: $LBIP

Then just create your env var and run kubectl like this:

export LBIP="1.2.3.4"
envsubst < deploy.yml | kubectl apply -f -

You just put regular Bash variables into whatever file you want to use, in this case the YAML manifest, and have ensubst read that file. It will output the file with the env vars replaced by their values. You can also use it to create new files like this:

envsubst < input.yml > output.yml

envsubst is available in e.g. Ubuntu/Debian gettext package.

Answer 2

There was another pleasantly simple solution: I have a Google Compute Address my-address defined, and I can apparently use it in the service spec like so: loadBalancerIP: my-address.

With this as "external" source for IP addresses and secrets for passwords there is no more need for a provisioning tool (or templates) for my simple use case (within a GKE environment).

OBSOLETE NOW: I have decided on using a provisioning tool of sorts, namely "built-in" sed, after all.

My Deployment.yaml now contains a "template variable" e.g. in

loadBalancerIP: $$EXTERNAL_IP

and I deploy the service with, say, 1.2.3.4 as external IP address with

cat Deployment.yaml | sed s/\$\$EXTERNAL_IP/1.2.3.4/ | kubectl create -f -

Answer 3

You can write a simple pre-processor to do variable substitution on your yaml files (or you can use jsonnet to accomplish the same thing on json configuration files).

There is some discussion around adding templates directly into Kubernetes configuration but it isn't yet implemented or available.

Answer 4

Until templates are available, the easiest way to do it is to run a job which uses the Kubernetes API to update the service. A short shell script in an alpine-based image, coupled with a secret (containing the IP address) and a configmap (containing the template), should be simple enough. The difficult bit is correctly using the authentication and authorization features of the apiserver.

https://stackoverflow.com/questions/30690186/how-do-i-access-the-kubernetes-api-from-within-a-pod-container gives an example of accessing the API. Obviously, you'll want to POST to /api/v1/namespaces/default/services instead of the GET in that example.