How to set dynamic values with Kubernetes yaml file?

How to set dynamic values with Kubernetes yaml file?

For example, a deployment yaml file:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: guestbook
spec:
replicas: 2
template:
metadata:
labels:
app: guestbook
spec:
container:
– name: guestbook
image: {{Here want to read value from config file outside}}

There is a ConfigMap feature with Kubernetes, but that’s also write the key/value to the yaml file. Is there a way to set the key to environment variables?

Solutions/Answers:

Solution 1:

I don’t think it is possible to set image through variable or Config Map in Kubernetes. But you can use for example Helm to make your deployments much more flexible and configurable.

Solution 2:

You can’t do it automatically, you need to use an external script to “compile” your template, or use helm as suggested by @Jakub.

You may want to use a custom bash script, maybe integrated with your CI pipeline.

Given a template yml file called deploy.yml.template very similar to the one you provided, you can use something like this:

#!/bin/bash

# sample value for your variables
MYVARVALUE="nginx:latest"

# read the yml template from a file and substitute the string 
# {{MYVARNAME}} with the value of the MYVARVALUE variable
template=`cat "deploy.yml.template" | sed "s/{{MYVARNAME}}/$MYVARVALUE/g"`

# apply the yml with the substituted value
echo "$template" | kubectl apply -f -

Solution 3:

One line:

cat app-deployment.yaml | sed "s/{{BITBUCKET_COMMIT}}/$BITBUCKET_COMMIT/g" | kubectl apply -f -

In yaml:

  ...
  containers:
  - name: ulisses
    image: niceuser/niceimage:{{BITBUCKET_COMMIT}}
  ...

Solution 4:

You can also use envsubst when deploying.

e.g.

cat $app/deployment.yaml | envsubst | kubectl apply ...

It will replace all variables in the file with their values.
We are successfully using this approach on our CI when deploying to multiple environments, also to inject the CI_TAG etc into the deployments.

Related:  Import data to config map from kubernetes secret

Solution 5:

My approach:

tools/jinja2-cli.py:

#!/usr/bin/env python3
import os
import sys
from jinja2 import Environment, FileSystemLoader

sys.stdout.write(Environment(loader=FileSystemLoader('templates/')).from_string(sys.stdin.read()).render(env=os.environ) + "\n")

Make rule:

_GENFILES = $(basename $(TEMPLATES))
GENFILES = $(_GENFILES:templates/%=%)

$(GENFILES): %: templates/%.j2 $(MKFILES) tools/jinja2-cli.py .env
        env $$(cat .env | xargs) tools/jinja2-cli.py < $< > $@ || (rm -f $@; false)

Inside the .j2 template file you can use any jinja syntax construct, e.g. {{env.GUEST}} will be replaced by the value of GUEST defined in .env

So your templates/deploy.yaml.j2 would look like:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: guestbook
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: guestbook
      spec:
        container:
          - name: guestbook
            image: {{env.GUEST}}

Another approach (using just bash builtins and xargs) might be

env $(cat .env | xargs) cat <<EOF | kubectl create -f -
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: guestbook
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: guestbook
      spec:
        container:
          - name: guestbook
            image: ${GUEST}
EOF

Solution 6:

I create a script called kubectl_create and use it to run the create command. It will substitute any value in the template that is referenced in an environment variable.

#!/bin/bash
set -e
eval "cat <<EOF
$(<$1)
EOF
" | kubectl create -f -

For example, if the template file has:

apiVersion: v1
kind: Service

metadata:
  name: nginx-external
  labels:
    app: nginx

spec:
  loadBalancerIP: ${PUBLIC_IP}
  type: LoadBalancer
  ports:
  - name: http
    port: 80
    targetPort: 80
  - name: https
    port: 443
    targetPort: 443

  selector:
    app: nginx

Run kubectl_create nginx-service.yaml and then the environment variable PUBLIC_IP will be substituted before running the actual kubectl create command.

Solution 7:

I have been using kubetpl

It has three different template flavors and supports ConfigMap/Secret freezing.

Solution 8:

I create a script called kubectl_apply. It loads variables from .env, replace ${CUSTOMVAR} in yml and pass it to kubectl command

  #!/bin/bash
  set -a
  source .env
  set +a
  eval "cat <<EOF
  $(<$1)
  EOF
  " | kubectl apply -f -

Solution 9:

Helm is exactly meant for such things and a lot more. It handle complex set of resource deployment as a group etc.

Related:  Restart container within pod

But if we are still looking for some simple alternative then how about using ant?

If you want to modify the file as part of build process or test process then you can go with ant task as well.

Using ant you can load all environment values as property or you can simply load properties file like:

<property environment="env" />
<property file="build.properties" />

Then you can have a target which converts template files into your desired yaml file.

<target name="generate_from_template">

    <!-- Copy task to replaces values and create new file -->
    <copy todir="${dest.dir}" verbose="true" overwrite="true" failonerror="true">

        <!-- List of files to be processed -->
        <fileset file="${source.dir}/xyz.template.yml" />

        <!-- Mapper to transform filename. Removes '.template' from the file
            name when copying the file to output directory -->
        <mapper type="regexp" from="(.*).template(.*)" to="\1\2" />

        <!-- Filter chain that replaces the template values with actual values 
            fetched from properties file -->
        <filterchain>
            <expandproperties />
        </filterchain>
    </copy>
</target>

Of course you can use a fileset instead of file in case you want to change values dynamically for multiple files (nested or whatever)

Your template file xyz.template.yml should look like:

apiVersion: v1
kind: Service
metadata:
  name: ${XYZ_RES_NAME}-ser
  labels:
    app: ${XYZ_RES_NAME}
    version: v1
spec:
  type: NodePort
  ports:
    - port: ${env.XYZ_RES_PORT}
      protocol: TCP
  selector:
    app: ${XYZ_RES_NAME}
    version: v1

env. property being loaded from environment variables and other from property file

Hope it helped 🙂

References