Use Travis CI to automate the deployment of a Python Gunicorn app to Kubernetes

You have written a great new python application and now its time to move it from development into the real world. Where are you going to host it? One of the first choices that comes to mind these days is Kubernetes running in your cloud provider of choice. What we really mean when we say run it in Kubernetes (K8s) is we want to containerize our application and have some way to automate the deployment and operation of it. As the K8s website say, Kubernetes is a “system for automating deployment, scaling, and management of containerized applications.”

While I use Python and Gunicorn in this document you could use any application you want. The interesting part of this story is having Travis CI do the work to build your Docker images and get them running in K8s.

Some prerequisites if you want to follow along.
1) A cloud account somewhere with a Kubernetes cluster running.
2) Docker installed on your workstation (for creating the container and testing)
3) kubectl installed on your workstation.
4) A GitHub account (https://github.com/)
5) A travis-ci account (https://travis-ci.com)
6) A Docker Hub account (https://hub.docker.com/)

Step 1 — Get app containerized and running on local system, then push to Docker Hub

First lets layout our directory structure and create a few files.

mkdir ~/kube-general/
mkdir mycoolapp
mkdir mycoolapp/kube
mkdir mycoolapp/src
mkdir mycoolapp/scripts
mkdir mycoolapp/supervisor
cd mycoolapp
touch kube/deployment.yml
touch kube/service.yml
touch src/server.py
touch scripts/runapp.sh
touch scripts/deploy.sh
touch scripts/updateKube.sh
touch supervisor/app.conf
touch Dockerfile
touch requirements.txt
touch .dockerignore
touch README.md

Now lets create a simple python Flask app for testing. Edit the src/server.py file in your preferred editor.

import os
from flask import Flask
from datetime import datetime
app = Flask(__name__)
@app.route("/")
def index():
    today = datetime.now()
    return "Hi. The date and time is: {}".format(today.strftime("%Y-%m-%d %H:%M:%S"))
if __name__ == "__main__":
    SERVER_PORT = os.environ.get("SERVER_PORT", "8000")
    app.run(host="", port=int(SERVER_PORT))

Now edit the requirement.txt file. This tells the system what python packages to install.

Flask>=1.1.2
gevent>=1.5.0
greenlet>=0.4.13 
gunicorn>=20.0.4

Now we need to edit the scripts/runapp.sh file to tell Docker how to start our app. Make sure to “chmod +x scripts/runapp.sh”

#!/bin/sh
gunicorn --name "Test App" --chdir /app/src --bind 0.0.0.0:$SERVER_PORT server:app gevent --worker-connections 1000 --workers 4 --log-file appgunicorn.log

Now for the app.conf file in the supervisor directory.

[program:flaskapp]
command=/usr/bin/runapp.sh
autostart=true
autorestart=true

Last but not least, lets populate the Dockerfile. I recommend using PyPy instead of the default Cpython implementation. Cpython is great for testing and running scripts, but if you are building a web app PyPy is just faster.

FROM pypy:3-7.3
# Set the working directory to /app 
WORKDIR /app
# Copy the current directory contents into the container at /app 
ADD . /app
# Run python's package manager and install the flask package 
RUN pip install -r requirements.txt
# Configure ports 
EXPOSE 80
# Run apt-get, to install the SSH server, and supervisor 
RUN apt-get update \
  && apt-get install -y supervisor \
  && rm -rf /var/lib/apt/lists/* \
  && apt-get clean
# start scripts 
COPY scripts/runapp.sh /usr/bin/
# supervisor config 
ADD supervisor/app.conf /etc/supervisor/conf.d/
# Run the chmod command to change permissions on above file in the /bin directory 
RUN chmod 755 /usr/bin/runapp.sh
# Default environmental variables 
ENV SERVER_PORT 80
# run commands in supervisor 
CMD ["supervisord", "-n"]

Time to build the container

docker build -t <dockerUser>:pypytest:0.1 .

When the build is done you should see something like “Successfully built”. We can now test that it works on out workstation

docker run -p 80:80 <dockerUser>:pypytest:0.1

You should be able to connect to http://localhost

Note:
If you want to connect to the running containers shell you will need to find the container name “docker container ls”, Then docker exec -it <container> /bin/bash

Log into Docker and push your container to Docker Hub

docker login
docker push <dockerUser>/pypytest

Step 2 — Create a K8s service account, role and role binding

Outside of this project directory create a kube-general directory, I put mine in my home directory. We are going to create an account yml, roll yml, and role-binding yml.

In this directory create a file cicd-service-account.yml and edit it.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cicd
  namespace: default

Now create cicd-role.yml and edit it.

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: cicd
  namespace: default
rules:
  - apiGroups: ["", "apps", "batch", "extensions"]
    resources: ["deployments", "services", "replicasets", "pods", "jobs", "cronjobs"]
    verbs: ["*"]

and finely create cicd-role-binding.yml and edit it.

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: cicd
  namespace: default
subjects:
  - kind: ServiceAccount
    name: cicd
    namespace: default
roleRef:
  kind: Role
  name: cicd
  apiGroup: rbac.authorization.k8s.io

Apply these files to your cluster

kubectl apply -f ~/kube-general/

Step 3 — Create K8s deployment and service

edit kube/deployment.yml Make sure you replace <dockerUser> VERSION is what sed is looking for and replaces that with the build number

apiVersion: apps/v1
kind: Deployment
metadata:
  name: pypytest
  namespace: default
  labels:
    app: pypytest
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pypytest
  template:
    metadata:
      labels:
        app: pypytest
    spec:
      containers:
        - name: pypytest
          image: <dockerUser>/pypytest:VERSION
          ports:
            - containerPort: 80
              name: http

edit kube/service.yml I created this as a type LoadBalancer so it will be exposed to the internet

apiVersion: v1
kind: Service
metadata:
  name: pypytest
  namespace: default
  labels:
    app: pypytest
spec:
  selector:
    app: pypytest
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: http
      name: http

Now apply those files to the cluster

kubectl apply -f kube/

Step 4 — Create git repo and push files

Make sure you create the repo in github first

git init .
git remote add origin https://github.com/<gitUser>/mycoolapp
git add --all
git commit -m "initial commit"
git push -u origin master

Step 5 — travis.yml

The .travis.yml tells Travis CI what it needs to do. These can go from mild to wild. For this project I am going to keep it simple. In the real world you should have some testing and other validation built into this. Again, remember to replace <dockerUser>

language: bash
sudo: required
branches:
  only:
  - master
services:
- docker
before_install:
  - curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
  - chmod +x kubectl && sudo mv kubectl /usr/local/bin/kubectl
  - echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
  - docker build -t <dockerUser>/pypytest:$TRAVIS_BUILD_NUMBER .
  - docker push <dockerUser>/pypytest
script:
- bash scripts/updateKube.sh
deploy:
  provider: script
  script: bash scripts/deploy.sh
  on:
    branch: master

We are telling Travis our project uses bash. Doesn’t really matter but bash keeps it simple and Travis doesn’t expect some of the options it would with other languages. We then tell Travis we need the docker service.

In before_install we download kubectl and move it. Login to Docker Hub. Build our container and version it with the TRAVIS_BUILD_NUMBER. Push this build to Docker Hub

In scripts we update our deployment.yml with the Docker image version.

In deploy we update your Kubernetes and tell it to download the new container.

Step 6 — Scripts needed

updatekube.sh uses sed to update the Kubernetes deployment.yml with the TRAVIS_BUILD_NUMBER

#!/bin/sh
sed -i "s/VERSION/${TRAVIS_BUILD_NUMBER}/g" kube/deployment.yml
cat kube/deployment.yml

deploy.sh applies the deployment.yml to the cluster and then sets the image used to the correct version. Replace <dockerUser>

#!/bin/bash
echo "$KUBERNETES_CLUSTER_CERTIFICATE" | base64 --decode > cert.crt
/usr/local/bin/kubectl \
  --kubeconfig=/dev/null \
  --server=$KUBERNETES_SERVER \
  --certificate-authority=cert.crt \
  --token=$KUBERNETES_TOKEN \
  apply -f ./kube/
echo "The build number is ${TRAVIS_BUILD_NUMBER}"
/usr/local/bin/kubectl \
  --kubeconfig=/dev/null \
  --server=$KUBERNETES_SERVER \
  --certificate-authority=cert.crt \
  --token=$KUBERNETES_TOKEN \
  set image deployment/pypytest pypytest=<dockerUser>/pypytest:${TRAVIS_BUILD_NUMBER} --record

Step 7 — Setup Travis ci

Link travis to github or your repo of choice
https://docs.travis-ci.com/user/tutorial/?utm_source=help-page&utm_medium=travisweb#to-get-started-with-travis-ci-using-github

In Travis go to the settings for your repo. Here you can add environment variables.

DOCKER_PASSWORD = Your Docker Hub password

DOCKER_USER = Your Docker HUB user

KUBERNETES_CLUSTER_CERTIFICATE = You can find this in your ~/.kube/config file listed as certificate-authority-data

KUBERNETES_SERVER = This is also listed in the .kube/config file listed as server

KUBERNETES_TOKEN = To get the token for your cicd user run
kubectl get secret $(kubectl get secret | grep cicd-token | awk ‘{print $1}’) -o jsonpath=’{.data.token}’ | base64 — decode

At this point we should be all done. You can push a change to your repo and watch Travis get to work. You can run “kubectl get pods” to see your pod restart with the new image.

2) Docker installed on your workstation (for creating the container and testing)

3) kubectl installed on your workstation.

4) A GitHub account (https://github.com/)

5) A travis-ci account (https://travis-ci.com)

6) A Docker Hub account (https://hub.docker.com/)