In this post I provide an example of how to use Skaffold with Travis CI to build a CI/CD pipeline that deploys your code in a production Kubernetes cluster. I also have a quick discussion on why I switched from AWS Serverless with Lambda to Kubernetes. Skip the overview for procedural steps.

Overview

While working on a new project I decided to switch from AWS Lambda to running my own k8s cluster. This choice was primarily due to the use of Redis in my application and the difficulty of maintaining a Serverless application in AWS. One of the first challenges I encountered when trying to build a CI/CD pipeline with this new model is the lack of native k8s support in Travis CI. While this looked like a show stopper at first I quickly found a work-around.

So why create yet another article?  I’m using Skaffold to build my application in my k8s cluster and I did not find a good online resources for this type of integration. If you’re not familiar with Skaffold it is a great tool to help deploy your application in the development k8s cluster and rebuild the deployment anytime you update your application code – I’ll automate just about anything if it saves me from constantly switching between my editor and the CLI.

This document serves as a way to teach others as well as remind myself how to do this when I create my next proof of concept in few months.

So what is CI/CD?  CI stands for continuous integration. This means we validate our code against unit tests and integration tests every time we commit code to our Git repository. This helps ensure our code is ready to go into production and helps foster the “commit often” mindset. Another benefit is that you’ll quickly know if your commit breaks the build process. The CD stands for continuous deployment. This allows our code to immediately be pushed into production if all unit and integration testing was successful. This allows for a quick feed-back loop and iteration of our code – a core principle of DevOps.

So why k8s versus AWS Serverless with Lambda?  One of my biggest gripes about a Serverless application using AWS lambda is how difficult it was to maintain after months of not having my head in the code base. Either an issue/bug would pop-up or a feature request would be submitted and I’d have to re-learn how to build out a test lambda environment in AWS to make sure everything works before I deployed into production. My primary job no longer includes development but I still write code due to necessity or to prove out that a concept is possible. Therefor, I need an environment that is easy to wrap my head around even if I haven’t touched the code base in several months – this is why I choose Kubernetes because I can treat infrastructure as code.

I also need to focus on the code and not the infrastructure – this is why I choose Skaffold. Skaffold allows me to focus on building out the infrastructure once at the start of a project and from then on I can focus on my application code. This fits my development process perfectly because in 3 months I don’t have to remember how to build/deploy I can simply check my code into my Git repository and everything happens “magically” for me.

So why go through the trouble of a CI/CD pipeline?  Remember how I said everything happens “magically” for me – well this is what the CD portion of my pipeline takes care of. At the beginning of any new project I spend a little extra time focusing on what the application needs to function. With this data I can build a k8s cluster that contains the support resources my application needs (such as Redis, load balancing, etc.). I then configure Skaffold to deploy my application into the k8s cluster. This allows me to easily update my production version via a GitHub with the Travis CI integration.

Configuration

Skaffold

You’ll need to create a skaffold.yaml configuration file. The key attributes we need to configure are the imageName and manifests. The imageName is the Docker Hub repository (or private repository) the application will be published to. The manifests defines where Skaffold should look in your code base for the Kubernetes deployment configuration files. In the example below, I have a folder named k8s and any file starting with “k8s-” will be processed by Skaffold.

Kubernetes

For my example I’ve combined my k8s configurations into a single file. This is a common k8s service and deployment configuration so I won’t go into any detail here.

Travis CI

Travis CI will build a small nodeJS container to test my application. I need to modify this container so it can talk to my Docker repository and my Kubernetes cluster (example configuration file below).

before_install: this section of the configuration file tells Travis CI to download and install the kubectl and Skaffold binaries.

after_success: this section of the configuration file executes after a successful unit and integration test. There are several parts to this section so I’ll break them down into smaller topics:

echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin

Travis CI has native support for Docker. This command logs the container into Docker Hub so we can upload the application upon a successful test.

mkdir ${HOME}/.kube
cp ./k8s/config.yaml ${HOME}/.kube/config
cd ${HOME}/.kube

We need to build the .kube directory and copy the skeleton kubectl configuration file.

sed -i 's/KUBE_CLUSTER_NAME/'"$KUBE_CLUSTER_NAME"'/g' config
sed -i 's/KUBE_CLUSTER_CERTIFICATE/'"$KUBE_CLUSTER_CERTIFICATE"'/g' config
sed -i 's/KUBE_CLIENT_CERTIFICATE/'"$KUBE_CLIENT_CERTIFICATE"'/g' config
sed -i 's/KUBE_CLIENT_KEY/'"$KUBE_CLIENT_KEY"'/g' config
sed -i 's/KUBE_ADMIN_PWD/'"$KUBE_ADMIN_PWD"'/g' config

We need to replace variables in our skeleton configuration file with secure values stored in Travis CI. The names are self-explanatory and can be copied from your local ~/.kube/config file into the Travis CI configuration.

• KUBE_CLUSTER_NAME
• KUBE_CLUSTER_CERTIFICATE
• KUBE_CLIENT_CERTIFICATE
• KUBE_CLIENT_KEY
• KUBE_ADMIN_PWD

The full Travis CI configuration is posted below:

Conclusion

You should now be able to check your code into the Git repository and your production K8S cluster will update automatically.  I hope this article helped you and if you have any questions please feel free to post them below.