Skaffold is a tool which handles the workflow of building, pushing and deploying container images and has the added benefit of facilitating an excellent local dev loop.
In this post I will be exploring using Skaffold for local development of a Java based application
Installing Skaffold
Installing Skaffold locally is straightforward, and explained well here. It works great with minikube as a local kubernetes development environment.
Skaffold Configuration
My sample application is available in a github repository here - https://github.com/bijukunjummen/hello-skaffold-gke
Skaffold requires at a minimum, a configuration expressed in a skaffold.yml file, with details of
- How to build an image
- Where to push the image
- How to deploy the image - Kubernetes artifacts which should be hydrated with the details of the published image and used for deployment.
In my project, the skaffold.yml file looks like this:
apiVersion: skaffold/v2beta16
kind: Config
metadata:
name: hello-skaffold-gke
build:
artifacts:
- image: hello-skaffold-gke
jib: {}
deploy:
kubectl:
manifests:
- kubernetes/hello-deployment.yaml
- kubernetes/hello-service.yaml
This tells Skaffold:
- that the container image should be built using the excellent jib tool
- The location of the kubernetes deployment artifacts, in my case a deployment and a service describing the application
The Kubernetes manifests need not hardcode the container image tag, instead they can use a placeholder which gets hydrated by Skaffold:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-skaffold-gke-deployment
spec:
replicas: 1
selector:
matchLabels:
app: hello-skaffold-gke
template:
metadata:
labels:
app: hello-skaffold-gke
spec:
containers:
- name: hello-skaffold-gke
image: hello-skaffold-gke
ports:
- containerPort: 8080
The image section gets populated with real tagged image name by Skaffold.
Now that we have a Skaffold descriptor in terms of skaffold.yml file and Kubernetes manifests, let's see some uses of Skaffold.
Building a local Image
A local image is built using the "skaffold build" command, trying it on my local environment:
skaffold build --file-output artifacts.json
results in an image published to the local docker registry, along with a artifact.json file with a content pointing to the created image
{
"builds": [
{
"imageName": "hello-skaffold-gke",
"tag": "hello-skaffold-gke:a44382e0cd08ba65be1847b5a5aad099071d8e6f351abd88abedee1fa9a52041"
}
]
}
If I wanted to tag the image with the coordinates to the Artifact Registry, I can specify an additional flag "default-repo", the following way:
skaffold build --file-output artifacts.json --default-repo=us-west1-docker.pkg.dev/myproject/sample-repo
resulting in a artifacts.json file with content that looks like this:
{
"builds": [
{
"imageName": "hello-skaffold-gke",
"tag": "us-west1-docker.pkg.dev/myproject/sample-repo/hello-skaffold-gke:a44382e0c008bf65be1847b5a5aad099071d8e6f351abd88abedee1fa9a52041"
}
]
}
The kubernetes manifests can now be hydrated using a command which looks like this:
skaffold render -a artifacts.json --digest-source=local
which hydrates the manifests, and the output looks like this:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-skaffold-gke-deployment
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: hello-skaffold-gke
template:
metadata:
labels:
app: hello-skaffold-gke
spec:
containers:
- image: us-west1-docker.pkg.dev/myproject/sample-repo/hello-skaffold-gke:a44382e0c008bf65be1847b5a5aad099071d8e6f351abd88abedee1fa9a52041
name: hello-skaffold-gke
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: hello-skaffold-gke-service
namespace: default
spec:
ports:
- name: hello-skaffold-gke
port: 8080
selector:
app: hello-skaffold-gke
type: LoadBalancer
The right image name now gets plugged into the Kubernetes manifests and can be used for deploying to any Kubernetes environment.
Deploying
Local Development loop with Skaffold
The additional benefit of having a Skaffold configuration file is in the excellent local development loop provided by Skaffold. All that needs to be done to get into the development loop is to run the following command:
skaffold dev --port-forward
which builds an image, renders the kubernetes artifacts pointing to the image and deploying the Kubernetes artifacts to the relevant local Kubernetes environment, minikube in my case:
➜ hello-skaffold-gke git:(main) ✗ skaffold dev --port-forward Listing files to watch... - hello-skaffold-gke Generating tags... - hello-skaffold-gke -> hello-skaffold-gke:5aa5435-dirty Checking cache... - hello-skaffold-gke: Found Locally Tags used in deployment: - hello-skaffold-gke -> hello-skaffold-gke:a44382e0c008bf65be1847b5a5aad099071d8e6f351abd88abedee1fa9a52041 Starting deploy... - deployment.apps/hello-skaffold-gke-deployment created - service/hello-skaffold-gke-service created Waiting for deployments to stabilize... - deployment/hello-skaffold-gke-deployment is ready. Deployments stabilized in 2.175 seconds Port forwarding service/hello-skaffold-gke-service in namespace default, remote port 8080 -> http://127.0.0.1:8080 Press Ctrl+C to exit Watching for changes...
The dev loops kicks in if any of the file is changed in the project, the image gets rebuilt and deployed again and is surprisingly quick with a tool like jib for creating images.
Debugging with Skaffold
Debugging also works great with skaffold, it starts the appropriate debugging agent for the language being used, so for java, if I were to run the following command:
skaffold debug --port-forward
and attach a debugger in Intellij using a "Remote process" pointing to the debug port
It would pause execution when a code with breakpoint is invoked!
Debugging Kubernetes artifacts
Since real Kubernetes artifacts are being used in the dev loop, we get to test the artifacts and see if there is any typos in them. So for eg, if I were to make a mistake and refer to "port" as "por", it would show up in the dev loop with an error the following way:
WARN[0003] deployer cleanup:kubectl create: running [kubectl --context minikube create --dry-run=client -oyaml -f /Users/biju/learn/hello-skaffold-gke/kubernetes/hello-deployment.yaml -f /Users/biju/learn/hello-skaffold-gke/kubernetes/hello-service.yaml] - stdout: "apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: hello-skaffold-gke-deployment\n namespace: default\nspec:\n replicas: 1\n selector:\n matchLabels:\n app: hello-skaffold-gke\n template:\n metadata:\n labels:\n app: hello-skaffold-gke\n spec:\n containers:\n - image: hello-skaffold-gke\n name: hello-skaffold-gke\n ports:\n - containerPort: 8080\n" - stderr: "error: error validating \"/Users/biju/learn/hello-skaffold-gke/kubernetes/hello-service.yaml\": error validating data: [ValidationError(Service.spec.ports[0]): unknown field \"por\" in io.k8s.api.core.v1.ServicePort, ValidationError(Service.spec.ports[0]): missing required field \"port\" in io.k8s.api.core.v1.ServicePort]; if you choose to ignore these errors, turn validation off with --validate=false\n" - cause: exit status 1 subtask=-1 task=DevLoop kubectl create: running [kubectl --context minikube create --dry-run=client -oyaml -f /Users/biju/learn/hello-skaffold-gke/kubernetes/hello-deployment.yaml -f /Users/biju/learn/hello-skaffold-gke/kubernetes/hello-service.yaml] - stdout: "apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: hello-skaffold-gke-deployment\n namespace: default\nspec:\n replicas: 1\n selector:\n matchLabels:\n app: hello-skaffold-gke\n template:\n metadata:\n labels:\n app: hello-skaffold-gke\n spec:\n containers:\n - image: hello-skaffold-gke\n name: hello-skaffold-gke\n ports:\n - containerPort: 8080\n" - stderr: "error: error validating \"/Users/biju/learn/hello-skaffold-gke/kubernetes/hello-service.yaml\": error validating data: [ValidationError(Service.spec.ports[0]): unknown field \"por\" in io.k8s.api.core.v1.ServicePort, ValidationError(Service.spec.ports[0]): missing required field \"port\" in io.k8s.api.core.v1.ServicePort]; if you choose to ignore these errors, turn validation off with --validate=false\n" - cause: exit status 1
This is a great way to make sure that the Kubernetes manifests are tested in some way before deployment
Conclusion
Skaffold is an awesome tool to have in my toolbox, it facilitates building of container images, tagging them with sane names, hydrating the Kubernetes manifests using the images, deploying the manifests to a Kubernetes environment. In addition it provides a great development and debugging loop.
No comments:
Post a Comment