Running werf in GitLab CI/CD without a Docker server
werf is an Open Source tool for building applications and deploying them to Kubernetes. This article discusses the new experimental werf operating mode that does not require a Docker server to be run. You will learn how to ensure this mode works properly on your machine, build an image, and use the Kubernetes executor to automate builds in GitLab CI/CD.
In order to render the build environment predictable and reproducible, you have to run builds in separate, isolated containers or single-use virtual machines. In the case of GitLab CI/CD, you can use the Docker executor for builds. The Docker executor starts the Docker container on the GitLab Runner. The shell instructions defined in the GitLab CI are then executed in this container. In the case of werf, these instructions include
werf converge, and
Alternatively, you can use the Kubernetes executor: like its Docker counterpart, it renders the environment for running werf predictable and reproducible, with the building performed in the Kubernetes Pods. This renders scaling easier by moving workloads from GitLab Runners to Kubernetes while allowing other K8s cluster resources to be used.
There are ready-made images to run werf in Docker/Kubernetes. They can also be used to execute GitLab CI instructions in a Docker or Kubernetes executor. We’ll show you how to use them below.
How this works in werf
werf supports two operating modes: with and without a Docker server. The latter mode is experimental.
When operating in the latter mode, werf uses the built-in Buildah in rootless mode instead of the Docker server/client.
Currently, this mode only supports Dockerfile-based building. The alternative Stapel builder will be available in the near future.
Let’s look at how you can use the new mode to build images in Kubernetes without a Docker server. To do so, you first need to set up the environment and make sure the system supports Buildah. After that, you can build the image using the new mode and automate the assembly process using the Kubernetes executor in GitLab CI/CD.
1. Preparing the environment
A Docker server is not required in Linux operating systems where werf can natively use Buildah. In Windows and macOS, you can’t do that without a Docker server due to a compatibility layer (a Docker image containing Buildah) required to run Buildah in these operating systems.
Now let’s look at using this mode to enable and run werf on Linux.
Before enabling the experimental mode, make sure that your system supports the rootless OverlayFS file system. Note that Linux kernel versions 5.11 and up support it by default (strictly speaking, a bugfix to enable rootless OverlayFS in SELinux was first introduced in version 5.13; however, many major Linux distributions have backported it into v5.11). You can use fuse-overlayfs if your system kernel does not support rootless OverlayFS.
As mentioned above, there are pre-made werf images that you can use. The images are kept up to date and follow a release schedule: the trdl package manager delivers werf as part of the release process within various release channels (from alpha to stable).
There are several images available. We recommend using the
ghcr.io/werf/werf:latest image. It contains the werf 1.2-stable and is based on Alpine Linux (hence its full name is
ghcr.io/werf/werf:1.2-stable-alpine). That’s the image we’re going to use in our example.
You can learn more about all the available images here.
In Buildah mode, werf can also be used without containers, e.g., if you want to build images containing werf yourself instead of using ready-made ones provided by developers. See the documentation to learn more.
2. Choosing the operating mode in the container
Depending on the user’s needs as well as the system parameters, there are three ways to use werf in Buildah mode in containers:
- Linux kernel with rootless OverlayFS: In this mode, all you need to do is disable the
AppArmorprofiles in the werf container.
- Linux kernel without rootless OverlayFS and a privileged container: In the event that your Linux kernel does not support rootless OverlayFS, Buildah and werf will use fuse-overlayfs instead. Note that you have to run werf in a privileged container in order to enable fuse-overlayfs.
- Linux kernel without rootless OverlayFS and a non-privileged container with additional settings: Similar to the previous mode, Buildah and werf will use fuse-overlayfs as the file system. To use fuse-overlayfs without a privileged container, run werf in a container with the following parameters:
seccompprofiles are disabled.
/dev/fusedevice is enabled.
In our case, we will use the most common and preferred operation mode – #1 (unprivileged container with rootless OverlayFS).
3. Building an image without a Docker server
Run the following command to start the build:
docker run \ --security-opt seccomp=unconfined --security-opt apparmor=unconfined \ ghcr.io/werf/werf:latest werf_command
The above command will run the selected container (
ghcr.io/werf/werf:latest) and execute the werf command (
werf_command) it contains.
4. Setting up a build in GitLab CI/CD that uses the Kubernetes executor
Configure GitLab runner in Kubernetes
[[runners]] name = "kubernetes-runner-for-werf" executor = "kubernetes" ... [runners.kubernetes] ... pod_annotations = ["container.apparmor.security.beta.kubernetes.io/werf-converge=unconfined"]
NB. See the official documentation to learn more about additional Kubernetes executor parameters.
Configure access to the Kubernetes cluster
There are two ways to configure access to the cluster in which the application is deployed:
- The first one involves a Service Account for the Kubenetes executor;
- The second one is based on the
kubeconfigfile with the appropriate settings.
Let’s look at each option in more detail.
Note that this method is only suitable if the Kubernetes executor is running in the target Kubernetes cluster.
See an example of
gitlab-kubernetes-runner-deploy Service Account:
apiVersion: v1 kind: ServiceAccount metadata: name: gitlab-kubernetes-runner-deploy --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: gitlab-kubernetes-runner-deploy roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: gitlab-kubernetes-runner-deploy namespace: default
Configure the GitLab runner (
/etc/gitlab-runner/config.toml) to use this Service Account:
[[runners]] name = "kubernetes-runner-for-werf" ... [runners.kubernetes] service_account = "gitlab-kubernetes-runner-deploy" ...
This method requires assigning the base64-encoded contents of the
~/.kube/config file to the
WERF_KUBECONFIG_BASE64 environment variable in the GitLab project. werf will automatically use this configuration to connect to the target Kubernetes cluster.
This method works well if the Kubernetes executor’s cluster and the target Kubernetes cluster are different.
Now for the final step. Below is an example of a basic task for building and deploying a project in GitLab CI/CD:
stages: - build-and-deploy Build and deploy application: stage: build-and-deploy image: ghcr.io/werf/werf script: - source $(werf ci-env gitlab --as-file) - werf converge tags: ["kubernetes-runner-for-werf"]
There we have it. Now everything is ready to run the build inside a werf container running in a Kubernetes cluster.
In this article, we discussed how werf can be used in GitLab CI/CD without relying on a Docker server, thereby rendering the build environment predictable and reproducible. We used a Kubernetes executor to run the build in a separate Pod in the cluster using the pre-made werf container.
We strongly recommend checking the werf documentation for a detailed description of all the available operating modes that do not require a Docker server to be run, as well as for issues that may arise and solutions for them.