Using GitLab as a convenient Helm charts repository

GitLab is a powerful yet user-friendly software management tool. Like any large and self-sufficient product, it is constantly evolving and improving. This article explores its newly added functionality named Helm Chart Registry, which allows you to store Helm charts in the GitLab Package Registry. You can already use this feature and benefit from it. In this article, for simplicity’s sake, I will call it the GitLab Helm repo.

Helm charts are often the basis for defining the Kubernetes infrastructure. It is clear that for teams dealing with a large number of projects, it makes sense to standardize the way these charts are handled. Starting with GitLab 14.1, it is now possible to configure the storage of shared charts for all the projects you work with.

Briefly about the concept

Before GitLab introduced such a feature, you could use, for example, ChartMuseum or store the manifests right inside the repository with the application code instead of a dedicated chart repo. Sure, it didn’t have any obvious shortcomings in terms of functionality. Still, standardizing the project layout didn’t turn out to be such a picnic as the repository turned into a big mess over time.

On the other hand, the GitLab Helm repo allows you to store commonly used Helm charts for popular infrastructure components as well as the files needed to run your app in Kubernetes in a shared repository in your corporate GitLab account. For example, you can store a Helm chart there for stateless applications, which includes manifests to configure the following Kubernetes resources:

  • Deployment;
  • ConfigMap;
  • Secret;
  • Service;
  • HPA;
  • VPA;
  • PDB.

Imagine this: all you need is the values.yaml file that describes the entire application logic, while all the necessary manifests are rendered during deployment.

How does it work? A secondary (or client) repository is created when you install and configure GitLab for a new project. Its CI pipeline can connect to your corporate repo, pull the latest Helm charts from it, or re-push charts to the local project repository. This simple mechanism allows you to update, change and fix all components in one place, leaving the task of pulling updates to the client repositories. In addition, it will enable you to address the issue of standardization in developing new projects.

Setting up the main repository

In this section, we will set up several repositories step by step and show you how to configure the interaction between them. Note that our CI pipeline will be based on the werf tool.

werf is not mandatory for the method discussed below. However, it facilitates the entire process through a comprehensive approach to working with subcharts. Also, we have extensive experience with this tool.

Now let’s create the central repository for storing the shared Helm charts.

1. Create a new repository:

2. Create directories and core files for the CI/CD:

.
├── .gitlab-ci.yml
├── .helm
│   ├── Chart.lock
│   ├── charts
│   │   └── my-chart
│   │       ├── Chart.yaml
│   │       ├── templates
│   │       │   └── main.yaml
│   │       └── values.yaml
│   └── Chart.yaml
├── README.md
└── werf.yaml

We will use the following basic ConfigMap manifest that prints the chart name as the main.yaml file:

#---- .helm/charts/my-chart/templates/main.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ .Chart.Name }}
data:
  file: |
    {{ .Chart.Name }}

#---- werf.yaml
project: charts-repo
configVersion: 1

Let’s take a closer look at the .gitlab-ci.yml file. It’s pretty complex, so we’ve added comments to the lengthy and uncommon commands:

stages:
- publish-charts

variables:
  REPO_URL: "${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/packages/helm/api/stable/charts"

before_script:
  - set -eo pipefail
  # Activate werf
  - type trdl && . $(trdl use werf 1.2 stable)
  - type werf && source $(werf ci-env GitLab --as-file)
  - |
    # Update the available Helm repo via werf
    werf helm repo update
    # Look for all the files with chart descriptions and use them to build dependencies
    find . -type f -regex '.*/\(Chart.ya?ml\|requirements.ya?ml\)' -exec \
      sh -c 'werf helm dependency build $(dirname "{}") --skip-refresh' \;

"publish charts":
  stage: publish-charts
  script:
  - |
    # Browse all the directories with the charts and pack the charts by putting them in the .packages directory.
    mkdir -p .packages
    while read chart; do
      echo "[PACKAGING CHART $chart]"
      werf helm package "$chart" -d .packages
    done < <(find .helm/charts -mindepth 1 -maxdepth 1 -type d)
  - |
    # Define two variables: CHART_NAME and CHART_VERSION
    find .packages -mindepth 1 -maxdepth 1 -type f -name '*.tgz' -exec sh -c 'basename "$0"' '{}' \; | while read package; do
      CHART_NAME=$(echo $package | sed -e 's/-[0-9]\.[0-9]\.[0-9]\.tgz$//g')
      CHART_VERSION=$(echo $package | sed -e 's/^[a-zA-Z-].*-//g' | sed -e 's/.tgz$//g')
      # Check if a chart with this name and version already exists in a Helm repo
      CHART_EXISTS=$(werf helm search repo -l $REPO_NAME/$CHART_NAME | { egrep "$REPO_NAME/$CHART_NAME\s"||true; } | { egrep "$CHART_VERSION\s"||true; } | wc -l)
      # If it does not exist, then push it into the package registry; otherwise,
      # display a message that the chart is already present in the Helm repo
      if [ $CHART_EXISTS = 0 ]; then
        curl -sSl --post301 --form "chart=@.packages/$package" --user "$REPO_PUSH:$REPO_PUSH_SECRET" "$REPO_URL"
      else
        echo "Chart package $package already exists in Helm repo! Skip!"
      fi
    done
  only:
  - tags

Now you need to create some keys and make them available as environment variables in order to use this CI process (that is, to push the chart into the package registry).

3. Go to Repository settings (Settings -> Repository -> Deploy tokens) and create a token with the read_package_registry and write_package_registry permissions.

4. Go to CI/CD settings (Settings -> CI/CD -> Variables) and create the following environment variables:

  • REPO_NAME is an alias (for example, my-charts);
  • REPO_PUSH is the name of the token from step 3;
  • REPO_PUSH_SECRET is a Secret for the token from step 3.

5. Exec into the machine with the GitLab Runner (it will run the CI pipeline for this project) and register the Helm repo (the werf helm commands here are similar to the regular helm commands):

werf helm repo add --username $REPO_PUSH --password $REPO_PUSH_SECRET $REPO_NAME ${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/packages/helm/stable
werf helm repo update

Remember to replace environment variables with actual values!

Note: we used the token/Secret pair from step 3 instead of creating an additional one for GitLab Runner. However, in real-life situations, we recommend generating a separate read_package_registry-only token to configure GitLab Runners.

6. Commit, push changes, create a new tag (e.g., my-chart-1.0.0), and navigate to the created pipeline. Once the Job is completed, go to Packages & Registries -> Package Registry and check that our Helm chart is there.

Note: we use tag-based deployment since it lets you find the base commit for a particular Helm chart quickly. In addition, there are cases when the package registry is missing: in such a situation, it will be empty upon recovery. Tags will allow you to populate the package registry with the latest versions of Helm charts (those in the main branch); however,  problems may occur if some applications use outdated Helm chart versions. By having a different tag for each version of the chart, we can quickly retrieve the component required for the application.

7. Check the status of the repository on GitLab Runner using werf:

werf helm repo update
werf helm search repo my-charts
    
NAME                  CHART VERSION    APP VERSION    DESCRIPTION
my-charts/my-chart    1.0.0

As you can see, the Helm chart is now in the repository.

Setting up client repositories

The client (secondary, i.e. project-side) repository that uses the shared chart is configured a little differently.

1. Create an empty repository with the following layout:

.
├── .gitlab-ci.yml
├── .helm
│   └── Chart.yaml
├── README.md
└── werf.yaml

2. Here are the contents of its main files:

#---- .helm/Chart.yaml
apiVersion: v2
name: client-charts-repo
version: 1.0.0
dependencies:
- name: my-chart
  version: ~1.0
  repository: "@my-charts"
#---- werf.yaml
project: client-charts-repo
configVersion: 1
#---- .gitlab-ci.yml
stages:
- publish-charts

variables:
  REPO_URL: "${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/packages/helm/api/stable/charts"
  HELM_URL: "${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/packages/helm/stable"

default:
  before_script:
  - set -eo pipefail
  - type trdl && . $(trdl use werf 1.2 stable)
  - type werf && source $(werf ci-env GitLab --as-file)

.base_publish_charts:
  stage: publish-charts
  script: |
    werf helm repo add --force-update --username $MAIN_REPO_PULL --password $MAIN_REPO_PULL_SECRET $MAIN_REPO_NAME $MAIN_HELM_URL
    werf helm repo update
    werf helm dependency update .helm/
    find .helm/charts -mindepth 1 -maxdepth 1 -type f -name '*.tgz' -exec sh -c 'basename "$0"' '{}' \; | while read package; do
      CHART_NAME=$(echo $package | sed -e 's/-[0-9]\.[0-9]\.[0-9]\.tgz$//g')
      CHART_VERSION=$(echo $package | sed -e 's/^[a-zA-Z-].*-//g' | sed -e 's/.tgz$//g')
      CHART_EXISTS=$(werf helm search repo $REPO_NAME | { egrep "$REPO_NAME/$CHART_NAME\s" || true; } | { egrep "$CHART_VERSION\s" || true; } | wc -l)
      if [ $CHART_EXISTS = 0 ]; then
        curl -sSl --post301 --form "chart=@.helm/charts/$package" --user "$REPO_PUSH:$REPO_PUSH_SECRET" "$REPO_URL"
      else
        echo "Chart package $package already exists in Helm repo! Skip!"
      fi
    done
    werf helm repo add --username $REPO_PULL --password $REPO_PULL_SECRET $REPO_NAME $HELM_URL
    werf helm repo update
    echo "Configuring the local PC."
    echo "REPO_URL: $REPO_URL"
    echo "werf helm repo add --username $REPO_PULL --password $REPO_PULL_SECRET $REPO_NAME $HELM_URL"
  rules:
  - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
    when: on_success
    allow_failure: true

"publish charts":
  extends:
  - .base_publish_charts
  tags:
  - werf

3. Now, let’s look at the environment variables used in the project settings:

  • MAIN_REPO_PULL is the name of the read_package_registry token in the main repository;
  • MAIN_REPO_PULL_SECRET is the token’s Secret with read_package_registry scope in the main repository;
  • MAIN_REPO_NAME is the alias of the main repository;
  • MAIN_HELM_URL is the URL for accessing the main repository;
  • CLIENT_REPO_NAME is the alias of the client repository;
  • CLIENT_REPO_PUSH is the token’s name with the write_package_registry permissions in the client repository;
  • CLIENT_REPO_PUSH_SECRET is the token’s Secret with the write_package_registry scope in the client repository;
  • CLIENT_REPO_PULL is the token’s name with the read_package_registry scope in the client repository;
  • CLIENT_REPO_PULL_SECRET is the token’s Secret with the read_package_registry scope in the client repository.

4. Configure the environment variables of the client repository to access the main repository:

  • Navigate to the main repository to Settings -> Repository -> Deploy tokens tab and create a new token with the scope read_package_registry.
  • Copy and paste it to the client repository’s MAIN_REPO_PULL and MAIN_REPO_PULL_SECRET environment variables.
    • MAIN_REPO_NAME is the same as the REPO_NAME you set in step 4 when configuring the main repository; MAIN_HELM_URL must match the main repository’s value of ${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/packages/helm/stable.

5. In the client repository, go to Settings -> Repository -> Deploy tokens and create two tokens.

The first token must have the write_package_registry scope. Insert its data into the appropriate environment variables:

  • CLIENT_REPO_PUSH — token name;
  • CLIENT_REPO_PUSH_SECRET — token Secret.

The second token must have the read_package_registry scope. Insert its data into the appropriate environment variables:

  • CLIENT_REPO_PULL — token name;
  • CLIENT_REPO_PULL_SECRET — token Secret.

Next, create an alias for the Helm repo and put it in the CLIENT_REPO_NAME variable, e.g. CLIENT_REPO_NAME = client-charts-repo.

This completes the configuration of the client repository. If you run the pipeline on the main branch in this configuration, the Job will fetch all the charts specified as dependencies in the .helm/Chart.yaml file from the main Helm repo. Such a mechanism adds extra flexibility seeing as it only allows you to include the required Helm Charts in the project.

In this CI implementation, only the most recent Helm chart from the main Helm repo is pulled to the client repository. You can modify the CI so that it re-pushes all the available Helm chart versions. You can also add some logic (like rendering and validating Helm charts before pushing them). However, since that is beyond the scope of this article, I will omit examples of them.

At this point, we have successfully set up all of the components. Now when you push a Helm chart package to the main Package registry, it can be downloaded through the CI pipelines of the client repositories. You can also configure CI pipelines to suit your needs; e.g., you can configure client CIs to start on schedule and pull chart updates automatically (without running the pipeline manually).

Takeaways

In this article, we explored configuring GitLab to pull Helm сharts from centralized storage in another repository. This handy GitLab feature allows you to manage Helm charts in a quick and clear-cut fashion.

Comments

Your email address will not be published. Required fields are marked *