Introducing trdl: an Open Source solution for secure and continuous software delivery
We are excited to announce the official launch of our new Open Source project! trdl (short for “true delivery”) provides a secure channel for delivering updates from the Git repository to the user’s host.
trdl incorporates three core components that help protect the update system from potential attacks: the HashiCorp Vault, the CNCF-approved TUF repository, and Git.
This article explains how trdl works, its advantages compared to existing update systems, and how to begin using it.
trdl security is based on Vault, TUF, and Git. trdl prevents potential attacks on the update system and minimizes any damage they may bring about. The following three components work in unison to make this possible:
- HashiCorp Vault is the platform the trdl server is run on. It enables the secure management of encryption keys.
- A TUF-based (The Update Framework) repository to download updates from. It protects against unauthorized software access as well as key compromise/loss. It also ensures the relevance, consistency, and integrity of the data. (TUF is a CNCF project that graduated in 2019.)
- Git stores commit code, configurations, and GPG signatures to verify operations.
Release channels instead of software versions: The user chooses a release channel with the desired compatibility and stability levels: alpha, beta, early access, stable, and rock-solid. Developers then use these channels to distribute new software versions whenever they like.
GPG quorum: Each commit in Git is signed by “M of N” developers using GPG signatures. The quorum is used for both releasing new software versions (tags) and publishing release channels.
Versatility: trdl is a cross-platform software update tool for macOS, Windows, and Linux. It supports all popular shells (command language interpreters).
Simplicity: There is no need for developers to have to acquire any additional knowledge: Git is the basic tool for using trdl. Furthermore, ready-made GitHub Actions are provided for a quick start. All users need to do is install the trdl-client to update and operate the software.
Continuous updates: You can update and operate the software in a continuous fashion. The trdl use command takes into account the specifics of working as part of a CI system: the update process runs in the background while the user operates the local version of the software until the shell session ends (more scripts and commands will follow).
What’s the point of having our own “package manager”?
We develop products that have to be updated continuously and securely. However, existing update methods and systems have critical flaws that render them unsuitable for production.
Limitations of security algorithms: HTTPS, SSL, and TLS only secure the connection; they do not secure the contents of the stuff you download from the repository. The PGP algorithm is vulnerable to many modern attacks on the update process. GPG signatures cannot protect your laptop, e.g., from being compromised. To render the update process as secure as possible, you have to build a system for “approving code” and signing the assembled files that is immune to human error. The existing tools and algorithms are good on their own, but they cannot deliver a robust upgrade system.
Limitations of continuous delivery: Continuous CI/CD-based delivery is suitable for SaaS products but not so good for delivering target files to user hosts.
Limitations of package managers: Most package managers are not flexible enough. Each platform requires its own manager. At the same time, their level of automation is nearly non-existent: the user has to manually add a package repository, search for a package, as well as install, update, and remove it.
So we decided to develop our own update system free of the above drawbacks.
How trdl works (with a basic example)
In terms of structure, trdl is a client-server application. The trdl server reliably populates and organizes the TUF repository, while the trdl client ensures reliable and continuous delivery of updates and software operation. Here, an update means an arbitrary set of files for all supported platforms: binaries, shell scripts, Ansible playbooks, etc.
The renewal process involves three main steps:
- Releasing a new software version.
- Publishing the release channel (switching the latest version in the release channel(s)).
- Pulling the release via the update channel.
Let’s look at the first two in detail.
Releasing a new software version
The developer creates a Git tag with the new software version (v1.0.1 in the picture below) and signs it with their GPG signature. Upon approval by the project quorum, the tag gets submitted to the trdl server. If the Git tag contains the minimum set of allowed GPG signatures, the build initiates. The resulting build (along with the metadata) gets pushed to the TUF repository. Meanwhile, the client continues to run the old software version — v1.0.0.
- GPG-1-3 are the signatures used by the developer and the project quorum;
- vault-plugin-secrets-trdl is a Vault plugin that checks for mandatory Git signatures and triggers a build;
- Artifact is the result of an assembly (binary files, scripts, or packages);
- Release channel is a channel to be used for updates;
- trdl-client is a client installed on the server (VM) or the user’s laptop.
Publishing the release channels
The developer makes changes to the release channels and commits them to the Git repository while signing with the GPG signature. Subsequent to quorum approval, the commit gets pushed to the Vault plugin. The plugin checks whether a commit contains the minimum allowable set of GPG signatures; if it does, the plugin signs an updated list of channels and their associated releases. The updated channels (along with the metadata) get pushed to the TUF repository. Users start getting updates according to the new release channels configuration.
Is trdl ready for production?
We have been successfully using trdl for some time now to continuously deliver werf, our CI/CD tool, to CI runners and user hosts. As a result, the werf development team now boasts a convenient and secure delivery process; meanwhile, users get to enjoy a user-friendly, convenient client to keep the utility up-to-date.
How do I use trdl?
The Quickstart section shows you how to install and configure the essential elements of the system. There you can find basic scenarios for the administrator, the developer, and the user.
The Reference contains a set of basic commands for working with the CLI and the Vault plugin, as well as examples of YAML configurations and tips on organizing a TUF repository.
The Security section details how trdl security works and what it can and cannot protect you from.
trdl is an Open Source project. We encourage all enthusiastic users to take part in its development. Please share your suggestions and comments, and feel free to open a PR. We would also appreciate your GitHub stars!