New werf mode: combining werf & Argo CD into a unified CI/CD process

This article takes a look at a new experimental operation mode that connects the werf Open Source utility with Argo CD. It combines the features and user-friendliness of both tools into a unified CI/CD process. Note that these werf features are still being refined. That said, you can take the opportunity to check them out and see if they are right for you.

Introduction

Argo CD and werf are tools for delivering applications to a Kubernetes cluster using the Git repository as the single source of truth. They have similar goals, but their approaches to the process are different, rendering them good candidates to be used within the same process.

Argo CD is a Kubernetes GitOps operator for continuous delivery. It runs in a K8s cluster, monitors the repository with code or artifacts that have been built, and deploys the application to the cluster, ensuring that its state matches the one depicted in the repository. With it, you can easily track and manage deployments using sophisticated strategies, such as blue-green and canary, which are implemented in Argo Rollouts.

werf can be integrated into any CI/CD system (be it GitLab, GitHub Actions, or any other delivery management system) that developers and users are accustomed to. With werf, you can deploy applications to various environments, build artefacts in response to Git repository changes, or set up a local development environment on your machine.

Both of these tools are CI/CD-oriented. They follow different approaches and are designed for slightly different tasks. The main distinction between werf and Argo CD is that werf is not an operator that monitors the cluster state and aligns it to the desired schema and application versions (so-called self-healing). On the other hand, Argo CD is an operator and is capable of rendering these kinds of tasks. It is well-known in the broader community and has proven to be a good choice, so combining these two tools can render the application development and deployment process to a cluster even more straightforward and user-friendly.

Refer to the diagram below for more details on what parts of the CI/CD reference cycle are covered by each tool:

The reference CI/CD cycle and werf’s and Argo CD’s place in it

As the above diagram shows, Argo CD only handles the last step, deploying an app to a cluster. werf, in turn, adds things like local development, building, and publishing to the container registry; quick testing of commits in the pipeline; release artefact building; and running acceptance tests (optional).

Who could benefit from such a combination?

The combination of werf and Argo CD allows you to fully integrate Argo CD into any CI/CD system. Consequently, you end up with the benefits and the features of each tool.

For example, werf adds labels and tags to the built artefacts. You can use them to keep track of which branch and which commit a particular image was based on (there is a special plugin for Argo CD designed to do this; read more about it below). You can also start a deployment using argo sync right in a pipeline Job (granted, this option doesn’t quite fit the GitOps paradigm).

The following are the strengths Argo CD brings into the unified process:

• The pull model: artefacts and the cluster state are monitored by the operator running in it. The operator pulls changes, according to its own schedule and settings, and implements a self-healing cluster (the cluster state is reverted back to the one depicted in the Git/container registry in the case that manual changes have been made to it).
• A straightforward web interface to help the user monitor the state of the cluster and the processes running in it in real time, as well as manage its components.
• Support for multi-clusters: the user controls multiple tiers.
• Cold cluster is a standby cluster. Its state is kept in sync with the main one in order to quickly restore the system in case any failures occur.

Argo Rollouts allow you to implement deployment scenarios such as Blue-Green or Canary (you can read more about these and other deployment strategies in this article).

The following are the strengths werf brings into a unified process:

• all the information you need for developing and debugging is stored in the CI system;
• a consistent, efficient method for developing and publishing final artefacts for deployment to production: local application development (e.g., using minikube), testing (unit tests, linters, integration and acceptance tests, etc.), and easy-to-set-up review environments;
• standardisation of project build and deployment configuration with the option to combine application release building and publishing out-of-the-box;
• integration with any CI/CD system.

Many end-users, from developers to cluster administrators, can benefit from such a suite. Developers get a convenient CI/CD system they are familiar with that integrates with Git, allows for creating pull requests and deployment to review environments, and provides good observability for pipelines and Jobs linked to Git. Administrators, on the other hand, get the courtesy of a single entry point for all changes to the cluster in the form of Git and Argo CD.

How the development process goes for the user

Once the setup is complete (links to instructions are provided below), the typical process for the end user goes more or less as follows:

• The user develops the project locally on their machine: builds the images and deploys the application to a local Kubernetes cluster.
• Next, they push the changes into the branch in the Git repository and create a pull request.
• The created PR triggers the CI/CD system, which builds images based on it, and runs quick tests (unit tests and linters) using werf.
• The user can optionally deploy the application into a review environment (using werf) by clicking a button in the CI/CD system interface.
• If the application works as expected, the PR gets merged into the main branch of the project.
• A release artefact is then published. It can be used for more testing or for deployment into a production-like environment or production environment.
• The long-running tests, such as acceptance, e2e, and so on, are initiated. There are two ways to do this: by deploying the release artefact with werf or Argo CD.
• The release artefact is then deployed into the Kubernetes tier using Argo CD.

See the documentation for more info on how this process works.

How it works on a fundamental level

The process can be divided into two parts:

• werf builds and publishes the release artefact (so-called bundle) in the container registry.
• Argo CD pulls the release artefact from the container registry and deploys it to production.

Let’s take a closer look at each part.

werf

Let’s revert back to the “reference” CI/CD process (see the chart above). It clearly captures the jist of what happens during the release artefact build and publication stages.

First, the user comes up with new features or makes changes to existing ones locally. werf renders this job significantly easier with its breadth of options for local development, such as tracking file changes or new commits in the Git repository in the local project directory (.git).

The changes are then committed to the repository. For local development, this results in rebuilding and redeploying the application on the local cluster. By contrast, for remote repository, a new commit triggers the CI/CD system to start building using werf on the worker allocated to that system.

At that point, the application is tested and inspected. Once the test stage is complete, werf produces a release artefact, ensuring that all the necessary images are built. Helm charts configured to use the assembled images are then published to the container registry, completing the creation of the so-called bundle.

Argo CD

Argo CD’s job is to detect new bundles in the container registry and deploy them to the cluster. This can be done either manually or automatically, if the Argo CD Image Updater with the necessary patch is set up. This patch monitors the repository for new bundles from werf and initiates the deployment of a new application version to the cluster.

A special plugin has been created for the Argo CD as a ready-made image (registry.werf.io/werf/werf-argocd-cmp-sidecar:VERSION) for integrating werf and Argo CD as described above. This plugin takes care of rendering the published bundle. The plugin is optional, but without it the connection between the CI/CD system and the deployed application won’t work, since it allows the components of the deployed application to be matched with the corresponding commits in the source Git repository. Without this plugin, it would be impossible to view the tags and labels werf assigned during the build.

For more information about the features of the plugin and the patch, refer to the official documentation.

How to try it out

Trying out the new mode is easy. Just make sure that you have the latest version of werf installed. On top of that, you will need to prepare the Kubernetes cluster by installing Argo CD and the necessary dependencies, and then configuring your CI/CD system to work with Argo CD and werf.

Conclusion

Thus, we’ve explored a new experimental werf mode in which the user is encouraged to take advantage of the capabilities of two tools to deliver applications to the K8s cluster, leveraging the strengths of both to achieve the best results. The new mode is still undergoing development and testing, but you can already try it by following the instructions in the documentation. Feedback, comments, and suggestions are much appreciated!

This article has been written by our technical writer Konstantin Nezhbert.