Full support for popular Docker Registry implementations in werf

Published in

werf blog

6 min readMay 6, 2020

Please note this post was moved to a new blog: https://blog.werf.io/ — follow it if you want to stay in touch with the project’s news!

Container registries tend to support the Docker Registry HTTP API, allowing their users to rely on the same tools to operate them. However, some implementations have their peculiarities and limitations. Thus, you have to take into account their specifics when using them as part of your CI/CD toolchain. That is exactly what happened when we decided to improve the way our werf GitOps utility manages the lifecycle of images.

In this article, we will discuss the main peculiarities of Docker Registry implementations supported by werf as well as resulting improvements in our tool.

Storing images

As you know, you need a registry to store and distribute Docker images. Strictly speaking, a registry is just a service to store various repositories (AWS ECR, Azure CR, Docker Hub, and so on). The repository stores images grouped by name.

When building applications and/or deploying them to Kubernetes with werf, you can use --images-repo and --images-repo-mode parameters. They allow you to specify where and how (in a single or multiple repositories*) application images will be stored in the registry.

* You can learn more about these parameters (including the problem definition and history of development) in our “Monorepo/multirepo support in werf (and what does it have to do with Docker Registry?)” article.

The --images-repo parameter can be either a registry address or a repository address. In essence, its value serves as the basis for composing the name of an image and not necessarily specifies the repository where the images will be stored (take a look at templates below to clarify this point).

The --images-repo-mode parameter supports two values that define the template for composing the final image name:

IMAGES_REPO:IMAGE_NAME-TAG is a template for the monorepo mode;
IMAGES_REPO/IMAGE_NAME:TAG is a template for the multirepo mode.

Note: The --images-repo-mode parameter does not make any sense when using a single and unnamed image (image: ~) since the final image will always be using the--images-repo parameter’s value as a repository (IMAGES_REPO:TAG).

Since images cannot be stored in the registry, there is no reason to use the monorepo mode along with the registry. For the same reason, you cannot use an unnamed image (image: ~) with the registry regardless of the --images-repo-mode value.

Supported options

Thus, there are three possible parameter combinations available to the user. In the table below, there is a list of various Docker Registry implementations along with the support for these combinations:

* AWS ECR does not allow the automatic creation of a repository when publishing an image. So, you have to create it manually (using UI or an API) before you can use it.

As the table shows, the only restriction applies to Docker Hub, GitHub Packages, and Quay: they do not support multi-level repository names (REGISTRY/REPO/SUBREPO).

werf automatically detects the type of registry as well as the value of the --images-repo-mode parameter. For implementations that support all possible combinations, werf sets the default (multirepo) mode. It automatically sets the mode for other registry types depending on the value of the --images-repo parameter.

The --implementation parameter allows you to set the type of registry manually. It might come in handy if, for example, you are using a hosts alias instead of an address, or if you need to manually start the cleaning of GitLab Registry up using werf (if necessary, the ci-env command can set the required value if werf is used as part of the pipeline).

Example

And now it is time for a little illustration of a case in point. Let us consider the following werf.yaml:

project: flant
configVersion: 1
---
image: backend
dockerfile: Dockerfile
target: backend
---
image: frontend
dockerfile: Dockerfile
target: frontend

Also, you will need a Docker Hub account (it is named flant in our case).

The --images-repo=flant parameter will result in the following tags:

flant/frontend:tag
flant/backend:tag

werf build-and-publish -s=:local -i=flant --tag-custom=tag

On the other hand, the --images-repo=flant/project parameter will result in the following tags:

flant/project:frontend-tag
flant/project:backend-tag

werf build-and-publish -s=:local -i=flant/project --tag-custom=tag

Note on `werf build-and-publish`

This command builds images specified in the werf.yaml and publishes them to the registry. Here is the description of the parameters used above:

-s is the short version of the --stages-storage parameter. It specifies the stages-storage (you can read more in the documentation);
-i is the short version of --images-repo;
--tag-custom is one of the available tagging strategies (learn more about naming images and tagging strategies).

Images cleanup

Resources cleanup is another task that is required when using container registries. In the table below, you can see various registry implementations along with the required API for deleting tags:

For container registries that support the deletion of tags using the Docker Registry API, all you need to do is execute a docker login command and provide a token with the relevant scopes/permissions.

Other implementations, such as AWS ECR, Azure CR, Docker Hub, and GitHub Packages, do not fully support the Docker Registry API. Thus, you will need to use their built-in APIs to delete tags. Also, some supplementary information might be required for a cleanup.

Details

Let’s talk a bit about the peculiarities of working with these implementations when cleaning up tags using werf:

In the case of Azure, you have to install the CLI (az) and assign one of the following roles to the user: Owner, Contributor, or AcrDelete. You can read more about it in the docs: Azure CR roles and permissions.
You have to use AWS SDK to delete tags in AWS ECR. You can install the AWS CLI and perform the configuration (aws configure), or you can set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables before making a cleanup.
werf uses the Docker Hub API to clean up tags in the Docker Hub repository. However, you will have to specify the token. You can specify a token* ( --images-repo-docker-hub-token) or set up a user with a password (i.e.--images-repo-docker-hub-username & --images-repo-docker-hub-password).

Note that you cannot use a personal access token to clean up resources in the Docker Hub since they can only be deleted using the admin credentials.

* Below is the script to get an access token automatically:

HUB_USERNAME=USERNAME
HUB_PASSWORD=PASSWORD
HUB_TOKEN=$(curl -s -H "Content-Type: application/json" -X POST -d '{"username": "'${HUB_USERNAME}'", "password": "'${HUB_PASSWORD}'"}' https://hub.docker.com/v2/users/login/ | jq -r .token)

In GitHub, you can delete a version of a private package using GraphQL. To do this, you must provide a token (--images-repo-github-token) with the following scopes: read:packages, write:packages, delete:packages, and repo.

Summary

Currently, werf supports the following set of Docker Registry implementations:

AWS ECR;
Azure CR;
Docker Hub;
Default (DTR);
GCR;
GitHub Packages;
GitLab Registry;
Harbor;
Quay.

All these variants pass through a full testing cycle, and we are always ready to respond to changes.

The features described above are available in werf versions starting with V1.1.9+fix2.

The current status, list of changes, and peculiarities related to various Docker Registry implementations are available in the project documentation.