@@ -4,7 +4,7 @@ This project implements a GitLab CI/CD template to build, check and inspect your
...
@@ -4,7 +4,7 @@ This project implements a GitLab CI/CD template to build, check and inspect your
## Usage
## Usage
This template can be used both as a [CI/CD component](https://docs.gitlab.com/ee/ci/components/#use-a-component)
This template can be used both as a [CI/CD component](https://docs.gitlab.com/ee/ci/components/#use-a-component)
or using the legacy [`include:project`](https://docs.gitlab.com/ee/ci/yaml/index.html#includeproject) syntax.
or using the legacy [`include:project`](https://docs.gitlab.com/ee/ci/yaml/index.html#includeproject) syntax.
### Use as a CI/CD component
### Use as a CI/CD component
...
@@ -41,11 +41,11 @@ variables:
...
@@ -41,11 +41,11 @@ variables:
The template supports following ways of building container images:
The template supports following ways of building container images:
1. The former **Docker-in-Docker (DinD)** technique, that was widely used for years because of no other alternative, but that
1. The former **Docker-in-Docker (DinD)** technique, that was widely used for years because of no other alternative, but that
is now commonly recognized to have **significant security issues** ([read this post](https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/) for more info),
is now commonly recognized to have **significant security issues** ([read this post](https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/) for more info),
2. Or using [kaniko](https://github.com/GoogleContainerTools/kaniko), an open-source, daemonless tool from Google for building Docker
2. Or using [kaniko](https://github.com/GoogleContainerTools/kaniko), an open-source, daemonless tool from Google for building Docker
images, and that solves Docker-in-Docker security issues (and also speeds-up build times).
images, and that solves Docker-in-Docker security issues (and also speeds-up build times).
3. Or using [buildah](https://buildah.io/), an open-source, daemonless tool backed by RedHat for building Docker
3. Or using [buildah](https://buildah.io/), an open-source, daemonless tool backed by RedHat for building Docker
images, and that solves Docker-in-Docker security issues (and also speeds-up build times), and can also be configured to run rootless.
images, and that solves Docker-in-Docker security issues (and also speeds-up build times), and can also be configured to run rootless.
By default, the template uses the [kaniko](https://docs.gitlab.com/ee/ci/docker/using_kaniko.html) way, but you may
By default, the template uses the [kaniko](https://docs.gitlab.com/ee/ci/docker/using_kaniko.html) way, but you may
select an alternate build tool by using the `DOCKER_BUILD_TOOL` variable (see below).
select an alternate build tool by using the `DOCKER_BUILD_TOOL` variable (see below).
...
@@ -56,24 +56,23 @@ select an alternate build tool by using the `DOCKER_BUILD_TOOL` variable (see be
...
@@ -56,24 +56,23 @@ select an alternate build tool by using the `DOCKER_BUILD_TOOL` variable (see be
The Docker template uses some global configuration used throughout all jobs.
The Docker template uses some global configuration used throughout all jobs.
| Input / Variable | Description | Default value |
| Input / Variable | Description | Default value |
| `build-tool` / `DOCKER_BUILD_TOOL` | The build tool to use for building container image, possible values are `kaniko`, `buildah` or `dind`| `kaniko` |
| `build-tool` / `DOCKER_BUILD_TOOL`| The build tool to use for building container image, possible values are `kaniko`, `buildah` or `dind` | `kaniko` |
| `kaniko-image` / `DOCKER_KANIKO_IMAGE` | The image used to run `kaniko` - _for kaniko build only_ | `gcr.io/kaniko-project/executor:debug` (use `debug` images for GitLab) |
| `kaniko-image` / `DOCKER_KANIKO_IMAGE`| The image used to run `kaniko` - _for kaniko build only_ | `gcr.io/kaniko-project/executor:debug` (use `debug` images for GitLab) |
| `buildah-image` / `DOCKER_BUILDAH_IMAGE` | The image used to run `buildah` - _for buildah build only_ | `quay.io/buildah/stable` |
| `buildah-image` / `DOCKER_BUILDAH_IMAGE` | The image used to run `buildah` - _for buildah build only_ | `quay.io/buildah/stable` |
| `image` / `DOCKER_IMAGE` | The Docker image used to run the docker client (see [full list](https://hub.docker.com/r/library/docker/)) - _for Docker-in-Docker build only_ | `registry.hub.docker.com/library/docker:latest` |
| `image` / `DOCKER_IMAGE`| The Docker image used to run the docker client (see [full list](https://hub.docker.com/r/library/docker/)) - _for Docker-in-Docker build only_ | `registry.hub.docker.com/library/docker:latest`|
| `dind-image` / `DOCKER_DIND_IMAGE` | The Docker image used to run the Docker daemon (see [full list](https://hub.docker.com/r/library/docker/)) - _for Docker-in-Docker build only_ | `registry.hub.docker.com/library/docker:dind` |
| `dind-image` / `DOCKER_DIND_IMAGE`| The Docker image used to run the Docker daemon (see [full list](https://hub.docker.com/r/library/docker/)) - _for Docker-in-Docker build only_ | `registry.hub.docker.com/library/docker:dind`|
| `file` / `DOCKER_FILE` | The path to your `Dockerfile`| `Dockerfile` |
| `file` / `DOCKER_FILE`| The path to your `Dockerfile` | `Dockerfile` |
| `context-path` / `DOCKER_CONTEXT_PATH` | The Docker [context path](https://docs.docker.com/engine/reference/commandline/build/#build-with-path)(working directory) | _none_ _only set if you want a context path different from the Dockerfile location_ |
| `context-path` / `DOCKER_CONTEXT_PATH`| The Docker [context path](https://docs.docker.com/engine/reference/commandline/build/#build-with-path)(working directory) | _none_ _only set if you want a context path different from the Dockerfile location_ |
In addition to this, the template supports _standard_ Linux proxy variables:
In addition to this, the template supports _standard_ Linux proxy variables:
| Input / Variable | Description | Default value |
| Input / Variable | Description | Default value |
As you can see, the Docker template is configured by default to use the GitLab container registry.
As you can see, the Docker template is configured by default to use the GitLab container registry.
You may perfectly override this and use another Docker registry, but be aware of a few things:
You may perfectly override this and use another Docker registry, but be aware of a few things:
* the `DOCKER_SNAPSHOT_IMAGE` requires a Docker registry that allows tag overwrite,
- the `DOCKER_SNAPSHOT_IMAGE` requires a Docker registry that allows tag overwrite,
* the `DOCKER_RELEASE_IMAGE` _may_ use a Docker registry that doesn't allow tag overwrite, but:
- the `DOCKER_RELEASE_IMAGE` _may_ use a Docker registry that doesn't allow tag overwrite, but:
1. you should avoid overwriting a Git tag (at it will obviously fail while trying to (re)push the Docker image),
1. you should avoid overwriting a Git tag (at it will obviously fail while trying to (re)push the Docker image),
2. you have to deactivate publish on `main` (or `master`) branch by setting the `$DOCKER_PROD_PUBLISH_STRATEGY` variable to `none` (as it would lead to the `main` tag being overwritten).
2. you have to deactivate publish on `main` (or `master`) branch by setting the `$DOCKER_PROD_PUBLISH_STRATEGY` variable to `none` (as it would lead to the `main` tag being overwritten).
### Registries and credentials
### Registries and credentials
...
@@ -121,38 +120,38 @@ If you use the **same registry** for both snapshot and release images, you shall
...
@@ -121,38 +120,38 @@ If you use the **same registry** for both snapshot and release images, you shall
#### Setting your own Docker configuration file (advanced)
#### Setting your own Docker configuration file (advanced)
There might be cases where you need to provide the complete [Docker configuration file](https://docs.docker.com/engine/reference/commandline/cli/#configuration-files):
There might be cases where you need to provide the complete [Docker configuration file](https://docs.docker.com/engine/reference/commandline/cli/#configuration-files):
* need to declare authentication credentials for other registries than the 2 predefined ones (snapshot & release),
- need to declare authentication credentials for other registries than the 2 predefined ones (snapshot & release),
* need to declare a [credentials store](https://docs.docker.com/engine/reference/commandline/login/#credentials-store)(ex: in order to [publish to Amazon ECR](https://github.com/GoogleContainerTools/kaniko#pushing-to-amazon-ecr) with Kaniko for instance),
- need to declare a [credentials store](https://docs.docker.com/engine/reference/commandline/login/#credentials-store)(ex: in order to [publish to Amazon ECR](https://github.com/GoogleContainerTools/kaniko#pushing-to-amazon-ecr) with Kaniko for instance),
* need to declare [proxies](https://docs.docker.com/engine/reference/commandline/cli/#automatic-proxy-configuration-for-containers),
- need to declare [proxies](https://docs.docker.com/engine/reference/commandline/cli/#automatic-proxy-configuration-for-containers),
* ...
- ...
If you are in one of those cases, you will need to use the `DOCKER_CONFIG_FILE` variable, expected to declare the path to your custom Docker configuration file (JSON). You may:
If you are in one of those cases, you will need to use the `DOCKER_CONFIG_FILE` variable, expected to declare the path to your custom Docker configuration file (JSON). You may:
* leave the default value (`.docker/config.json`) or override it to some alternate location in your project repository and create the file **without any secret in it** using our dynamic variables replacement (see below),
- leave the default value (`.docker/config.json`) or override it to some alternate location in your project repository and create the file **without any secret in it** using our dynamic variables replacement (see below),
* or override it as a GitLab project variable of type [File](https://docs.gitlab.com/ee/ci/variables/#cicd-variable-types), possibly inlining your secret credentials in it.
- or override it as a GitLab project variable of type [File](https://docs.gitlab.com/ee/ci/variables/#cicd-variable-types), possibly inlining your secret credentials in it.
| Input / Variable | Description | Default value |
| Input / Variable | Description | Default value |
| `config-file` / `DOCKER_CONFIG_FILE` | Path to the Docker configuration file (JSON) | `.docker/config.json` |
| `config-file` / `DOCKER_CONFIG_FILE` | Path to the Docker configuration file (JSON) | `.docker/config.json` |
Moreover, this file supports **dynamic environment variables replacement**.
Moreover, this file supports **dynamic environment variables replacement**.
...
@@ -160,57 +159,56 @@ That means it may contain references to other environment variables (in the form
...
@@ -160,57 +159,56 @@ That means it may contain references to other environment variables (in the form
by the template before evaluation.
by the template before evaluation.
In addition to you own defined variables, you may use the following variables (provided and managed by the template):
In addition to you own defined variables, you may use the following variables (provided and managed by the template):
*`${docker_snapshot_authent_token}`: the authentication token required by the snapshot registry (computed from configured `DOCKER_REGISTRY_SNAPSHOT_USER` / `DOCKER_REGISTRY_SNAPSHOT_PASSWORD` variables)
-`${docker_snapshot_authent_token}`: the authentication token required by the snapshot registry (computed from configured `DOCKER_REGISTRY_SNAPSHOT_USER` / `DOCKER_REGISTRY_SNAPSHOT_PASSWORD` variables)
*`${docker_snapshot_registry_host}`: the snapshot registry host (based on the configured `DOCKER_SNAPSHOT_IMAGE` variable)
-`${docker_snapshot_registry_host}`: the snapshot registry host (based on the configured `DOCKER_SNAPSHOT_IMAGE` variable)
*`${docker_release_authent_token}`: the authentication token required by the release registry (computed from configured `DOCKER_REGISTRY_RELEASE_USER` / `DOCKER_REGISTRY_RELEASE_PASSWORD` variables)
-`${docker_release_authent_token}`: the authentication token required by the release registry (computed from configured `DOCKER_REGISTRY_RELEASE_USER` / `DOCKER_REGISTRY_RELEASE_PASSWORD` variables)
*`${docker_release_registry_host}`: the release registry host (based on the configured `DOCKER_RELEASE_IMAGE` variable)
-`${docker_release_registry_host}`: the release registry host (based on the configured `DOCKER_RELEASE_IMAGE` variable)
Example 1: Docker configuration file inlined in the project repository (`.docker/config.json`) with **dynamic variables replacement**:
Example 1: Docker configuration file inlined in the project repository (`.docker/config.json`) with **dynamic variables replacement**:
```json
```json
{
{
"auths":{
"auths":{
"${docker_snapshot_registry_host}":{
"${docker_snapshot_registry_host}":{
"auth":"${docker_release_authent_token}"
"auth":"${docker_release_authent_token}"
},
},
"${docker_release_registry_host}":{
"${docker_release_registry_host}":{
"auth":"${docker_snapshot_authent_token}"
"auth":"${docker_snapshot_authent_token}"
},
},
"my-readonly-repo-to-pull":{
"my-readonly-repo-to-pull":{
"auth":"${MY_OWN_REGISTRY_TOKEN}"
"auth":"${MY_OWN_REGISTRY_TOKEN}"
}
}
}
}
}
}
```
```
This file uses:
This file uses:
* template-managed `${docker_snapshot_authent_token}`, `${docker_snapshot_registry_host}`, `${docker_release_authent_token}` and `${docker_release_registry_host}` variables,
- template-managed `${docker_snapshot_authent_token}`, `${docker_snapshot_registry_host}`, `${docker_release_authent_token}` and `${docker_release_registry_host}` variables,
* the user-defined `${MY_OWN_REGISTRY_TOKEN}` (:information_source: an authentication token can be obtained with command `echo "user:password" | base64` and then be stored as a masked GitLab CI/CD project variable).
- the user-defined `${MY_OWN_REGISTRY_TOKEN}` (:information_source: an authentication token can be obtained with command `echo "user:password" | base64` and then be stored as a masked GitLab CI/CD project variable).
Example 2: Docker configuration file declared as a GitLab project variable of type [File](https://docs.gitlab.com/ee/ci/variables/#cicd-variable-types) with **dynamic variables replacement**:
Example 2: Docker configuration file declared as a GitLab project variable of type [File](https://docs.gitlab.com/ee/ci/variables/#cicd-variable-types) with **dynamic variables replacement**:
```json
```json
{
{
"auths":{
"auths":{
"$${docker_snapshot_registry_host}":{
"$${docker_snapshot_registry_host}":{
"auth":"$${docker_release_authent_token}"
"auth":"$${docker_release_authent_token}"
},
},
"$${docker_release_registry_host}":{
"$${docker_release_registry_host}":{
"auth":"$${docker_snapshot_authent_token}"
"auth":"$${docker_snapshot_authent_token}"
},
},
"my-readonly-repo-to-pull":{
"my-readonly-repo-to-pull":{
"auth":"ZG9ja2VyZHVkZTpnb3RjaGEh"
"auth":"ZG9ja2VyZHVkZTpnb3RjaGEh"
}
}
}
}
}
}
```
```
This file uses:
This file uses:
* template-managed `${docker_snapshot_authent_token}`, `${docker_snapshot_registry_host}`, `${docker_release_authent_token}` and `${docker_release_registry_host}` variables (:warning: mind the double `$$` to prevent GitLab from [trying to evaluate the variable](https://docs.gitlab.com/ee/ci/variables/index.html#use-the--character-in-variables)),
- template-managed `${docker_snapshot_authent_token}`, `${docker_snapshot_registry_host}`, `${docker_release_authent_token}` and `${docker_release_registry_host}` variables (:warning: mind the double `$$` to prevent GitLab from [trying to evaluate the variable](https://docs.gitlab.com/ee/ci/variables/index.html#use-the--character-in-variables)),
* the user-defined authentication may be inlined as a GitLab project variable is a place safe enough to store secrets.
- the user-defined authentication may be inlined as a GitLab project variable is a place safe enough to store secrets.
## Multi Dockerfile support
## Multi Dockerfile support
...
@@ -231,12 +229,12 @@ You can do so by adding a patch to the `.docker-base` job in your `.gitlab-ci.ym
...
@@ -231,12 +229,12 @@ You can do so by adding a patch to the `.docker-base` job in your `.gitlab-ci.ym
If you need to redefine a variable with the same value for all your Dockerfiles, you can just declare this variable as a global variable. For example, if you want to build all your images using `buildah`, you can simply define the `DOCKER_BUILD_TOOL` variable as a global variable with value `buildah`:
If you need to redefine a variable with the same value for all your Dockerfiles, you can just declare this variable as a global variable. For example, if you want to build all your images using `buildah`, you can simply define the `DOCKER_BUILD_TOOL` variable as a global variable with value `buildah`:
...
@@ -251,13 +249,13 @@ variables:
...
@@ -251,13 +249,13 @@ variables:
Here are some advices about your **secrets** (variables marked with a :lock:):
Here are some advices about your **secrets** (variables marked with a :lock:):
1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#for-a-project):
1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#for-a-project):
*[**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable) to prevent them from being inadvertently
-[**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable) to prevent them from being inadvertently
displayed in your job logs,
displayed in your job logs,
*[**protected**](https://docs.gitlab.com/ee/ci/variables/#protected-cicd-variables) if you want to secure some secrets
-[**protected**](https://docs.gitlab.com/ee/ci/variables/#protected-cicd-variables) if you want to secure some secrets
you don't want everyone in the project to have access to (for instance production secrets).
you don't want everyone in the project to have access to (for instance production secrets).
2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable),
2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable),
simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`:
simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`:
it will then be possible to mask it and the template will automatically decode it prior to using it.
it will then be possible to mask it and the template will automatically decode it prior to using it.
3. Don't forget to escape special characters (ex: `$` -> `$$`).
3. Don't forget to escape special characters (ex: `$` -> `$$`).
## Jobs
## Jobs
...
@@ -268,11 +266,11 @@ This job performs a [Lint](https://github.com/hadolint/hadolint) on your `Docker
...
@@ -268,11 +266,11 @@ This job performs a [Lint](https://github.com/hadolint/hadolint) on your `Docker
It is bound to the `build` stage, and uses the following variables:
It is bound to the `build` stage, and uses the following variables:
| Input / Variable | Description | Default value |
| Input / Variable | Description | Default value |
In case you have to disable some rules, either add `--ignore XXXX` to the `DOCKER_HADOLINT_ARGS` variable or create a [Hadolint configuration file](https://github.com/hadolint/hadolint#configure) named `hadolint.yaml` at the root of your repository.
In case you have to disable some rules, either add `--ignore XXXX` to the `DOCKER_HADOLINT_ARGS` variable or create a [Hadolint configuration file](https://github.com/hadolint/hadolint#configure) named `hadolint.yaml` at the root of your repository.
...
@@ -288,10 +286,10 @@ RUN cd /tmp && echo "hello!"
...
@@ -288,10 +286,10 @@ RUN cd /tmp && echo "hello!"
In addition to a textual report in the console, this job produces the following reports, kept for one day:
In addition to a textual report in the console, this job produces the following reports, kept for one day:
| `reports/docker-hadolint-*.native.json` | native hadolint test report (json) | [DefectDojo integration](https://defectdojo.github.io/django-DefectDojo/integrations/parsers/#hadolint)<br/>_This report is generated only if DefectDojo template is detected_ |
| `reports/docker-hadolint-*.native.json` | native hadolint test report (json) | [DefectDojo integration](https://defectdojo.github.io/django-DefectDojo/integrations/parsers/#hadolint)<br/>_This report is generated only if DefectDojo template is detected_ |
| `registry-mirror` / `DOCKER_REGISTRY_MIRROR` | URL of a Docker registry mirror to use during the image build (instead of default `https://index.docker.io`) <br>:warning: Used by the `kaniko` and `dind` options only | _(none)_ |
| `registry-mirror` / `DOCKER_REGISTRY_MIRROR`| URL of a Docker registry mirror to use during the image build (instead of default `https://index.docker.io`) <br>:warning: Used by the `kaniko` and `dind` options only | _(none)_ |
| `container-registries-config-file` / `CONTAINER_REGISTRIES_CONFIG_FILE` | The [`registries.conf`](https://www.redhat.com/sysadmin/manage-container-registries) configuration to be used<br>:warning: Used by the `buildah` build only | _(none)_ |
| `container-registries-config-file` / `CONTAINER_REGISTRIES_CONFIG_FILE` | The [`registries.conf`](https://www.redhat.com/sysadmin/manage-container-registries) configuration to be used<br>:warning: Used by the `buildah` build only | _(none)_ |
| `metadata` / `DOCKER_METADATA` | Additional `docker build`/`kaniko` arguments to set label | OCI Image Format Specification |
| `metadata` / `DOCKER_METADATA`| Additional `docker build`/`kaniko` arguments to set label | OCI Image Format Specification |
| `kaniko-snapshot-image-cache` / `KANIKO_SNAPSHOT_IMAGE_CACHE` | Snapshot image repository that will be used to store cached layers (leave empty to use default: snapshot image repository + `/cache`)<br>:warning: Used by the `kaniko` build only | _none_ (default cache path) |
| `kaniko-snapshot-image-cache` / `KANIKO_SNAPSHOT_IMAGE_CACHE`| Snapshot image repository that will be used to store cached layers (leave empty to use default: snapshot image repository + `/cache`)<br>:warning: Used by the `kaniko` build only | _none_ (default cache path) |
| `build-cache-disabled` / `DOCKER_BUILD_CACHE_DISABLED` | Set to `true` to disable the build cache.<br/>Cache can typically be disabled when there is a network latency between the container registry and the runner. | _none_ (i.e cache enabled) |
| `build-cache-disabled` / `DOCKER_BUILD_CACHE_DISABLED`| Set to `true` to disable the build cache.<br/>Cache can typically be disabled when there is a network latency between the container registry and the runner. | _none_ (i.e cache enabled) |
This job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
This job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
| `healthcheck-disabled` / `DOCKER_HEALTHCHECK_DISABLED` | Set to `true` to disable health check | _(none: enabled by default)_ |
| `healthcheck-disabled` / `DOCKER_HEALTHCHECK_DISABLED`| Set to `true` to disable health check | _(none: enabled by default)_ |
| `healthcheck-timeout` / `DOCKER_HEALTHCHECK_TIMEOUT` | When testing a Docker Health (test stage), how long (in seconds) wait for the [HealthCheck status](https://docs.docker.com/engine/reference/builder/#healthcheck) | `60` |
| `healthcheck-timeout` / `DOCKER_HEALTHCHECK_TIMEOUT`| When testing a Docker Health (test stage), how long (in seconds) wait for the [HealthCheck status](https://docs.docker.com/engine/reference/builder/#healthcheck) | `60`|
| `healthcheck-options` / `DOCKER_HEALTHCHECK_OPTIONS` | Docker options for health check such as port mapping, environment... | _(none)_ |
| `healthcheck-options` / `DOCKER_HEALTHCHECK_OPTIONS`| Docker options for health check such as port mapping, environment... | _(none)_ |
| `healthcheck-container-args` / `DOCKER_HEALTHCHECK_CONTAINER_ARGS` | Set arguments sent to the running container for health check | _(none)_ |
| `healthcheck-container-args` / `DOCKER_HEALTHCHECK_CONTAINER_ARGS` | Set arguments sent to the running container for health check | _(none)_ |
In case your Docker image is not intended to run as a service and only contains a *client tool* (like curl, Ansible, ...) you can test it by overriding the Health Check Job. See [this example](#overriding-docker-healthcheck).
In case your Docker image is not intended to run as a service and only contains a _client tool_ (like curl, Ansible, ...) you can test it by overriding the Health Check Job. See [this example](#overriding-docker-healthcheck).
:warning: Keep in mind that the downloading of the snapshot image by the GitLab runner will be done during the waiting time (max `DOCKER_HEALTHCHECK_TIMEOUT`).
:warning: Keep in mind that the downloading of the snapshot image by the GitLab runner will be done during the waiting time (max `DOCKER_HEALTHCHECK_TIMEOUT`).
In case your image takes quite some time to be downloaded by the runner, increase the value of `DOCKER_HEALTHCHECK_TIMEOUT` in your `.gitlab-ci.yml` file.
In case your image takes quite some time to be downloaded by the runner, increase the value of `DOCKER_HEALTHCHECK_TIMEOUT` in your `.gitlab-ci.yml` file.
...
@@ -411,22 +409,22 @@ variables:
...
@@ -411,22 +409,22 @@ variables:
It is bound to the `package-test` stage, and uses the following variables:
It is bound to the `package-test` stage, and uses the following variables:
| Input / Variable | Description | Default value |
| Input / Variable | Description | Default value |
| `reports/docker-trivy-*.native.json` | native Trivy report format (json) | [DefectDojo integration](https://defectdojo.github.io/django-DefectDojo/integrations/parsers/#trivy)<br/>_This report is generated only if DefectDojo template is detected_ |
| `reports/docker-trivy-*.native.json` | native Trivy report format (json) | [DefectDojo integration](https://defectdojo.github.io/django-DefectDojo/integrations/parsers/#trivy)<br/>_This report is generated only if DefectDojo template is detected_ |
| `reports/docker-trivy-*.gitlab.json` | [Trivy report format for GitLab](https://aquasecurity.github.io/trivy/v0.30.4/docs/integrations/gitlab-ci/) format | [GitLab integration](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportscontainer_scanning) |
| `reports/docker-trivy-*.gitlab.json` | [Trivy report format for GitLab](https://aquasecurity.github.io/trivy/v0.30.4/docs/integrations/gitlab-ci/) format | [GitLab integration](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportscontainer_scanning)|
### `docker-sbom` job
### `docker-sbom` job
...
@@ -434,34 +432,34 @@ This job generates a [SBOM](https://cyclonedx.org/) file listing installed packa
...
@@ -434,34 +432,34 @@ This job generates a [SBOM](https://cyclonedx.org/) file listing installed packa
It is bound to the `package-test` stage, and uses the following variables:
It is bound to the `package-test` stage, and uses the following variables:
| Input / Variable | Description | Default value |
| Input / Variable | Description | Default value |
| `sbom-disabled` / `DOCKER_SBOM_DISABLED` | Set to `true` to disable this job | _none_ |
| `sbom-disabled` / `DOCKER_SBOM_DISABLED` | Set to `true` to disable this job | _none_ |
| `sbom-image` / `DOCKER_SBOM_IMAGE` | The docker image used to emit SBOM | `registry.hub.docker.com/anchore/syft:debug` |
| `sbom-image` / `DOCKER_SBOM_IMAGE`| The docker image used to emit SBOM | `registry.hub.docker.com/anchore/syft:debug`|
| `sbom-opts` / `DOCKER_SBOM_OPTS` | Options for syft used for SBOM analysis | `--override-default-catalogers rpm-db-cataloger,alpm-db-cataloger,apk-db-cataloger,dpkg-db-cataloger,portage-cataloger` |
| `sbom-opts` / `DOCKER_SBOM_OPTS`| Options for syft used for SBOM analysis | `--override-default-catalogers rpm-db-cataloger,alpm-db-cataloger,apk-db-cataloger,dpkg-db-cataloger,portage-cataloger` |
### `docker-publish` job
### `docker-publish` job
This job pushes (_promotes_) the built image as the _release_ image [skopeo](https://github.com/containers/skopeo).
This job pushes (_promotes_) the built image as the _release_ image [skopeo](https://github.com/containers/skopeo).
| Input / Variable | Description | Default value |
| Input / Variable | Description | Default value |
| `skopeo-image` / `DOCKER_SKOPEO_IMAGE` | The Docker image used to run [skopeo](https://github.com/containers/skopeo)| `quay.io/skopeo/stable:latest` |
| `skopeo-image` / `DOCKER_SKOPEO_IMAGE`| The Docker image used to run [skopeo](https://github.com/containers/skopeo) | `quay.io/skopeo/stable:latest` |
| `prod-publish-strategy` / `DOCKER_PROD_PUBLISH_STRATEGY` | Defines the publish to production strategy. One of `manual` (i.e. _one-click_), `auto` or `none` (disabled). | `manual` |
| `prod-publish-strategy` / `DOCKER_PROD_PUBLISH_STRATEGY`| Defines the publish to production strategy. One of `manual` (i.e. _one-click_), `auto` or `none` (disabled). | `manual` |
| `release-extra-tags-pattern` / `DOCKER_RELEASE_EXTRA_TAGS_PATTERN` | Defines the image tag pattern that `$DOCKER_RELEASE_IMAGE` should match to push extra tags (supports capturing groups - [see below](#using-extra-tags)) | `^v?(?P<major>[0-9]+)\\.(?P<minor>[0-9]+)\\.(?P<patch>[0-9]+)(?P<suffix>(?P<prerelease>-[0-9A-Za-z-\\.]+)?(?P<build>\\+[0-9A-Za-z-\\.]+)?)$` _(SemVer pattern)_ |
| `release-extra-tags-pattern` / `DOCKER_RELEASE_EXTRA_TAGS_PATTERN` | Defines the image tag pattern that `$DOCKER_RELEASE_IMAGE` should match to push extra tags (supports capturing groups - [see below](#using-extra-tags)) | `^v?(?P<major>[0-9]+)\\.(?P<minor>[0-9]+)\\.(?P<patch>[0-9]+)(?P<suffix>(?P<prerelease>-[0-9A-Za-z-\\.]+)?(?P<build>\\+[0-9A-Za-z-\\.]+)?)$` _(SemVer pattern)_ |
| `release-extra-tags` / `DOCKER_RELEASE_EXTRA_TAGS` | Defines extra tags to publish the _release_ image (supports capturing group references from `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` - [see below](#using-extra-tags)) | _(none)_ |
| `release-extra-tags` / `DOCKER_RELEASE_EXTRA_TAGS`| Defines extra tags to publish the _release_ image (supports capturing group references from `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` - [see below](#using-extra-tags)) | _(none)_ |
| `semrel-release-disabled` / `DOCKER_SEMREL_RELEASE_DISABLED` | Set to `true` to disable [semantic-release integration](#semantic-release-integration)| _none_ (enabled) |
| `semrel-release-disabled` / `DOCKER_SEMREL_RELEASE_DISABLED`| Set to `true` to disable [semantic-release integration](#semantic-release-integration) | _none_ (enabled) |
This job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
This job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
They may be freely used in downstream jobs (for instance to deploy the upstream built Docker image, whatever the branch or tag).
They may be freely used in downstream jobs (for instance to deploy the upstream built Docker image, whatever the branch or tag).
...
@@ -469,26 +467,26 @@ They may be freely used in downstream jobs (for instance to deploy the upstream
...
@@ -469,26 +467,26 @@ They may be freely used in downstream jobs (for instance to deploy the upstream
When publishing the _release_ image, the Docker template might publish it again with additional tags (aliases):
When publishing the _release_ image, the Docker template might publish it again with additional tags (aliases):
* the original published image tag (extracted from `$DOCKER_RELEASE_IMAGE`) must match `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` ([semantic versioning](https://semver.org/) pattern by default),
- the original published image tag (extracted from `$DOCKER_RELEASE_IMAGE`) must match `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` ([semantic versioning](https://semver.org/) pattern by default),
* extra tags to publish can be defined in the `$DOCKER_RELEASE_EXTRA_TAGS` variable, each separated with a whitespace.
- extra tags to publish can be defined in the `$DOCKER_RELEASE_EXTRA_TAGS` variable, each separated with a whitespace.
:information_source: the Docker template supports [group references substitution](https://learnbyexample.github.io/py_regular_expressions/groupings-and-backreferences.html) to evaluate extra tags:
:information_source: the Docker template supports [group references substitution](https://learnbyexample.github.io/py_regular_expressions/groupings-and-backreferences.html) to evaluate extra tags:
*`v([0-9]+)\.([0-9]+)\.([0-9]+)` has 3 (unnamed) capturing groups, each capturing any number of digits
-`v([0-9]+)\.([0-9]+)\.([0-9]+)` has 3 (unnamed) capturing groups, each capturing any number of digits
*`v(?<major>[0-9]+)\.(?<minor>[0-9]+)\.(?<patch>[0-9]+)` has 3 **named** capturing groups (_major_, _minor_ and _patch_), each capturing any number of digits
-`v(?<major>[0-9]+)\.(?<minor>[0-9]+)\.(?<patch>[0-9]+)` has 3 **named** capturing groups (_major_, _minor_ and _patch_), each capturing any number of digits
*`$DOCKER_RELEASE_EXTRA_TAGS` supports capturing group references from `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN`:
-`$DOCKER_RELEASE_EXTRA_TAGS` supports capturing group references from `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN`:
*`\g1` is a reference to capturing group number 1
-`\g1` is a reference to capturing group number 1
*`\g<major>` is a reference to capturing group named _major_
-`\g<major>` is a reference to capturing group named _major_
:information_source: the default value of `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` matches and captures all parts of a standard [semantic versioning](https://semver.org/)-compliant tag:
:information_source: the default value of `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` matches and captures all parts of a standard [semantic versioning](https://semver.org/)-compliant tag:
* the **major** group captures the major version
- the **major** group captures the major version
* the **minor** group captures the minor version
- the **minor** group captures the minor version
* the **patch** group captures the patch version
- the **patch** group captures the patch version
* the **prerelease** group captures the (optional) pre-release version (including the leading `-`)
- the **prerelease** group captures the (optional) pre-release version (including the leading `-`)
* the **build** group captures the (optional) build version (including the leading `+`)
- the **build** group captures the (optional) build version (including the leading `+`)
* the **suffix** group captures the (optional) entire suffix (including pre-release and/or build)
- the **suffix** group captures the (optional) entire suffix (including pre-release and/or build)
Example: publish latest, major.minor and major aliases for a SemVer release:
Example: publish latest, major.minor and major aliases for a SemVer release:
...
@@ -500,21 +498,21 @@ variables:
...
@@ -500,21 +498,21 @@ variables:
With this contiguration, the following extra tags would be published:
With this contiguration, the following extra tags would be published:
If you activate the [`semantic-release-info` job from the `semantic-release` template](https://gitlab.com/to-be-continuous/semantic-release/#semantic-release-info-job), the `docker-publish` job will rely on the generated next version info.
If you activate the [`semantic-release-info` job from the `semantic-release` template](https://gitlab.com/to-be-continuous/semantic-release/#semantic-release-info-job), the `docker-publish` job will rely on the generated next version info.
* the release will only be performed if a semantic release is present
- the release will only be performed if a semantic release is present
* the tag will be based on `SEMREL_INFO_NEXT_VERSION`, it will override `DOCKER_RELEASE_IMAGE` by simply substituting the tag, or adding a tag when there's none.
- the tag will be based on `SEMREL_INFO_NEXT_VERSION`, it will override `DOCKER_RELEASE_IMAGE` by simply substituting the tag, or adding a tag when there's none.
The published Docker image will be `registry.gitlab.com/$CI_PROJECT_NAME:$SEMREL_INFO_NEXT_VERSION` and all subsequent jobs relying on the `docker_image` variable will be provided with this tag.
The published Docker image will be `registry.gitlab.com/$CI_PROJECT_NAME:$SEMREL_INFO_NEXT_VERSION` and all subsequent jobs relying on the `docker_image` variable will be provided with this tag.
:warning: When `semantic-release` detects no release (i.e. either the semantic-release template is misconfigured, or there were simply no `feat`/`fix` commits), the `docker-publish` job will report a warning and *no image will be pushed* in the release registry. In such a case, the `docker_image` remains unchanged, and will refer to the snapshot image. Any subsequent job that may deploy to production (Kubernetes or Openshift), should thus be configured *not to deploy* in this situation. Refer to deployment template for more information.
:warning: When `semantic-release` detects no release (i.e. either the semantic-release template is misconfigured, or there were simply no `feat`/`fix` commits), the `docker-publish` job will report a warning and _no image will be pushed_ in the release registry. In such a case, the `docker_image` remains unchanged, and will refer to the snapshot image. Any subsequent job that may deploy to production (Kubernetes or Openshift), should thus be configured _not to deploy_ in this situation. Refer to deployment template for more information.
Finally, the semantic-release integration can be disabled with the `DOCKER_SEMREL_RELEASE_DISABLED` variable.
Finally, the semantic-release integration can be disabled with the `DOCKER_SEMREL_RELEASE_DISABLED` variable.
...
@@ -544,7 +543,7 @@ Here is a `.gitlab-ci.yaml` using an external Docker registry:
...
@@ -544,7 +543,7 @@ Here is a `.gitlab-ci.yaml` using an external Docker registry:
| `TBC_VAULT_IMAGE` | The [Vault Secrets Provider](https://gitlab.com/to-be-continuous/tools/vault-secrets-provider) image to use (can be overridden) | `registry.gitlab.com/to-be-continuous/tools/vault-secrets-provider:latest` |
| `TBC_VAULT_IMAGE`| The [Vault Secrets Provider](https://gitlab.com/to-be-continuous/tools/vault-secrets-provider) image to use (can be overridden) | `registry.gitlab.com/to-be-continuous/tools/vault-secrets-provider:latest` |
| `vault-base-url` / `VAULT_BASE_URL` | The Vault server base API url | _none_ |
| `vault-base-url` / `VAULT_BASE_URL` | The Vault server base API url | _none_ |
| `vault-oidc-aud` / `VAULT_OIDC_AUD` | The `aud` claim for the JWT | `$CI_SERVER_URL` |
| `vault-oidc-aud` / `VAULT_OIDC_AUD` | The `aud` claim for the JWT | `$CI_SERVER_URL` |
| :lock: `VAULT_ROLE_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) RoleID | **must be defined** |
| :lock: `VAULT_ROLE_ID`| The [AppRole](https://www.vaultproject.io/docs/auth/approle) RoleID | **must be defined** |
| :lock: `VAULT_SECRET_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) SecretID | **must be defined** |
| :lock: `VAULT_SECRET_ID`| The [AppRole](https://www.vaultproject.io/docs/auth/approle) SecretID | **must be defined** |
#### Usage
#### Usage
...
@@ -602,9 +601,9 @@ Then you may retrieve any of your secret(s) from Vault using the following synta
...
@@ -602,9 +601,9 @@ Then you may retrieve any of your secret(s) from Vault using the following synta
| `TBC_GCP_PROVIDER_IMAGE` | The [GCP Auth Provider](https://gitlab.com/to-be-continuous/tools/gcp-auth-provider) image to use (can be overridden) | `registry.gitlab.com/to-be-continuous/tools/gcp-auth-provider:latest` |
| `TBC_GCP_PROVIDER_IMAGE`| The [GCP Auth Provider](https://gitlab.com/to-be-continuous/tools/gcp-auth-provider) image to use (can be overridden) | `registry.gitlab.com/to-be-continuous/tools/gcp-auth-provider:latest` |
| `gcp-oidc-aud` / `GCP_OIDC_AUD` | The `aud` claim for the JWT token | `$CI_SERVER_URL` |
| `gcp-oidc-aud` / `GCP_OIDC_AUD`| The `aud` claim for the JWT token | `$CI_SERVER_URL` |
| `gcp-oidc-provider` / `GCP_OIDC_PROVIDER` | Default Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) | _none_ |
| `gcp-oidc-provider` / `GCP_OIDC_PROVIDER`| Default Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) | _none_ |
| `gcp-oidc-account` / `GCP_OIDC_ACCOUNT` | Default Service Account to which impersonate with OpenID Connect authentication | _none_ |
| `gcp-oidc-account` / `GCP_OIDC_ACCOUNT`| Default Service Account to which impersonate with OpenID Connect authentication | _none_ |
| `gcp-snapshot-oidc-provider` / `GCP_SNAPSHOT_OIDC_PROVIDER` | Workload Identity Provider to push the snapshot image _(only define to override default)_ | _none_ |
| `gcp-snapshot-oidc-provider` / `GCP_SNAPSHOT_OIDC_PROVIDER` | Workload Identity Provider to push the snapshot image _(only define to override default)_ | _none_ |
| `gcp-snapshot-oidc-account` / `GCP_SNAPSHOT_OIDC_ACCOUNT` | Service Account to use to push the snapshot image _(only define to override default)_ | _none_ |
| `gcp-snapshot-oidc-account` / `GCP_SNAPSHOT_OIDC_ACCOUNT`| Service Account to use to push the snapshot image _(only define to override default)_ | _none_ |
| `gcp-release-oidc-provider` / `GCP_RELEASE_OIDC_PROVIDER` | Workload Identity Provider to push the release image _(only define to override default)_ | _none_ |
| `gcp-release-oidc-provider` / `GCP_RELEASE_OIDC_PROVIDER`| Workload Identity Provider to push the release image _(only define to override default)_ | _none_ |
| `gcp-release-oidc-account` / `GCP_RELEASE_OIDC_ACCOUNT` | Service Account to use to push the release image _(only define to override default)_ | _none_ |
| `gcp-release-oidc-account` / `GCP_RELEASE_OIDC_ACCOUNT`| Service Account to use to push the release image _(only define to override default)_ | _none_ |
:warning: if using Kaniko, don't forget to either create the cache repository (snapshot image repository + `/cache`) or override `$KANIKO_SNAPSHOT_IMAGE_CACHE`
:warning: if using Kaniko, don't forget to either create the cache repository (snapshot image repository + `/cache`) or override `$KANIKO_SNAPSHOT_IMAGE_CACHE`
to use the snapshot image repository (will host your snapshot image as well as cached layers).
to use the snapshot image repository (will host your snapshot image as well as cached layers).
...
@@ -661,7 +660,7 @@ to use the snapshot image repository (will host your snapshot image as well as c
...
@@ -661,7 +660,7 @@ to use the snapshot image repository (will host your snapshot image as well as c
| `TBC_AWS_PROVIDER_IMAGE` | The [AWS Auth Provider](https://gitlab.com/to-be-continuous/tools/aws-auth-provider) image to use (can be overridden) | `registry.gitlab.com/to-be-continuous/tools/aws-auth-provider:latest` |
| `TBC_AWS_PROVIDER_IMAGE`| The [AWS Auth Provider](https://gitlab.com/to-be-continuous/tools/aws-auth-provider) image to use (can be overridden) | `registry.gitlab.com/to-be-continuous/tools/aws-auth-provider:latest` |
| `aws-region` / `AWS_REGION` | Default region (where the ECR registry is located) | _none_ |
| `aws-region` / `AWS_REGION`| Default region (where the ECR registry is located) | _none_ |
| `aws-snapshot-region` / `AWS_SNAPSHOT_REGION` | Region of the ECR registry for the snapshot image _(only define to override default)_ | _none_ |
| `aws-snapshot-region` / `AWS_SNAPSHOT_REGION` | Region of the ECR registry for the snapshot image _(only define to override default)_ | _none_ |
| `aws-release-region` / `AWS_RELEASE_REGION` | Region of the ECR registry for the release image _(only define to override default)_ | _none_ |
| `aws-release-region` / `AWS_RELEASE_REGION`| Region of the ECR registry for the release image _(only define to override default)_ | _none_ |
:warning: if using Kaniko, don't forget to either create the cache repository (snapshot image repository + `/cache`) or override `$KANIKO_SNAPSHOT_IMAGE_CACHE`
:warning: if using Kaniko, don't forget to either create the cache repository (snapshot image repository + `/cache`) or override `$KANIKO_SNAPSHOT_IMAGE_CACHE`
to use the snapshot image repository (will host your snapshot image as well as cached layers).
to use the snapshot image repository (will host your snapshot image as well as cached layers).
...
@@ -712,29 +711,29 @@ to use the snapshot image repository (will host your snapshot image as well as c
...
@@ -712,29 +711,29 @@ to use the snapshot image repository (will host your snapshot image as well as c
This is the recommended authentication method. In order to use it, first carefuly follow [GitLab's documentation](https://docs.gitlab.com/ee/ci/cloud_services/aws/),
This is the recommended authentication method. In order to use it, first carefuly follow [GitLab's documentation](https://docs.gitlab.com/ee/ci/cloud_services/aws/),
then set the required configuration.
then set the required configuration.
| Input / Variable | Description | Default value |
| Input / Variable | Description | Default value |
| `aws-oidc-aud` / `AWS_OIDC_AUD` | The `aud` claim for the JWT token | `$CI_SERVER_URL` |
| `aws-oidc-aud` / `AWS_OIDC_AUD`| The `aud` claim for the JWT token | `$CI_SERVER_URL` |
| `aws-oidc-role-arn` / `AWS_OIDC_ROLE_ARN` | Default IAM Role ARN associated with GitLab | _none_ |
| `aws-oidc-role-arn` / `AWS_OIDC_ROLE_ARN`| Default IAM Role ARN associated with GitLab | _none_ |
| `aws-snapshot-oidc-role-arn` / `AWS_SNAPSHOT_OIDC_ROLE_ARN` | IAM Role ARN associated with GitLab for the snapshot image _(only define to override default)_| _none_ |
| `aws-snapshot-oidc-role-arn` / `AWS_SNAPSHOT_OIDC_ROLE_ARN` | IAM Role ARN associated with GitLab for the snapshot image _(only define to override default)_| _none_ |
| `aws-release-oidc-role-arn` / `AWS_RELEASE_OIDC_ROLE_ARN` | IAM Role ARN associated with GitLab for the release image _(only define to override default)_| _none_ |
| `aws-release-oidc-role-arn` / `AWS_RELEASE_OIDC_ROLE_ARN`| IAM Role ARN associated with GitLab for the release image _(only define to override default)_| _none_ |