diff --git a/.gitlab/README.md.template b/.gitlab/README.md.template index 1930a95804013198858b1b159fafddd224b71fad..e77bcf401d67ef99ca8d095e2fc09e54b64cf959 100644 --- a/.gitlab/README.md.template +++ b/.gitlab/README.md.template @@ -210,8 +210,8 @@ plan: extends: [.opentofu:plan] parallel: matrix: - - TF_ROOT: test/ - - TF_ROOT: prod/ + - GITLAB_TOFU_ROOT_DIR: test/ + - GITLAB_TOFU_ROOT_DIR: prod/ ``` Have a look at the [`full-pipeline`](templates/job-templates.yml) for how it's constructed. @@ -238,13 +238,6 @@ beautifully renders the inputs for each templates - check it out! <RENDER> -### Variables - -(🚧 *This section is work in progress*) - -Have a look at the [`src/gitlab-tofu.sh`](src/gitlab-tofu.sh) script and how the `TF_`-prefixed -variables are being used. You may set them according to your needs. - ### Auto-forwarded predefined CI variables The `gitlab-tofu` script auto-forwards some "popular" @@ -328,7 +321,7 @@ we may make it the default behavior if possible. #### Lockfile Handling If you commit the Lockfile (`.terraform.lock.hcl`) to your repository -we recommend setting either the `TF_INIT_FLAGS` (handled by this component) +we recommend setting either the `GITLAB_TOFU_INIT_FLAGS` (handled by this component) or `TF_CLI_ARGS_init` (handled by OpenTofu directly) to `-lockfile=readonly` to prevent any changes to the lockfile during the pipeline job and with that ensuring that OpenTofu really uses the locked dependencies. @@ -516,8 +509,10 @@ When migrating from the GitLab Terraform CI/CD templates you can use the followi - `deploy` -> `apply` - Migrate the `TF_ROOT` variable to the `root_dir` input. - Although the `TF_ROOT` variable is still used and maybe overwritten after the import on individual jobs. + - Note that this component deprecated the `TF_ROOT` variable and uses `GITLAB_TOFU_ROOT_DIR` instead. - Migrate the `TF_STATE_NAME` variable to the `state_name` input. - Although the `TF_STATE_NAME` variable is still used and maybe overwritten after the import on individual jobs. + - Note that this component deprecated the `TF_STATE_NAME` variable and uses `GITLAB_TOFU_STATE_NAME` instead. - Migrate the `TF_AUTO_DEPLOY` variable to custom `rules` inputs. - Used other variables -> Use the same variables with this component. diff --git a/.gitlab/scripts/generate-readme.sh b/.gitlab/scripts/generate-readme.sh index da5daebfb22221022d86533956eb26f05a907b82..ced073a6e7364e5ee045af7c07948b80c3df9801 100755 --- a/.gitlab/scripts/generate-readme.sh +++ b/.gitlab/scripts/generate-readme.sh @@ -3,6 +3,7 @@ set -o errexit script_dir=$(dirname "$0") +project_root_dir="$(dirname "$0")/../.." csplit -sf readme -n 1 "${script_dir}/../README.md.template" '/<RENDER>/' @@ -17,6 +18,13 @@ csplit -sf readme -n 1 "${script_dir}/../README.md.template" '/<RENDER>/' echo 'The following OpenTofu versions are available with this component via the `opentofu_version` input:' echo yq '.".data".supported_versions | .[] | "- [`" + . + "`](https://github.com/opentofu/opentofu/releases/tag/v" + . + ")"' "${script_dir}/../../opentofu_versions.yaml" + echo + echo '### Environment Variables' + echo + echo 'The following environment variables are respected by the `gitlab-tofu` script:' + echo + awk '/^# %%%%/{i++} i==1 && NR>1' "${project_root_dir}/src/gitlab-tofu.sh" | tail -n+3 | cut -d'#' -f2- | cut -d' ' -f2- + echo tail -n+2 readme1 } > "${script_dir}/../../README.md" diff --git a/README.md b/README.md index 9a43bc784e874322ccbd70efdb262b186c0eaaaa..f45c9e48822b8351e42c4d4b56083ad953ec84ef 100644 --- a/README.md +++ b/README.md @@ -212,8 +212,8 @@ plan: extends: [.opentofu:plan] parallel: matrix: - - TF_ROOT: test/ - - TF_ROOT: prod/ + - GITLAB_TOFU_ROOT_DIR: test/ + - GITLAB_TOFU_ROOT_DIR: prod/ ``` Have a look at the [`full-pipeline`](templates/job-templates.yml) for how it's constructed. @@ -253,12 +253,65 @@ The following OpenTofu versions are available with this component via the `opent - [`1.7.3`](https://github.com/opentofu/opentofu/releases/tag/v1.7.3) - [`1.6.2`](https://github.com/opentofu/opentofu/releases/tag/v1.6.2) -### Variables +### Environment Variables + +The following environment variables are respected by the `gitlab-tofu` script: + +#### Respected Environment Variables + +- `GITLAB_TOFU_DEBUG`: if set to true will enable xtrace. +- `GITLAB_TOFU_SOURCE`: forces this script in source-mode. Required when source auto-detection fails. +- `GITLAB_TOFU_APPLY_NO_PLAN`: if set to true, the apply command does not use a plan cache file. +- `GITLAB_TOFU_PLAN_NAME`: the name of the plan cache and json files. Defaults to `plan`. +- `GITLAB_TOFU_PLAN_CACHE`: if set to the full path of the plan cache file. Defaults to `<root>/$GITLAB_TOFU_PLAN_NAME.cache` +- `GITLAB_TOFU_PLAN_JSON`: if set to the full path of the plan json file. Defaults to `<root>/$GITLAB_TOFU_PLAN_NAME.json` +- `GITLAB_TOFU_IMPLICIT_INIT`: if set to true will perform an implicit `tofu init` before any command that require it. Defaults to `true`. +- `GITLAB_TOFU_IGNORE_INIT_ERRORS`: if set to true will ignore errors in the `tofu init` command. +- `GITLAB_TOFU_INIT_NO_RECONFIGURE`: if set to true will not pass `-reconfigure` to the `tofu init` command. Defaults to `false`. +- `GITLAB_TOFU_STATE_NAME`: the name of the GitLab-managed Terraform state backend endpoint. +- `GITLAB_TOFU_STATE_ADDRESS`: the address of the GitLab-managed Terraform state backend. Defaults to `$CI_API_V4_URL/projects/$CI_PROJECT_ID/terraform/state/$GITLAB_TOFU_STATE_NAME`. +- `GITLAB_TOFU_USE_DETAILED_EXITCODE`: if set to true, `-detailed-exitcode` is supplied to `tofu plan`. Defaults to `false`. +- `GITLAB_TOFU_PLAN_WITH_JSON`: if set to true, will directly generate a JSON plan file when running `gitlab-tofu plan`. Defaults to `false`. +- `GITLAB_TOFU_VAR_FILE`: if set to a path it will pass `-var-file` to all `tofu` commands that support it. + +#### Respected OpenTofu Environment Variables + +> these are variables that are +> respected if set and avoid using +> the gitlab-tofu values for them. + +- `TF_HTTP_USERNAME`: username for the HTTP backend. Defaults to `gitlab-ci-token`. +- `TF_HTTP_PASSWORD`: password for the HTTP backend. Defaults to `$CI_JOB_TOKEN`. +- `TF_HTTP_ADDRESS`: address for the HTTP backend. Defaults to `$CI_API_V4_URL/projects/$CI_PROJECT_ID/terraform/state/<urlencode($GITLAB_TOFU_STATE_NAME)>`. +- `TF_HTTP_LOCK_ADDRESS`: lock address for the HTTP backend. Defaults to `$TF_HTTP_ADDRESS/lock`. +- `TF_HTTP_LOCK_METHOD`: lock method for the HTTP backend. Defaults to `POST`. +- `TF_HTTP_UNLOCK_ADDRESS`: unlock address for the HTTP backend. Defaults to `lock`. +- `TF_HTTP_UNLOCK_METHOD`: unlock address for the HTTP backend. Defaults to `unlock`. +- `TF_HTTP_RETRY_WAIT_MIN`: retry minimum waiting time in seconds. Defaults to `5`. +- `TF_CLI_CONFIG_FILE`: config file path. Defaults to `$HOME/.terraformrc` if it exists. + +#### Respected GitLab CI/CD Variables + +> these are variables exposed by +> GitLab CI/CD and respected by +> the gitlab-tofu script for +> certain configurations. + +- `CI_JOB_TOKEN`: + - used as default value for `TF_HTTP_PASSWORD`. + - used as value for `TF_TOKEN_<host>` variable. +- `CI_PROJECT_DIR`: + - used as default value for root directory. +- `CI_PROJECT_ID`: + - used as default value in constructing the `GITLAB_TOFU_STATE_ADDRESS`. +- `CI_API_V4_URL`: + - used as default value in constructing the `GITLAB_TOFU_STATE_ADDRESS`. +- `CI_SERVER_HOST`: + - used to construct for `TF_TOKEN_<host>` variable. +- `CI_SERVER_PROTOCOL`: + - used to construct for `TF_TOKEN_<host>` variable. -(🚧 *This section is work in progress*) -Have a look at the [`src/gitlab-tofu.sh`](src/gitlab-tofu.sh) script and how the `TF_`-prefixed -variables are being used. You may set them according to your needs. ### Auto-forwarded predefined CI variables @@ -343,7 +396,7 @@ we may make it the default behavior if possible. #### Lockfile Handling If you commit the Lockfile (`.terraform.lock.hcl`) to your repository -we recommend setting either the `TF_INIT_FLAGS` (handled by this component) +we recommend setting either the `GITLAB_TOFU_INIT_FLAGS` (handled by this component) or `TF_CLI_ARGS_init` (handled by OpenTofu directly) to `-lockfile=readonly` to prevent any changes to the lockfile during the pipeline job and with that ensuring that OpenTofu really uses the locked dependencies. @@ -531,8 +584,10 @@ When migrating from the GitLab Terraform CI/CD templates you can use the followi - `deploy` -> `apply` - Migrate the `TF_ROOT` variable to the `root_dir` input. - Although the `TF_ROOT` variable is still used and maybe overwritten after the import on individual jobs. + - Note that this component deprecated the `TF_ROOT` variable and uses `GITLAB_TOFU_ROOT_DIR` instead. - Migrate the `TF_STATE_NAME` variable to the `state_name` input. - Although the `TF_STATE_NAME` variable is still used and maybe overwritten after the import on individual jobs. + - Note that this component deprecated the `TF_STATE_NAME` variable and uses `GITLAB_TOFU_STATE_NAME` instead. - Migrate the `TF_AUTO_DEPLOY` variable to custom `rules` inputs. - Used other variables -> Use the same variables with this component. diff --git a/src/gitlab-tofu.sh b/src/gitlab-tofu.sh index 3e9c167d34abe5f53acb24bc9d215f848412b3bb..0c2489a1590da95e88c323037cea7df84224f6e5 100644 --- a/src/gitlab-tofu.sh +++ b/src/gitlab-tofu.sh @@ -1,9 +1,76 @@ #!/usr/bin/env sh +# gitlab-tofu is a wrapper around the tofu command +# from the OpenTofu project. +# +# It's main purpose is to setup tofu to work inside +# GitLab pipelines and together with the +# OpenTofu CI/CD component. +# Detailed information about it is in the README: +# https://gitlab.com/components/opentofu +# +# %%%% +# +# #### Respected Environment Variables +# +# - `GITLAB_TOFU_DEBUG`: if set to true will enable xtrace. +# - `GITLAB_TOFU_SOURCE`: forces this script in source-mode. Required when source auto-detection fails. +# - `GITLAB_TOFU_APPLY_NO_PLAN`: if set to true, the apply command does not use a plan cache file. +# - `GITLAB_TOFU_PLAN_NAME`: the name of the plan cache and json files. Defaults to `plan`. +# - `GITLAB_TOFU_PLAN_CACHE`: if set to the full path of the plan cache file. Defaults to `<root>/$GITLAB_TOFU_PLAN_NAME.cache` +# - `GITLAB_TOFU_PLAN_JSON`: if set to the full path of the plan json file. Defaults to `<root>/$GITLAB_TOFU_PLAN_NAME.json` +# - `GITLAB_TOFU_IMPLICIT_INIT`: if set to true will perform an implicit `tofu init` before any command that require it. Defaults to `true`. +# - `GITLAB_TOFU_IGNORE_INIT_ERRORS`: if set to true will ignore errors in the `tofu init` command. +# - `GITLAB_TOFU_INIT_NO_RECONFIGURE`: if set to true will not pass `-reconfigure` to the `tofu init` command. Defaults to `false`. +# - `GITLAB_TOFU_STATE_NAME`: the name of the GitLab-managed Terraform state backend endpoint. +# - `GITLAB_TOFU_STATE_ADDRESS`: the address of the GitLab-managed Terraform state backend. Defaults to `$CI_API_V4_URL/projects/$CI_PROJECT_ID/terraform/state/$GITLAB_TOFU_STATE_NAME`. +# - `GITLAB_TOFU_USE_DETAILED_EXITCODE`: if set to true, `-detailed-exitcode` is supplied to `tofu plan`. Defaults to `false`. +# - `GITLAB_TOFU_PLAN_WITH_JSON`: if set to true, will directly generate a JSON plan file when running `gitlab-tofu plan`. Defaults to `false`. +# - `GITLAB_TOFU_VAR_FILE`: if set to a path it will pass `-var-file` to all `tofu` commands that support it. +# +# #### Respected OpenTofu Environment Variables +# +# > these are variables that are +# > respected if set and avoid using +# > the gitlab-tofu values for them. +# +# - `TF_HTTP_USERNAME`: username for the HTTP backend. Defaults to `gitlab-ci-token`. +# - `TF_HTTP_PASSWORD`: password for the HTTP backend. Defaults to `$CI_JOB_TOKEN`. +# - `TF_HTTP_ADDRESS`: address for the HTTP backend. Defaults to `$CI_API_V4_URL/projects/$CI_PROJECT_ID/terraform/state/<urlencode($GITLAB_TOFU_STATE_NAME)>`. +# - `TF_HTTP_LOCK_ADDRESS`: lock address for the HTTP backend. Defaults to `$TF_HTTP_ADDRESS/lock`. +# - `TF_HTTP_LOCK_METHOD`: lock method for the HTTP backend. Defaults to `POST`. +# - `TF_HTTP_UNLOCK_ADDRESS`: unlock address for the HTTP backend. Defaults to `lock`. +# - `TF_HTTP_UNLOCK_METHOD`: unlock address for the HTTP backend. Defaults to `unlock`. +# - `TF_HTTP_RETRY_WAIT_MIN`: retry minimum waiting time in seconds. Defaults to `5`. +# - `TF_CLI_CONFIG_FILE`: config file path. Defaults to `$HOME/.terraformrc` if it exists. +# +# #### Respected GitLab CI/CD Variables +# +# > these are variables exposed by +# > GitLab CI/CD and respected by +# > the gitlab-tofu script for +# > certain configurations. +# +# - `CI_JOB_TOKEN`: +# - used as default value for `TF_HTTP_PASSWORD`. +# - used as value for `TF_TOKEN_<host>` variable. +# - `CI_PROJECT_DIR`: +# - used as default value for root directory. +# - `CI_PROJECT_ID`: +# - used as default value in constructing the `GITLAB_TOFU_STATE_ADDRESS`. +# - `CI_API_V4_URL`: +# - used as default value in constructing the `GITLAB_TOFU_STATE_ADDRESS`. +# - `CI_SERVER_HOST`: +# - used to construct for `TF_TOKEN_<host>` variable. +# - `CI_SERVER_PROTOCOL`: +# - used to construct for `TF_TOKEN_<host>` variable. +# +# %%%% + # set some shell options set -o errexit -if [ "${DEBUG_OUTPUT}" = "true" ]; then +if [ "${GITLAB_TOFU_DEBUG}" = "true" ]; then set -o xtrace fi @@ -13,8 +80,8 @@ fi # There are no feature flags at the moment. -# Helpers - +# Source Mode +# =========== # Evaluate if this script is being sourced or executed directly. # See https://stackoverflow.com/a/28776166 sourced=0 @@ -56,111 +123,131 @@ if [ -n "$ZSH_VERSION" ]; then unsetopt sh_word_split fi -JQ_PLAN=' - ( - [.resource_changes[]?.change.actions?] | flatten - ) | { - "create":(map(select(.=="create")) | length), - "update":(map(select(.=="update")) | length), - "delete":(map(select(.=="delete")) | length) - } -' - -# If TF_USERNAME is unset then default to GITLAB_USER_LOGIN -TF_USERNAME="${TF_USERNAME:-${GITLAB_USER_LOGIN}}" +# Deprecations +# ============ +if [ -n "$TF_STATE_NAME" ]; then + echo 'WARNING: you have manually set the deprecated TF_STATE_NAME environment variable. Please use the GITLAB_TOFU_STATE_NAME environment variable instead. The TF_STATE_NAME variable will be removed soon.' >&2 -# If TF_PASSWORD is unset then default to gitlab-ci-token/CI_JOB_TOKEN -if [ -z "${TF_PASSWORD}" ]; then - TF_USERNAME="gitlab-ci-token" - TF_PASSWORD="${CI_JOB_TOKEN}" + if [ -n "$GITLAB_TOFU_STATE_NAME" ]; then + echo 'WARNING: you have set GITLAB_TOFU_STATE_NAME environment variable in addition to the deprecated TF_STATE_NAME. This causes a conflict and GITLAB_TOFU_STATE_NAME will be used exclusively' >&2 + else + GITLAB_TOFU_STATE_NAME="$TF_STATE_NAME" + fi fi -# If TF_ADDRESS is unset but TF_STATE_NAME is provided, then default to GitLab backend in current project -if [ -n "${TF_STATE_NAME}" ] && [ -z "${TF_ADDRESS}" ]; then - # auto url-encode TF_STATE_NAME - TF_STATE_NAME="$(jq -rn --arg x "${TF_STATE_NAME}" '$x|@uri')" - TF_ADDRESS="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${TF_STATE_NAME}" -fi +if [ -n "$TF_ADDRESS" ]; then + echo 'WARNING: you have manually set the deprecated TF_ADDRESS environment variable. Please use the GITLAB_TOFU_STATE_ADDRESS environment variable instead. The TF_ADDRESS variable will be removed soon.' >&2 -if [ -z "${TF_PLAN_NAME}" ]; then - TF_PLAN_NAME=plan + if [ -n "$GITLAB_TOFU_STATE_ADDRESS" ]; then + echo 'WARNING: you have set GITLAB_TOFU_STATE_ADDRESS environment variable in addition to the deprecated TF_ADDRESS. This causes a conflict and GITLAB_TOFU_STATE_ADDRESS will be used exclusively' >&2 + else + GITLAB_TOFU_STATE_ADDRESS="$TF_ADDRESS" + fi fi -if [ -z "${TF_APPLY_NO_PLAN}" ]; then - TF_APPLY_NO_PLAN=false -fi +if [ -n "$TF_ROOT" ]; then + echo 'WARNING: you have manually set the deprecated TF_ROOT environment variable. Please use the GITLAB_TOFU_ROOT_DIR environment variable instead. The TF_ROOT variable will be removed soon.' >&2 -# If TF_ROOT is set then use the -chdir option -if [ -n "${TF_ROOT}" ]; then - abs_tf_root=$(cd "${CI_PROJECT_DIR}"; realpath "${TF_ROOT}") + if [ -n "$GITLAB_TOFU_ROOT_DIR" ]; then + echo 'WARNING: you have set GITLAB_TOFU_ROOT_DIR environment variable in addition to the deprecated TF_ROOT. This causes a conflict and GITLAB_TOFU_ROOT_DIR will be used exclusively' >&2 + else + GITLAB_TOFU_ROOT_DIR="$TF_ROOT" + fi +fi - TF_CHDIR_OPT="-chdir=${abs_tf_root}" +# Handle environment variables +# ============================ - default_tf_plan_cache="${abs_tf_root}/${TF_PLAN_NAME}.cache" - default_tf_plan_json="${abs_tf_root}/${TF_PLAN_NAME}.json" -fi +# Backend related variables +backend_username="gitlab-ci-token" +backend_password="${CI_JOB_TOKEN}" +backend_state_name="$(jq -rn --arg x "${GITLAB_TOFU_STATE_NAME:-default}" '$x|@uri')" +backend_address="${GITLAB_TOFU_STATE_ADDRESS:-${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${backend_state_name}}" +# Root directory related variables +base_plan_name="${GITLAB_TOFU_PLAN_NAME:-plan}" +if [ -n "${GITLAB_TOFU_ROOT_DIR}" ]; then + abs_tf_root=$(cd "${CI_PROJECT_DIR}"; realpath "${GITLAB_TOFU_ROOT_DIR}") -# If TF_PLAN_CACHE is not set then use either the plan.cache file within TF_ROOT if set, or plan.cache in CWD -if [ -z "${TF_PLAN_CACHE}" ]; then - TF_PLAN_CACHE="${default_tf_plan_cache:-${TF_PLAN_NAME}.cache}" -fi + tf_chdir_opt="-chdir=${abs_tf_root}" -# If TF_PLAN_JSON is not set then use either the plan.json file within TF_ROOT if set, or plan.json in CWD -if [ -z "${TF_PLAN_JSON}" ]; then - TF_PLAN_JSON="${default_tf_plan_json:-${TF_PLAN_NAME}.json}" + default_tf_plan_cache="${abs_tf_root}/${base_plan_name}.cache" + default_tf_plan_json="${abs_tf_root}/${base_plan_name}.json" fi -# Set variables for the HTTP backend to default to TF_* values -export TF_HTTP_ADDRESS="${TF_HTTP_ADDRESS:-${TF_ADDRESS}}" -export TF_HTTP_LOCK_ADDRESS="${TF_HTTP_LOCK_ADDRESS:-${TF_ADDRESS}/lock}" -export TF_HTTP_LOCK_METHOD="${TF_HTTP_LOCK_METHOD:-POST}" -export TF_HTTP_UNLOCK_ADDRESS="${TF_HTTP_UNLOCK_ADDRESS:-${TF_ADDRESS}/lock}" -export TF_HTTP_UNLOCK_METHOD="${TF_HTTP_UNLOCK_METHOD:-DELETE}" -export TF_HTTP_USERNAME="${TF_HTTP_USERNAME:-${TF_USERNAME}}" -export TF_HTTP_PASSWORD="${TF_HTTP_PASSWORD:-${TF_PASSWORD}}" -export TF_HTTP_RETRY_WAIT_MIN="${TF_HTTP_RETRY_WAIT_MIN:-5}" - -# Expose Gitlab specific variables to terraform since no -tf-var is available -# The following variables are deprecated because they do not conform to -# HCL naming best practices. Use the lower snake_case variants below instead. -export TF_VAR_CI_JOB_ID="${TF_VAR_CI_JOB_ID:-${CI_JOB_ID}}" -export TF_VAR_CI_COMMIT_SHA="${TF_VAR_CI_COMMIT_SHA:-${CI_COMMIT_SHA}}" -export TF_VAR_CI_JOB_STAGE="${TF_VAR_CI_JOB_STAGE:-${CI_JOB_STAGE}}" -export TF_VAR_CI_PROJECT_ID="${TF_VAR_CI_PROJECT_ID:-${CI_PROJECT_ID}}" -export TF_VAR_CI_PROJECT_NAME="${TF_VAR_CI_PROJECT_NAME:-${CI_PROJECT_NAME}}" -export TF_VAR_CI_PROJECT_NAMESPACE="${TF_VAR_CI_PROJECT_NAMESPACE:-${CI_PROJECT_NAMESPACE}}" -export TF_VAR_CI_PROJECT_PATH="${TF_VAR_CI_PROJECT_PATH:-${CI_PROJECT_PATH}}" -export TF_VAR_CI_PROJECT_URL="${TF_VAR_CI_PROJECT_URL:-${CI_PROJECT_URL}}" - - -export TF_VAR_ci_job_id="${TF_VAR_ci_job_id:-${CI_JOB_ID}}" -export TF_VAR_ci_commit_sha="${TF_VAR_ci_commit_sha:-${CI_COMMIT_SHA}}" -export TF_VAR_ci_job_stage="${TF_VAR_ci_job_stage:-${CI_JOB_STAGE}}" -export TF_VAR_ci_project_id="${TF_VAR_ci_project_id:-${CI_PROJECT_ID}}" -export TF_VAR_ci_project_name="${TF_VAR_ci_project_name:-${CI_PROJECT_NAME}}" -export TF_VAR_ci_project_namespace="${TF_VAR_ci_project_namespace:-${CI_PROJECT_NAMESPACE}}" -export TF_VAR_ci_project_path="${TF_VAR_ci_project_path:-${CI_PROJECT_PATH}}" -export TF_VAR_ci_project_url="${TF_VAR_ci_project_url:-${CI_PROJECT_URL}}" - -# Use terraform automation mode (will remove some verbose unneeded messages) -export TF_IN_AUTOMATION=true - -DEFAULT_TF_CONFIG_PATH="$HOME/.terraformrc" - -# Set a Terraform CLI Configuration File -if [ -z "${TF_CLI_CONFIG_FILE}" ] && [ -f "${DEFAULT_TF_CONFIG_PATH}" ]; then - export TF_CLI_CONFIG_FILE="${DEFAULT_TF_CONFIG_PATH}" -fi +# Init related variables +init_flags=${GITLAB_TOFU_INIT_FLAGS} +should_do_implicit_init=${GITLAB_TOFU_IMPLICIT_INIT:-true} +should_ignore_init_errors=${GITLAB_TOFU_IGNORE_INIT_ERRORS:-false} +should_init_without_reconfigure=${GITLAB_TOFU_INIT_NO_RECONFIGURE:-false} + +# Plan variables +apply_without_plan=${GITLAB_TOFU_APPLY_NO_PLAN:-false} +plan_cache_path="${GITLAB_TOFU_PLAN_CACHE:-${default_tf_plan_cache:-${base_plan_name}.cache}}" +plan_json_path="${GITLAB_TOFU_PLAN_JSON:-${default_tf_plan_json:-${base_plan_name}.cache}}" +plan_with_detailed_exitcode=${GITLAB_TOFU_USE_DETAILED_EXITCODE:-false} +plan_with_json_file=${GITLAB_TOFU_PLAN_WITH_JSON:-false} +plan_jq_filter=' + ( + [.resource_changes[]?.change.actions?] | flatten + ) | { + "create":(map(select(.=="create")) | length), + "update":(map(select(.=="update")) | length), + "delete":(map(select(.=="delete")) | length) + } +' +# Misc variables +var_file="${GITLAB_TOFU_VAR_FILE}" + +# Helper functions +# ================ + +# configure_variables_for_tofu sets and exports all relevant variables for subsequent `tofu` command invocations. +configure_variables_for_tofu() { + # Use terraform automation mode (will remove some verbose unneeded messages) + export TF_IN_AUTOMATION=true + + # Set variables for the HTTP backend to default to TF_* values + export TF_HTTP_ADDRESS="${TF_HTTP_ADDRESS:-${backend_address}}" + export TF_HTTP_LOCK_ADDRESS="${TF_HTTP_LOCK_ADDRESS:-${backend_address}/lock}" + export TF_HTTP_LOCK_METHOD="${TF_HTTP_LOCK_METHOD:-POST}" + export TF_HTTP_UNLOCK_ADDRESS="${TF_HTTP_UNLOCK_ADDRESS:-${backend_address}/lock}" + export TF_HTTP_UNLOCK_METHOD="${TF_HTTP_UNLOCK_METHOD:-DELETE}" + export TF_HTTP_USERNAME="${TF_HTTP_USERNAME:-${backend_username}}" + export TF_HTTP_PASSWORD="${TF_HTTP_PASSWORD:-${backend_password}}" + export TF_HTTP_RETRY_WAIT_MIN="${TF_HTTP_RETRY_WAIT_MIN:-5}" + + # Expose Gitlab specific variables to terraform since no -tf-var is available + # The following variables are deprecated because they do not conform to + # HCL naming best practices. Use the lower snake_case variants below instead. + export TF_VAR_CI_JOB_ID="${TF_VAR_CI_JOB_ID:-${CI_JOB_ID}}" + export TF_VAR_CI_COMMIT_SHA="${TF_VAR_CI_COMMIT_SHA:-${CI_COMMIT_SHA}}" + export TF_VAR_CI_JOB_STAGE="${TF_VAR_CI_JOB_STAGE:-${CI_JOB_STAGE}}" + export TF_VAR_CI_PROJECT_ID="${TF_VAR_CI_PROJECT_ID:-${CI_PROJECT_ID}}" + export TF_VAR_CI_PROJECT_NAME="${TF_VAR_CI_PROJECT_NAME:-${CI_PROJECT_NAME}}" + export TF_VAR_CI_PROJECT_NAMESPACE="${TF_VAR_CI_PROJECT_NAMESPACE:-${CI_PROJECT_NAMESPACE}}" + export TF_VAR_CI_PROJECT_PATH="${TF_VAR_CI_PROJECT_PATH:-${CI_PROJECT_PATH}}" + export TF_VAR_CI_PROJECT_URL="${TF_VAR_CI_PROJECT_URL:-${CI_PROJECT_URL}}" + + export TF_VAR_ci_job_id="${TF_VAR_ci_job_id:-${CI_JOB_ID}}" + export TF_VAR_ci_commit_sha="${TF_VAR_ci_commit_sha:-${CI_COMMIT_SHA}}" + export TF_VAR_ci_job_stage="${TF_VAR_ci_job_stage:-${CI_JOB_STAGE}}" + export TF_VAR_ci_project_id="${TF_VAR_ci_project_id:-${CI_PROJECT_ID}}" + export TF_VAR_ci_project_name="${TF_VAR_ci_project_name:-${CI_PROJECT_NAME}}" + export TF_VAR_ci_project_namespace="${TF_VAR_ci_project_namespace:-${CI_PROJECT_NAMESPACE}}" + export TF_VAR_ci_project_path="${TF_VAR_ci_project_path:-${CI_PROJECT_PATH}}" + export TF_VAR_ci_project_url="${TF_VAR_ci_project_url:-${CI_PROJECT_URL}}" + + # Set a Terraform CLI Configuration File + default_tf_cli_config_file="$HOME/.terraformrc" + if [ -z "${TF_CLI_CONFIG_FILE}" ] && [ -f "${default_tf_cli_config_file}" ]; then + export TF_CLI_CONFIG_FILE="${default_tf_cli_config_file}" + fi +} -terraform_authenticate_private_registry() { - # From Terraform 1.2.0 and later (or all versions of OpenTofu), we can use TF_TOKEN_your_domain_name to authenticate to registry. - # The credential environment variable has the following requirements: - # - Domain names containing non-ASCII characters are converted to their punycode equivalent with an ACE prefix - # - Periods are encoded as underscores - # - Hyphens are encoded as double underscores - # For more info, see https://www.terraform.io/cli/config/config-file#environment-variable-credentials +# tofu_authenticate_private_registry sets the TF_TOKEN_* variable to authenticate private registries. +tofu_authenticate_private_registry() { if [ "${CI_SERVER_PROTOCOL}" = "https" ] && [ -n "${CI_SERVER_HOST}" ]; then tf_token_var_name=TF_TOKEN_$(idn2 "${CI_SERVER_HOST}" | sed 's/\./_/g' | sed 's/-/__/g') # If TF_TOKEN_ for the Gitlab domain is not set then use the CI_JOB_TOKEN @@ -170,74 +257,67 @@ terraform_authenticate_private_registry() { fi } -# If TF_IMPLICIT_INIT is not set, we set it to `true`. -# If set to `true` it will call `terraform init` prior -# to calling the wrapper `terraform` commands. -TF_IMPLICIT_INIT=${TF_IMPLICIT_INIT:-true} - -# Allows users to continue the actual command in case init failed -TF_IGNORE_INIT_ERRORS=${TF_IGNORE_INIT_ERRORS:-false} - -terraform_init() { - # If TF_INIT_NO_RECONFIGURE is not set to 'true', - # a `-reconfigure` flag is added to the `terraform init` command. - if [ "$TF_INIT_NO_RECONFIGURE" != 'true' ]; then - tf_init_reconfigure_flag='-reconfigure' +# tofu_init runs `tofu init` with all things considered. +tofu_init() { + if ! $should_init_without_reconfigure; then + tofu_init_reconfigure_flag='-reconfigure' fi - # We want to allow word splitting here for TF_INIT_FLAGS - # shellcheck disable=SC2086 - tofu "${TF_CHDIR_OPT}" init "${@}" -input=false ${tf_init_reconfigure_flag} ${TF_INIT_FLAGS} \ - 1>&2 || $TF_IGNORE_INIT_ERRORS + # shellcheck disable=SC2086 # We want to allow word splitting here for `init_flags` + tofu "${tf_chdir_opt}" init "${@}" -input=false ${tofu_init_reconfigure_flag} ${init_flags} \ + 1>&2 || $should_ignore_init_errors } -# If this script is executed and not sourced, a terraform command is ran. +# We always want to configure the tofu variables, even in source-mode. +configure_variables_for_tofu + +# If this script is executed and not sourced, a tofu command is ran. # Otherwise, nothing happens and the sourced shell can use the defined variables # and helper functions exposed by this script. if [ $sourced -eq 0 ]; then # Authenticate to private registry - terraform_authenticate_private_registry + tofu_authenticate_private_registry var_file_args="" - if [ -n "${GITLAB_TOFU_VAR_FILE}" ]; then - var_file_args="--var-file=${GITLAB_TOFU_VAR_FILE}" + if [ -n "${var_file}" ]; then + var_file_args="--var-file=${var_file}" fi case "${1}" in "apply") - $TF_IMPLICIT_INIT && terraform_init - if [ "$TF_APPLY_NO_PLAN" = false ]; then - tofu "${TF_CHDIR_OPT}" "${@}" -input=false -auto-approve "${TF_PLAN_CACHE}" + $should_do_implicit_init && tofu_init + if ! $apply_without_plan; then + tofu "${tf_chdir_opt}" "${@}" -input=false -auto-approve "${plan_cache_path}" else # shellcheck disable=SC2086 - tofu "${TF_CHDIR_OPT}" "${@}" -input=false -auto-approve ${var_file_args} + tofu "${tf_chdir_opt}" "${@}" -input=false -auto-approve ${var_file_args} fi ;; "destroy") - $TF_IMPLICIT_INIT && terraform_init - tofu "${TF_CHDIR_OPT}" "${@}" -auto-approve + $should_do_implicit_init && tofu_init + tofu "${tf_chdir_opt}" "${@}" -auto-approve ;; "fmt") - tofu "${TF_CHDIR_OPT}" "${@}" -check -diff -recursive + tofu "${tf_chdir_opt}" "${@}" -check -diff -recursive ;; "init") # shift argument list „one to the left“ to not call 'terraform init init' shift - terraform_init "${@}" + tofu_init "${@}" ;; "plan") plan_args='' - if [ "${GITLAB_TOFU_USE_DETAILED_EXITCODE}" = 'true' ]; then + if $plan_with_detailed_exitcode; then plan_args='-detailed-exitcode' fi - $TF_IMPLICIT_INIT && terraform_init + $should_do_implicit_init && tofu_init # shellcheck disable=SC2086 - tofu "${TF_CHDIR_OPT}" "${@}" -input=false -out="${TF_PLAN_CACHE}" ${var_file_args} ${plan_args} && ret=$? || ret=$? + tofu "${tf_chdir_opt}" "${@}" -input=false -out="${plan_cache_path}" ${var_file_args} ${plan_args} && ret=$? || ret=$? - if [ "${GITLAB_TOFU_PLAN_WITH_JSON}" = 'true' ]; then + if $plan_with_json_file; then if [ "$ret" -eq 0 ] || [ "$ret" -eq 2 ]; then - if ! tofu "${TF_CHDIR_OPT}" show -json "${TF_PLAN_CACHE}" | jq -r "${JQ_PLAN}" > "${TF_PLAN_JSON}"; then + if ! tofu "${tf_chdir_opt}" show -json "${plan_cache_path}" | jq -r "${plan_jq_filter}" > "${plan_json_path}"; then exit $? fi @@ -249,29 +329,29 @@ if [ $sourced -eq 0 ]; then exit "$ret" ;; "plan-json") - tofu "${TF_CHDIR_OPT}" show -json "${TF_PLAN_CACHE}" | jq -r "${JQ_PLAN}" > "${TF_PLAN_JSON}" + tofu "${tf_chdir_opt}" show -json "${plan_cache_path}" | jq -r "${plan_jq_filter}" > "${plan_json_path}" ;; "validate") - $TF_IMPLICIT_INIT && terraform_init -backend=false + $should_do_implicit_init && tofu_init -backend=false # shellcheck disable=SC2086 - tofu "${TF_CHDIR_OPT}" "${@}" ${var_file_args} + tofu "${tf_chdir_opt}" "${@}" ${var_file_args} ;; "test") - $TF_IMPLICIT_INIT && terraform_init -backend=false + $should_do_implicit_init && tofu_init -backend=false # shellcheck disable=SC2086 - tofu "${TF_CHDIR_OPT}" "${@}" ${var_file_args} + tofu "${tf_chdir_opt}" "${@}" ${var_file_args} ;; "graph") - $TF_IMPLICIT_INIT && terraform_init + $should_do_implicit_init && tofu_init # shellcheck disable=SC2086 - tofu "${TF_CHDIR_OPT}" "${@}" ${var_file_args} + tofu "${tf_chdir_opt}" "${@}" ${var_file_args} ;; --) shift - tofu "${TF_CHDIR_OPT}" "${@}" + tofu "${tf_chdir_opt}" "${@}" ;; *) - tofu "${TF_CHDIR_OPT}" "${@}" + tofu "${tf_chdir_opt}" "${@}" ;; esac else diff --git a/templates/apply.yml b/templates/apply.yml index a4b88ff94cb1d1966573ad631024a8c5a9e81d41..f8f6b4801b94db63692c76733a084e7930680c7a 100644 --- a/templates/apply.yml +++ b/templates/apply.yml @@ -75,7 +75,7 @@ spec: description: 'Whether a plan file should be used.' plan_name: default: 'plan' - description: 'The name of the plan file to use. Will be used for TF_PLAN_CACHE and TF_PLAN_JSON.' + description: 'The name of the plan file to use. Will be used for plan cache and json files.' var_file: default: '' type: string @@ -97,22 +97,22 @@ spec: '$[[ inputs.as ]]': stage: $[[ inputs.stage ]] environment: - name: $TF_STATE_NAME + name: $GITLAB_TOFU_STATE_NAME action: start - resource_group: $TF_STATE_NAME + resource_group: $GITLAB_TOFU_STATE_NAME rules: $[[ inputs.rules ]] cache: key: "$__CACHE_KEY_HACK" policy: $[[ inputs.cache_policy ]] paths: - - $TF_ROOT/.terraform/ + - $GITLAB_TOFU_ROOT_DIR/.terraform/ variables: # FIXME: work around to make slashes work in `cache:key`. see https://gitlab.com/gitlab-org/gitlab/-/issues/439898 __CACHE_KEY_HACK: "$[[ inputs.root_dir ]]" - TF_ROOT: $[[ inputs.root_dir ]] - TF_STATE_NAME: $[[ inputs.state_name ]] - TF_APPLY_NO_PLAN: $[[ inputs.no_plan ]] - TF_PLAN_NAME: $[[ inputs.plan_name ]] + GITLAB_TOFU_ROOT_DIR: $[[ inputs.root_dir ]] + GITLAB_TOFU_STATE_NAME: $[[ inputs.state_name ]] + GITLAB_TOFU_APPLY_NO_PLAN: $[[ inputs.no_plan ]] + GITLAB_TOFU_PLAN_NAME: $[[ inputs.plan_name ]] GITLAB_TOFU_VAR_FILE: '$[[ inputs.var_file ]]' image: name: '$[[ inputs.image_registry_base ]]/$[[ inputs.image_name ]]:$[[ inputs.version ]]-opentofu$[[ inputs.opentofu_version ]]-$[[ inputs.base_os ]]$[[ inputs.image_digest ]]' diff --git a/templates/custom-command.yml b/templates/custom-command.yml index 50dcdb5cab0db1fb94ffaa9677876118d9921294..c0b129b7593b86b64a94efd51097f10f5bd6f2aa 100644 --- a/templates/custom-command.yml +++ b/templates/custom-command.yml @@ -92,11 +92,11 @@ spec: key: "$__CACHE_KEY_HACK" policy: $[[ inputs.cache_policy ]] paths: - - $TF_ROOT/.terraform/ + - $GITLAB_TOFU_ROOT_DIR/.terraform/ variables: # FIXME: work around to make slashes work in `cache:key`. see https://gitlab.com/gitlab-org/gitlab/-/issues/439898 __CACHE_KEY_HACK: "$[[ inputs.root_dir ]]" - TF_ROOT: $[[ inputs.root_dir ]] + GITLAB_TOFU_ROOT_DIR: $[[ inputs.root_dir ]] image: name: '$[[ inputs.image_registry_base ]]/$[[ inputs.image_name ]]:$[[ inputs.version ]]-opentofu$[[ inputs.opentofu_version ]]-$[[ inputs.base_os ]]$[[ inputs.image_digest ]]' script: diff --git a/templates/delete-state.yml b/templates/delete-state.yml index 3305cc68653af3e5c9cc6403633f0e50b043690b..cf6eee89997b4eb6ccd367f10930e318f14dd9c7 100644 --- a/templates/delete-state.yml +++ b/templates/delete-state.yml @@ -27,10 +27,10 @@ spec: '$[[ inputs.as ]]': stage: $[[ inputs.stage ]] - resource_group: $TF_STATE_NAME + resource_group: $GITLAB_TOFU_STATE_NAME image: curlimages/curl:latest variables: - TF_STATE_NAME: $[[ inputs.state_name ]] + GITLAB_TOFU_STATE_NAME: $[[ inputs.state_name ]] script: - - curl --request DELETE -u "gitlab-ci-token:$CI_JOB_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/terraform/state/$TF_STATE_NAME" + - curl --request DELETE -u "gitlab-ci-token:$CI_JOB_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/terraform/state/$GITLAB_TOFU_STATE_NAME" rules: $[[ inputs.rules ]] diff --git a/templates/destroy.yml b/templates/destroy.yml index b9a8a68b9d1cfa0117bc8ff082b8e5762930a4b4..56306cc5bf4f9a72f5a1fadd96861f206f76cf6d 100644 --- a/templates/destroy.yml +++ b/templates/destroy.yml @@ -75,7 +75,7 @@ spec: description: 'Whether a plan file should be used.' plan_name: default: 'destroy-plan' - description: 'The name of the plan file to use. Will be used for TF_PLAN_CACHE and TF_PLAN_JSON.' + description: 'The name of the plan file to use. Will be used for the cache and json file.' var_file: default: '' type: string @@ -97,22 +97,22 @@ spec: '$[[ inputs.as ]]': stage: $[[ inputs.stage ]] environment: - name: $TF_STATE_NAME + name: $GITLAB_TOFU_STATE_NAME action: stop - resource_group: $TF_STATE_NAME + resource_group: $GITLAB_TOFU_STATE_NAME rules: $[[ inputs.rules ]] cache: key: "$__CACHE_KEY_HACK" policy: $[[ inputs.cache_policy ]] paths: - - $TF_ROOT/.terraform/ + - $GITLAB_TOFU_ROOT_DIR/.terraform/ variables: # FIXME: work around to make slashes work in `cache:key`. see https://gitlab.com/gitlab-org/gitlab/-/issues/439898 __CACHE_KEY_HACK: "$[[ inputs.root_dir ]]" - TF_ROOT: $[[ inputs.root_dir ]] - TF_STATE_NAME: $[[ inputs.state_name ]] - TF_APPLY_NO_PLAN: $[[ inputs.no_plan ]] - TF_PLAN_NAME: $[[ inputs.plan_name ]] + GITLAB_TOFU_ROOT_DIR: $[[ inputs.root_dir ]] + GITLAB_TOFU_STATE_NAME: $[[ inputs.state_name ]] + GITLAB_TOFU_APPLY_NO_PLAN: $[[ inputs.no_plan ]] + GITLAB_TOFU_PLAN_NAME: $[[ inputs.plan_name ]] GITLAB_TOFU_VAR_FILE: '$[[ inputs.var_file ]]' image: name: '$[[ inputs.image_registry_base ]]/$[[ inputs.image_name ]]:$[[ inputs.version ]]-opentofu$[[ inputs.opentofu_version ]]-$[[ inputs.base_os ]]$[[ inputs.image_digest ]]' diff --git a/templates/fmt.yml b/templates/fmt.yml index 5c4a6fe8e711488a438068ea6a1f9afa99973b5b..096ed6c26486c20bace7685c8175f011b534ac7f 100644 --- a/templates/fmt.yml +++ b/templates/fmt.yml @@ -98,7 +98,7 @@ spec: variables: # FIXME: work around to make slashes work in `cache:key`. see https://gitlab.com/gitlab-org/gitlab/-/issues/439898 __CACHE_KEY_HACK: "$[[ inputs.root_dir ]]" - TF_ROOT: $[[ inputs.root_dir ]] + GITLAB_TOFU_ROOT_DIR: $[[ inputs.root_dir ]] image: name: '$[[ inputs.image_registry_base ]]/$[[ inputs.image_name ]]:$[[ inputs.version ]]-opentofu$[[ inputs.opentofu_version ]]-$[[ inputs.base_os ]]$[[ inputs.image_digest ]]' script: diff --git a/templates/graph.yml b/templates/graph.yml index 885252d9784b88198f46484b0e3ee423028af961..5b1a7ed0636b89405b26672fe2ee163bbc1a52cf 100644 --- a/templates/graph.yml +++ b/templates/graph.yml @@ -99,12 +99,12 @@ spec: key: "$__CACHE_KEY_HACK" policy: $[[ inputs.cache_policy ]] paths: - - $TF_ROOT/.terraform/ + - $GITLAB_TOFU_ROOT_DIR/.terraform/ variables: # FIXME: work around to make slashes work in `cache:key`. see https://gitlab.com/gitlab-org/gitlab/-/issues/439898 __CACHE_KEY_HACK: "$[[ inputs.root_dir ]]" - TF_ROOT: $[[ inputs.root_dir ]] - TF_STATE_NAME: $[[ inputs.state_name ]] + GITLAB_TOFU_ROOT_DIR: $[[ inputs.root_dir ]] + GITLAB_TOFU_STATE_NAME: $[[ inputs.state_name ]] GITLAB_TOFU_VAR_FILE: '$[[ inputs.var_file ]]' image: name: '$[[ inputs.image_registry_base ]]/$[[ inputs.image_name ]]:$[[ inputs.version ]]-opentofu$[[ inputs.opentofu_version ]]-$[[ inputs.base_os ]]$[[ inputs.image_digest ]]' diff --git a/templates/module-release.yml b/templates/module-release.yml index 2d28d39a26bd77176d1580ce9cb669ad6b301b09..2b909e8325c43b0c8808ef5cbcbb7b067b169d31 100644 --- a/templates/module-release.yml +++ b/templates/module-release.yml @@ -23,7 +23,7 @@ spec: description: | System that this module can be used with. e.g. 'local' or 'azurerm' - + module_version: description: | Version of the module that should be published. @@ -37,13 +37,13 @@ spec: image: curlimages/curl:8.8.0 variables: TAR_FILENAME: /tmp/${CI_PROJECT_NAME}-${CI_COMMIT_SHA}.tgz - TF_ROOT: $[[ inputs.root_dir ]] + GITLAB_TOFU_ROOT_DIR: $[[ inputs.root_dir ]] UPLOAD_URL: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/terraform/modules/$[[ inputs.module_name ]]/$[[ inputs.module_system ]]/$[[ inputs.module_version ]]/file script: - >- tar -vczf "$TAR_FILENAME" - -C "$TF_ROOT" + -C "$GITLAB_TOFU_ROOT_DIR" --exclude=./.git --exclude=./.terraform* . diff --git a/templates/plan.yml b/templates/plan.yml index 9c6371ac1697b317d63de6e6524c4cdf9f861873..f016c3522f9b151f12ce300ead942a2bec960955 100644 --- a/templates/plan.yml +++ b/templates/plan.yml @@ -143,21 +143,21 @@ spec: # See https://docs.gitlab.com/ee/ci/yaml/#artifactsaccess for possible values. access: '$[[ inputs.artifacts_access ]]' paths: - - $TF_ROOT/$[[ inputs.plan_name ]].cache + - $GITLAB_TOFU_ROOT_DIR/$[[ inputs.plan_name ]].cache reports: - terraform: $TF_ROOT/$[[ inputs.plan_name]].json + terraform: $GITLAB_TOFU_ROOT_DIR/$[[ inputs.plan_name]].json rules: $[[ inputs.rules ]] cache: key: "$__CACHE_KEY_HACK" policy: $[[ inputs.cache_policy ]] paths: - - $TF_ROOT/.terraform/ + - $GITLAB_TOFU_ROOT_DIR/.terraform/ variables: # FIXME: work around to make slashes work in `cache:key`. see https://gitlab.com/gitlab-org/gitlab/-/issues/439898 __CACHE_KEY_HACK: "$[[ inputs.root_dir ]]" - TF_ROOT: $[[ inputs.root_dir ]] - TF_STATE_NAME: $[[ inputs.state_name ]] - TF_PLAN_NAME: $[[ inputs.plan_name ]] + GITLAB_TOFU_ROOT_DIR: $[[ inputs.root_dir ]] + GITLAB_TOFU_STATE_NAME: $[[ inputs.state_name ]] + GITLAB_TOFU_PLAN_NAME: $[[ inputs.plan_name ]] GITLAB_TOFU_PLAN_WITH_JSON: true GITLAB_TOFU_VAR_FILE: '$[[ inputs.var_file ]]' image: diff --git a/templates/test.yml b/templates/test.yml index c8cc5da1c6824986464751a12536271761b38597..dac7cbcb8e6b7d08c52ae94fec1038942171acff 100644 --- a/templates/test.yml +++ b/templates/test.yml @@ -102,12 +102,12 @@ spec: key: "$__CACHE_KEY_HACK" policy: $[[ inputs.cache_policy ]] paths: - - $TF_ROOT/.terraform/ + - $GITLAB_TOFU_ROOT_DIR/.terraform/ variables: # FIXME: work around to make slashes work in `cache:key`. see https://gitlab.com/gitlab-org/gitlab/-/issues/439898 __CACHE_KEY_HACK: "$[[ inputs.root_dir ]]" - TF_ROOT: $[[ inputs.root_dir ]] - TF_STATE_NAME: $[[ inputs.state_name ]] + GITLAB_TOFU_ROOT_DIR: $[[ inputs.root_dir ]] + GITLAB_TOFU_STATE_NAME: $[[ inputs.state_name ]] GITLAB_TOFU_VAR_FILE: '$[[ inputs.var_file ]]' image: name: '$[[ inputs.image_registry_base ]]/$[[ inputs.image_name ]]:$[[ inputs.version ]]-opentofu$[[ inputs.opentofu_version ]]-$[[ inputs.base_os ]]$[[ inputs.image_digest ]]' diff --git a/templates/validate.yml b/templates/validate.yml index 61e5ac2e434f4d4d4d53d919fc03ca4748a3db62..8ffc7b880dd33c37d02fdbaf2a7cd090d96cd9a8 100644 --- a/templates/validate.yml +++ b/templates/validate.yml @@ -98,13 +98,13 @@ spec: key: "$__CACHE_KEY_HACK" policy: $[[ inputs.cache_policy ]] paths: - - $TF_ROOT/.terraform/ + - $GITLAB_TOFU_ROOT_DIR/.terraform/ variables: # FIXME: work around to make slashes work in `cache:key`. see https://gitlab.com/gitlab-org/gitlab/-/issues/439898 __CACHE_KEY_HACK: "$[[ inputs.root_dir ]]" - TF_ROOT: $[[ inputs.root_dir ]] - TF_STATE_NAME: $[[ inputs.state_name ]] - TF_IGNORE_INIT_ERRORS: 'true' # Tofu can report errors which might be the reason init failed. + GITLAB_TOFU_ROOT_DIR: $[[ inputs.root_dir ]] + GITLAB_TOFU_STATE_NAME: $[[ inputs.state_name ]] + GITLAB_TOFU_IGNORE_INIT_ERRORS: 'true' # Tofu can report errors which might be the reason init failed. GITLAB_TOFU_VAR_FILE: '$[[ inputs.var_file ]]' image: name: '$[[ inputs.image_registry_base ]]/$[[ inputs.image_name ]]:$[[ inputs.version ]]-opentofu$[[ inputs.opentofu_version ]]-$[[ inputs.base_os ]]$[[ inputs.image_digest ]]' diff --git a/tests/integration-tests/Defaults.gitlab-ci.yml b/tests/integration-tests/Defaults.gitlab-ci.yml index 2c51ac10e9ddc19edc825cc2e78c1001f83b406b..7cea12e3e8e6e93e97cd23223b456a260f491731 100644 --- a/tests/integration-tests/Defaults.gitlab-ci.yml +++ b/tests/integration-tests/Defaults.gitlab-ci.yml @@ -5,8 +5,8 @@ include: version: $CI_COMMIT_SHA base_os: $GITLAB_OPENTOFU_BASE_IMAGE_OS opentofu_version: $OPENTOFU_VERSION - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME # Required to run everything immediately, instead of manually. fmt_rules: [{when: always}] validate_rules: [{when: always}] diff --git a/tests/integration-tests/Destroy.gitlab-ci.yml b/tests/integration-tests/Destroy.gitlab-ci.yml index b6e560579ec4c87c3ebf558ef7bb9d26c3c4adfc..2e124b87852383f3e8448ef71e841cb375b32044 100644 --- a/tests/integration-tests/Destroy.gitlab-ci.yml +++ b/tests/integration-tests/Destroy.gitlab-ci.yml @@ -6,8 +6,8 @@ include: opentofu_version: $OPENTOFU_VERSION as: 'setup:apply' stage: setup - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME no_plan: true # Required to run everything immediately, instead of manually. rules: [{when: always}] @@ -17,8 +17,8 @@ include: image_registry_base: $GITLAB_OPENTOFU_IMAGE_BASE version: $CI_COMMIT_SHA opentofu_version: $OPENTOFU_VERSION - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME # Required to run everything immediately, instead of manually. fmt_rules: [{when: always}] validate_rules: [{when: always}] diff --git a/tests/integration-tests/FullPipelineTriggerInChildPipeline.gitlab-ci.yml b/tests/integration-tests/FullPipelineTriggerInChildPipeline.gitlab-ci.yml index 03047e699c9e4b761bb81825c39a8f9be651a130..ee7e62d7dfe5d1e141a2fe0645e66ec2cc186b3b 100644 --- a/tests/integration-tests/FullPipelineTriggerInChildPipeline.gitlab-ci.yml +++ b/tests/integration-tests/FullPipelineTriggerInChildPipeline.gitlab-ci.yml @@ -5,8 +5,8 @@ include: version: $CI_COMMIT_SHA base_os: $GITLAB_OPENTOFU_BASE_IMAGE_OS opentofu_version: $OPENTOFU_VERSION - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME # Required to run everything immediately, instead of manually. fmt_rules: [{when: always}] validate_rules: [{when: always}] diff --git a/tests/integration-tests/JobTemplates.gitlab-ci.yml b/tests/integration-tests/JobTemplates.gitlab-ci.yml index 6c0acc9700cfa6cd1af2e5f49b0506d2560232f4..42d872115c88f6583f48be991e71010014229927 100644 --- a/tests/integration-tests/JobTemplates.gitlab-ci.yml +++ b/tests/integration-tests/JobTemplates.gitlab-ci.yml @@ -4,8 +4,8 @@ include: image_registry_base: $GITLAB_OPENTOFU_IMAGE_BASE version: $CI_COMMIT_SHA opentofu_version: $OPENTOFU_VERSION - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME stages: [validate, build, deploy, cleanup] diff --git a/tests/integration-tests/ModuleRelease.gitlab-ci.yml b/tests/integration-tests/ModuleRelease.gitlab-ci.yml index fd8dc3fac72e1d8ebedde2ffcbd30852c43e0352..6c80fcc0b11ec5a99abc5736516354c4a0b13d91 100644 --- a/tests/integration-tests/ModuleRelease.gitlab-ci.yml +++ b/tests/integration-tests/ModuleRelease.gitlab-ci.yml @@ -5,7 +5,7 @@ variables: include: - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/module-release@$CI_COMMIT_SHA inputs: - root_dir: $TEST_TF_ROOT/modules/random-pet + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR/modules/random-pet module_system: $MODULE_SYSTEM module_version: $MODULE_VERSION diff --git a/tests/integration-tests/TestJob.gitlab-ci.yml b/tests/integration-tests/TestJob.gitlab-ci.yml index 96e90eecd7990cf5976bb604ddd2fa901961f60d..f9094079328894981620fe32af3c7d7c110c2093 100644 --- a/tests/integration-tests/TestJob.gitlab-ci.yml +++ b/tests/integration-tests/TestJob.gitlab-ci.yml @@ -4,8 +4,8 @@ include: image_registry_base: $GITLAB_OPENTOFU_IMAGE_BASE version: $CI_COMMIT_SHA opentofu_version: $OPENTOFU_VERSION - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME # Required to run everything immediately, instead of manually. rules: [{when: always}] diff --git a/tests/integration-tests/TriggerInChildPipeline.gitlab-ci.yml b/tests/integration-tests/TriggerInChildPipeline.gitlab-ci.yml index d1fd3cf410b40189a8d5d9431e0c6743490d8671..fcd4a0cf418de356278e8f583f3975c83c6665ee 100644 --- a/tests/integration-tests/TriggerInChildPipeline.gitlab-ci.yml +++ b/tests/integration-tests/TriggerInChildPipeline.gitlab-ci.yml @@ -5,8 +5,8 @@ include: version: $CI_COMMIT_SHA base_os: $GITLAB_OPENTOFU_BASE_IMAGE_OS opentofu_version: $OPENTOFU_VERSION - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME fmt_rules: [{when: on_success}] validate_rules: [{when: on_success}] plan_rules: [{when: on_success}] @@ -19,7 +19,7 @@ include: - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/delete-state@$CI_COMMIT_SHA inputs: stage: cleanup - state_name: $TEST_TF_STATE_NAME + state_name: $TEST_GITLAB_TOFU_STATE_NAME rules: [{when: always}] stages: [test, cleanup] diff --git a/tests/integration-tests/ValidatePlanDestroyTriggerInChildPipeline.gitlab-ci.yml b/tests/integration-tests/ValidatePlanDestroyTriggerInChildPipeline.gitlab-ci.yml index be08033a372055d01dd130ec7fdfb7dd088e854e..dcc9253f0deed21ae451fdffaa829556b1a5f288 100644 --- a/tests/integration-tests/ValidatePlanDestroyTriggerInChildPipeline.gitlab-ci.yml +++ b/tests/integration-tests/ValidatePlanDestroyTriggerInChildPipeline.gitlab-ci.yml @@ -6,8 +6,8 @@ include: opentofu_version: $OPENTOFU_VERSION as: 'setup:apply' stage: setup - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME no_plan: true # Required to run everything immediately, instead of manually. rules: [{when: always}] @@ -17,8 +17,8 @@ include: image_registry_base: $GITLAB_OPENTOFU_IMAGE_BASE version: $CI_COMMIT_SHA opentofu_version: $OPENTOFU_VERSION - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME # Required to run everything immediately, instead of manually. fmt_rules: [{when: always}] validate_rules: [{when: always}] diff --git a/tests/integration-tests/VarFile.gitlab-ci.yml b/tests/integration-tests/VarFile.gitlab-ci.yml index f1505f31e66bcf391008879f7dfa3d19c9522c8f..dec183d53e8e26f0d42680099f14b4cf9174ea5e 100644 --- a/tests/integration-tests/VarFile.gitlab-ci.yml +++ b/tests/integration-tests/VarFile.gitlab-ci.yml @@ -5,8 +5,8 @@ include: version: $CI_COMMIT_SHA base_os: $GITLAB_OPENTOFU_BASE_IMAGE_OS opentofu_version: $OPENTOFU_VERSION - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME var_file: varfile.integration-test.tfvars # Required to run everything immediately, instead of manually. fmt_rules: [{when: always}] diff --git a/tests/integration-tests/WarningOnNonEmptyPlan.gitlab-ci.yml b/tests/integration-tests/WarningOnNonEmptyPlan.gitlab-ci.yml index 38c4b3fec0f9c161a656b67586a485813a0ecb38..1edddd7d70364a78abb67d8b4381025e72d17b83 100644 --- a/tests/integration-tests/WarningOnNonEmptyPlan.gitlab-ci.yml +++ b/tests/integration-tests/WarningOnNonEmptyPlan.gitlab-ci.yml @@ -5,14 +5,14 @@ include: version: $CI_COMMIT_SHA base_os: $GITLAB_OPENTOFU_BASE_IMAGE_OS opentofu_version: $OPENTOFU_VERSION - root_dir: $TEST_TF_ROOT - state_name: $TEST_TF_STATE_NAME + root_dir: $TEST_GITLAB_TOFU_ROOT_DIR + state_name: $TEST_GITLAB_TOFU_STATE_NAME warning_on_non_empty_plan: true # For CI Terraform state cleanup - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/delete-state@$CI_COMMIT_SHA inputs: - state_name: $TEST_TF_STATE_NAME + state_name: $TEST_GITLAB_TOFU_STATE_NAME rules: [{when: always}] stages: [build, cleanup, verify] diff --git a/tests/integration.gitlab-ci.yml b/tests/integration.gitlab-ci.yml index cef1df354aede6fc09fb6f5d8b5b2a46b543b943..9165372622a0bdd0ae329e2670b0b9baac351726 100644 --- a/tests/integration.gitlab-ci.yml +++ b/tests/integration.gitlab-ci.yml @@ -2,8 +2,8 @@ component: stage: test-integration variables: OPENTOFU_VERSION: $LATEST_OPENTOFU_VERSION - TEST_TF_STATE_NAME: ci-integration-$CI_PIPELINE_IID-$CI_NODE_INDEX - TEST_TF_ROOT: tests/iac + TEST_GITLAB_TOFU_STATE_NAME: ci-integration-$CI_PIPELINE_IID-$CI_NODE_INDEX + TEST_GITLAB_TOFU_ROOT_DIR: tests/iac trigger: include: tests/integration-tests/$PIPELINE_NAME.gitlab-ci.yml strategy: depend diff --git a/tests/unit/gitlab-tofu.bats b/tests/unit/gitlab-tofu.bats index 5b00ef1879529ec32d0fea08091ea368675e0c83..dcdc59c6ea3555002c836c21d0ef14905b685d1f 100644 --- a/tests/unit/gitlab-tofu.bats +++ b/tests/unit/gitlab-tofu.bats @@ -7,7 +7,7 @@ load "$BATS_LIBS/bats-support/load" load "$BATS_LIBS/bats-assert/load" setup() { - export DEBUG_OUTPUT=true + export GITLAB_TOFU_DEBUG=true # Change cwd to test specific directory cd "$BATS_TEST_TMPDIR" @@ -16,14 +16,14 @@ setup() { cp -r "$TEST_PROJECT_DIR" "$BATS_TEST_TMPDIR" # Set TF root directory to temporary location - export TF_ROOT="$BATS_TEST_TMPDIR/$(basename "$TEST_PROJECT_DIR")" + export GITLAB_TOFU_ROOT_DIR="$BATS_TEST_TMPDIR/$(basename "$TEST_PROJECT_DIR")" # Set state so that each test has its own - export TF_STATE_NAME="ci-unit-$CI_JOB_ID-$BATS_SUITE_TEST_NUMBER" + export GITLAB_TOFU_STATE_NAME="ci-unit-$CI_JOB_ID-$BATS_SUITE_TEST_NUMBER" } teardown() { - curl --request DELETE -u "gitlab-ci-token:$CI_JOB_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/terraform/state/$TF_STATE_NAME" + curl --request DELETE -u "gitlab-ci-token:$CI_JOB_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/terraform/state/$GITLAB_TOFU_STATE_NAME" } @test "gitlab-tofu init" { @@ -35,37 +35,37 @@ teardown() { } @test "gitlab-tofu init with environment variable flags" { - export TF_INIT_FLAGS="-get=true -no-color" + export GITLAB_TOFU_INIT_FLAGS="-get=true -no-color" gitlab-tofu init } @test "gitlab-tofu init with args and environment variable flags" { - export TF_INIT_FLAGS="-get=true" + export GITLAB_TOFU_INIT_FLAGS="-get=true" gitlab-tofu init -no-color } -@test "gitlab-tofu init within TF_ROOT set" { - cd "$TF_ROOT" - unset "$TF_ROOT" +@test "gitlab-tofu init within GITLAB_TOFU_ROOT_DIR set" { + cd "$GITLAB_TOFU_ROOT_DIR" + unset "$GITLAB_TOFU_ROOT_DIR" gitlab-tofu init -no-color } @test "gitlab-tofu init without reconfigure" { gitlab-tofu init - cat <<EOF > $TF_ROOT/backend_override.tf + cat <<EOF > $GITLAB_TOFU_ROOT_DIR/backend_override.tf terraform { backend "local" {} } EOF - export TF_INIT_NO_RECONFIGURE="true" + export GITLAB_TOFU_INIT_NO_RECONFIGURE="true" run ! gitlab-tofu init -no-color assert_output --partial "Error: Backend configuration changed" } @test "gitlab-tofu init with reconfigure" { - cat <<EOF > $TF_ROOT/backend_override.tf + cat <<EOF > $GITLAB_TOFU_ROOT_DIR/backend_override.tf terraform { backend "local" {} } @@ -82,56 +82,56 @@ EOF } @test "gitlab-tofu plan" { - export TF_PLAN_CACHE="test-plan.cache" + export GITLAB_TOFU_PLAN_CACHE="test-plan.cache" gitlab-tofu plan - if [ ! -f "$TF_ROOT/$TF_PLAN_CACHE" ]; then + if [ ! -f "$GITLAB_TOFU_ROOT_DIR/$GITLAB_TOFU_PLAN_CACHE" ]; then echo "expected to find a plan.cache file" exit 1 fi gitlab-tofu plan-json - if [ ! -f "$TF_ROOT/plan.json" ]; then + if [ ! -f "$GITLAB_TOFU_ROOT_DIR/plan.json" ]; then echo "expected to find a plan.json file" exit 1 fi } @test "gitlab-tofu apply" { - export TF_PLAN_CACHE="test-plan.cache" + export GITLAB_TOFU_PLAN_CACHE="test-plan.cache" gitlab-tofu plan gitlab-tofu apply } @test "gitlab-tofu destroy" { - export TF_PLAN_CACHE="test-plan.cache" + export GITLAB_TOFU_PLAN_CACHE="test-plan.cache" gitlab-tofu plan gitlab-tofu apply gitlab-tofu destroy } @test "gitlab-tofu validate without implicit init" { - export TF_IMPLICIT_INIT=false + export GITLAB_TOFU_IMPLICIT_INIT=false run ! gitlab-tofu validate -no-color assert_output --partial 'This module is not yet installed' } @test "gitlab-tofu plan without implicit init" { - export TF_IMPLICIT_INIT=false + export GITLAB_TOFU_IMPLICIT_INIT=false run ! gitlab-tofu plan -no-color assert_output --partial 'Error: Backend initialization required' } @test "gitlab-tofu apply without implicit init" { - export TF_IMPLICIT_INIT=false + export GITLAB_TOFU_IMPLICIT_INIT=false run ! gitlab-tofu apply -no-color assert_output --partial 'Error: Failed to load ' } @test "gitlab-tofu destroy without implicit init" { - export TF_IMPLICIT_INIT=false + export GITLAB_TOFU_IMPLICIT_INIT=false run ! gitlab-tofu destroy -no-color assert_output --partial 'Error: Backend initialization required' @@ -139,7 +139,7 @@ EOF @test "gitlab-tofu no wrap" { # NOTE: running `gitlab-tofu apply` wouldn't fail - # because of the implicit `terraform init`. + # because of the implicit `tofu init`. run gitlab-tofu -- apply -no-color assert_failure assert_output --partial 'Error: Backend initialization required, please run "tofu init"' @@ -187,9 +187,9 @@ EOF cat <<'EOF' > test.sh set -x -export TF_STATE_NAME=production/europe +export GITLAB_TOFU_STATE_NAME=production/europe . $(which gitlab-tofu) -test "$TF_STATE_NAME" = "production%2Feurope" +test "$backend_state_name" = "production%2Feurope" EOF $SHELL test.sh @@ -206,7 +206,7 @@ set -x export CI_SERVER_HOST=gitlab.example.com export TF_TOKEN_gitlab_example_com=mysecrettoken . $(which gitlab-tofu) -terraform_authenticate_private_registry +tofu_authenticate_private_registry test "$TF_TOKEN_gitlab_example_com" = "mysecrettoken" EOF @@ -224,7 +224,7 @@ set -x export CI_SERVER_HOST=gitlab.example.com export TF_TOKEN_gitlab_example_com=mysecrettoken . $(which gitlab-tofu) -terraform_authenticate_private_registry +tofu_authenticate_private_registry test "$TF_TOKEN_gitlab_example_com" = "mysecrettoken" EOF