Vault - Enabling JWT Secret Engine and Creating Secrets (Part II)

Assuming you have fallowed my previous guides on how to deploy Vault and your instance is up and running. Let's enable it to GitLab Runner.

1) Root Token

Use the root token with caution. Hashicorp recommends its use in production environments only for the initial setup or emergencies.

Login into your container and export your token to an environment variable.

export VAULT_TOKEN=<<SECRET_TOKEN>>

2) Enable JWT

GitLab Runner login using Json Web Token engine. Let's enable it.

vault auth enable jwt
Success! Enabled jwt auth method at: jwt/

3) Create JSON Web Key Set Auth Endpoint

Following the GitLab documentation the next step is to enable the JSON Web Key Set endpoint for the GitLab Authentication.

vault write auth/jwt/config \
	jwks_url="https://git.infoitech.co.uk/-/jwks" \
    bound_issuer="git.infoitech.co.uk"
Success! Data written to: auth/jwt/config

4) Enable KV2 Secrets Engine

GitLab is only compatible with the Key/Value V2 secrets engine at the moment this article is written.

Vault has an excelent documentation about secret engines. Let's also use the KV2 documentation to enable the secrets engine that our GitLab instance will use.

vault secrets enable kv-v2
Success! Enabled the kv-v2 secrets engine at: kv-v2/

5) Creating a Secret

In the step above we have enabled a secrets engine that has created the kv-v2 path. We will write a secret in that path. Vault has this guide with further information.

You can list the paths available with the command:

vault secrets list
Path          Type         Accessor              Description
----          ----         --------              -----------
cubbyhole/    cubbyhole    cubbyhole_            per-token private secret storage
identity/     identity     identity_             identity store
kv-v2/        kv           kv_                   n/a
sys/          system       system_               system endpoints used for control, policy and debugging

Use the command below to write a new secret.

vault kv put -mount=kv-v2 tucana/hv2/proxmox/api token="<<TOKEN>>" username="<<USERNAME>>"
======== Secret Path ========
kv-v2/data/tucana/hv2/proxmox/api

======= Metadata =======
Key                Value
---                -----
created_time       2022-09-05T14:08:22.739607087Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

6) Creating Policies

Policies in Vault controls what a user can access. Vault documentation has a more detailed explanation of how the policies works.

vault policy write tucana-hv2-proxmox-api - <<EOF
# Read-only permission on 'kv-v2/data/tucana/hv2/proxmox/api/*' path
 
path "kv-v2/data/tucana/hv2/proxmox/api" {
  capabilities = [ "read" ]
}
EOF
Success! Uploaded policy: tucana-hv2-proxmox-api

The policy above grants read access to our endpoint.

7) Creating Roles

When GitLab attempts to authenticate with Vault it creates a role.

JWT registers a role that matches to a policy and allow access to an endpoint.

You can check the link above for a more detailed explanation. Let's create a role to allow access to our to the kv-v2/data/tucana/hv2/proxmox/api/* issendpoint.

vault write auth/jwt/role/terraform-pid-51 - <<EOF
{
  "role_type": "jwt",
  "policies": ["tucana-hv2-proxmox-api"],
  "token_explicit_max_ttl": 60,
  "user_claim": "user_email",
  "bound_claims_type": "glob",
  "bound_claims": {
    "project_id": "51"
  }
}
EOF
Success! Data written to: auth/jwt/role/terraform-pid-51

8) Accessing Vault Secrets

There are many ways to access vault secrets. I will use the root token for this project however keep in mind that is dangerous to use your root token.

We can login to the vault container executing:

docker exec -it <<VAULT_CONTAINER_NAME>> /bin/sh

Once logged in export your root token to the environment variable VAULT_TOKEN.

export VAULT_TOKEN="<<YOUR_ROOT_TOKEN>>"

We can now issue commands with full privileges in the vault instance.

Before issuing commands to Vault you will need to unseal your vault.

There are many ways to read secrets from vault. If you are using the kv secrets engine. We can use the command below to read the secret created in step 5 of this guide.

vault kv get -field=username kv-v2/tucana/hv2/proxmox/api
hv2********@pve!hv2XXXXXXXXX
Vault returned the secret on the console.

After you finish with the desired operations vault can be resealed.

Keep tuned, there will be another guide showing how to access vault secrets with gitlab CI/CD.

Troubleshooting

If you get the below error when trying to set the JSON Web Key Set. It could be because of self-signed certificates. I have replaced my self-signed certificates with Let's Encrypt ones and it worked just fine. GitLab documentation has a solution to self-signed certs.

# vault write auth/jwt/config \
> jwks_url="https://git.infoitech.co.uk/-/jwks" \
> bound_issuer="git.infoitech.co.uk"
Error writing data to auth/jwt/config: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/auth/jwt/config
Code: 400. Errors:

* error checking jwks URL: fetching keys oidc: get keys failed Get "https://git.infoitech.co.uk/-/jwks": x509: certificate signed by unknown authority