GitlLab - Retrieving Secrets from Hashicorp Vault

GitLab has built in code to retrieve secrets from a Hashicorp Vault instance. However, this feature is only available to premium users.

This GitLab documentation details how to use the premium feature.

If you do not have a premium account it is still possible to fetch Vault secrets. Although the solution is not elegant as on the premium tiers. It works!

I am not going to rewrite the excellent GitLab documentation explaining how to get secrets from Vault without a premium account.

The code below has a CI/CD Job from one of my projects to retrieve secrets from my Vault instance and made it available as artifacts to other Jobs.

variables:
  VAULT_SERVER_URL: "https://vault.infoitech.co.uk"
  VAULT_AUTH_ROLE: "terraform-pid-51" 

# ...
# ...
# ...

# Get Secrets from Vault
get-secrets:
  stage: get-secrets
  environment:
    name: staging
  image: 
    name: vault:latest
    entrypoint:
      - '/usr/bin/env'
      - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'  
  script:
    # Check job's ref name
    - echo $CI_COMMIT_REF_NAME
    # and is this ref protected
    - echo $CI_COMMIT_REF_PROTECTED
    # Vault address can be provided here or as a CI/CD Variable
    - export VAULT_ADDR="${VAULT_SERVER_URL}"
    # Authenticate and get token. Token expiry time and other properties can be configured.
    # when configuring JWT auth.    
    - export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=terraform-pid-51 jwt=$CI_JOB_JWT)"    
    # Now use the vault token to read the secret and store it in an environment variable.
    # Proxmox API ID            
    - export PM_API_TOKEN_ID=$(vault kv get -field=username kv-v2/tucana/hv2/proxmox/api)
    - export PM_API_TOKEN_SECRET=$(vault kv get -field=token kv-v2/tucana/hv2/proxmox/api)
    # Test if the Token ID was returned.
    - |
      if [ -z "${PM_API_TOKEN_ID}" ]; then
        echo "[ERROR] Proxmox API Token ID is empty."
        exit 1
      fi        
# Test if the Token Secret was returned.    
    - |
      if [ -z "${PM_API_TOKEN_SECRET}" ]; then
        echo "[ERROR] Proxmox API Token Secret is empty."
        exit 1
      fi
# Save ENV vars into the artifact file.
    - echo "PM_API_TOKEN_ID=${PM_API_TOKEN_ID}" >> proxmox-secrets.env
    - echo "PM_API_TOKEN_SECRET=${PM_API_TOKEN_SECRET}" >> proxmox-secrets.env    
  artifacts:
    public: false
    when: on_success
    reports:
      dotenv: proxmox-secrets.env