This plugin was originally based on the the AWS S3 Secrets Buildkite Plugin
All secrets are base64 encoded in Vault It currently runs on an AWS based Buildkite stack, but it should work on any agent.
Expose secrets to your build steps. Secrets are stored encrypted-at-rest in HashiCorp Vault.
Different types of secrets are supported and exposed to your builds in appropriate ways:
ssh-agentfor SSH Private Keys- Environment Variables for strings
git-credentialvia git's credential.helper
The following pipeline downloads env secrets stored in https://my-vault-server/secret/buildkite/{pipeline}/env and git-credentials from https://my-vault-server/secret/buildkite/{pipeline}/git-credentials
The keys in the env secret are exposed in the checkout and command as environment variables. The git-credentials are exposed as an environment variable GIT_CONFIG_PARAMETERS and are also exposed in the checkout and command.
steps:
- command: ./run_build.sh
plugins:
- vault-secrets#v0.2.0:
server: "https://my-vault-server"
path: secret/buildkite
auth:
method: approle
role-id: "my-role-id"
secret-env: "VAULT_SECRET_ID"Secrets are uploading using the Vault CLI, as a base64 encoded blob in a field called value.
echo -n $(cat private_ssh_key | base64) | vault write data/buildkite/test-pipeline/private_ssh_key \
value=-examples/ has 2 sample helper script for adding environment variables or ssh keys to Vault for a pipeline.
Environment variable secrets are handled differently in this Vault plugin to the S3 plugin.
Each environment variable is treated as an individually secret under the env or environment nodes for a project.
eg.
project foo/env/var1
project foo/env/var2
etc
- Create policies to manage who can read and update pipeline secrets
The plugin needs at least read and list capabilities for the data. A sample read policy, this could be used by agents.
path "data/buildkite/*" {
capabilities = ["read", "list"]
}
A sample update policy for build engineers or developers. This would allow creation of secrets for pipelines, but not as defaults.
# Allow update of secrets
path "data/buildkite/*" {
capabilities = ["create", "update", "delete", "list"]
}
path "data/buildkite/env" {
capabilities = ["deny"]
}
path "data/buildkite/environment" {
capabilities = ["deny"]
}
path "data/buildkite/git-credentials" {
capabilities = ["deny"]
}
path "data/buildkite/private_ssh_key" {
capabilities = ["deny"]
}
This example uploads an ssh key and an environment file to the base of the Vault secret path, which means it matches all pipelines that use it. You use per-pipeline overrides by adding a path prefix of /my-pipeline/.
# generate a deploy key for your project
ssh-keygen -t rsa -b 4096 -f id_rsa_buildkite
pbcopy < id_rsa_buildkite.pub # paste this into your github deploy key
export my_pipeline=my-buildkite-secrets
echo -n $(cat id_rsa_buildkite | base64) | vault write data/buildkite/my_pipeline/private_ssh_key \
value=-For git over https, you can use a git-credentials file with credential urls in the format of:
https://user:password@host/path/to/repo
vault write data/buildkite/my_pipeline/git-credentials value=- <<< $(echo "https://user:password@host/path/to/repo" | base64)These are then exposed via a gitcredential helper which will download the credentials as needed.
Key values pairs can also be uploaded.
vault write data/buildkite/my_pipeline/environment value=- <<< $(echo "MY_SECRET=blah" | base64)vault write data/buildkite/my_pipeline/env_key value=- <<< $(echo "my secret"| base64)Can be loaded using:
steps:
- command: ./run_build.sh
plugins:
- vault-secrets#v0.1.0:
server: my-vault-server
secrets:
- keydefaults to: data/buildkite
This is expected to be a kv store
Alternative Base Path to use for Vault secrets
To run locally:
BUILDKITE_PLUGIN_VAULT_SECRETS_DUMP_ENV=true BUILDKITE_PLUGIN_VAULT_SECRETS_ADDR=http://0.0.0.0:8200 BUILDKITE_PIPELINE_SLUG=my_pipeline hooks/environmentTo test with BATS:
docker-compose -f docker-compose.yml run --rm testsIntegration test:
.buildkite/steps/test_envvar.shWhen writing test plans, note that secrets are processed in the order they appear in the list returned from the Vault.
- Document use of
TESTER_VAULT_VERSIONversion to set Vault version on tester service - Add
SVC_VAULT_VERSIONto specify version of Vault service - Document use of Makefile
- Merge compose files together
A huge thank you to the original author @mikeknox for providing the framework for this plugin
MIT (see LICENSE)