|
| 1 | +# Bootstrap for Cobalt IaC Pipelines on Azure DevOps |
| 2 | + |
| 3 | +## Overview |
| 4 | +This provides a starting point for anyone interested in having an Azure DevOps |
| 5 | +Project that will 'run' a target Cobalt template whenever that template changes. |
| 6 | + |
| 7 | +## Intended audience |
| 8 | + |
| 9 | +This article is for an infrastructure worker that is brand new to Cobalt templating, and it's *Cobalt Infrastructure |
| 10 | +Template* (CIT) developer workflow, who wants to stand-up Azure DevOps and Azure resources to enact a CI/CD process |
| 11 | +over their infrastructure as code work. |
| 12 | + |
| 13 | +### An example |
| 14 | +Imagine that you are building a new set of web applications which are destined |
| 15 | +to run on Azure. Those applications are going to use a few runtime resources, and you've |
| 16 | +wisely decided to use a Cobalt template to satisfy and centralize your Infrastructure as Code ("IaC") |
| 17 | +needs across all of those applications. |
| 18 | + |
| 19 | +To get started, you want a central Git repository to store your IaC in, |
| 20 | +and following [solid engineering fundamentals](https://github.com/microsoft/code-with-engineering-playbook/tree/master/continuous-integration#continuous-integration), |
| 21 | +you've decided to wrap CI/CD, and staged deployments around your IaC to help |
| 22 | +protect the quality of your infrastructure changes. |
| 23 | + |
| 24 | +This bootstrap (itself a Cobalt template) can be used to quickly, safely, and easily create |
| 25 | +the ADO resources (a Project, Git Repository, Build Definitions, Variable Groups, |
| 26 | +Service Connections, etc) that you'll need in that example. |
| 27 | + |
| 28 | +### Terminology |
| 29 | +Given that we're using a Cobalt template to create resources that will then operate other |
| 30 | +Cobalt templates, discussing matters can become a bit confusing. To help with that, and |
| 31 | +to get specific about a few other terms we'll use, let's define a few terms and then use |
| 32 | +them consistently, within this document and this bootstrap template itself. |
| 33 | + |
| 34 | +|Term|Meaning| |
| 35 | +|----|----| |
| 36 | +|ADO|Azure DevOps, AKA "AzDO," elsewhere| |
| 37 | +|Application(s)|The application that will rely on the resources managed by the Infrastructure Template| |
| 38 | +|Bootstrap Template|This Cobalt template, as described in the example, above| |
| 39 | +|IaC|Infrastructure as Code| |
| 40 | +|Infrastructure Template|Whichever Cobalt template that will manage the Application(s) runtime resources. _By default, the Bootstrap Template points to the 'Hello World' example template in the official Cobalt GitHub repository_| |
| 41 | +|Project|The ADO Project that the Bootstrap Template will add pipeline resources to| |
| 42 | +|IaC Repository|(empty) Git Repository in the Project where the Infrastructure Template is expected to be kept| |
| 43 | + |
| 44 | +## What the Bootstrap Template needs, provisions, and outputs |
| 45 | + |
| 46 | +To work properly, the Bootstrap Template needs some information about how you want things named, how many environments |
| 47 | +are desired (and their details), and little else. The template also creates many kinds of resources |
| 48 | + |
| 49 | +### Decide with Project to use |
| 50 | + |
| 51 | +Visit your ADO Organization site and select or create a project. This project will house the pipelines that |
| 52 | +will operate your IaC CI/CD pipelines. Note that project's name. We'll need to use it as an input, in the next section. |
| 53 | + |
| 54 | +### Inputs |
| 55 | + |
| 56 | +There are two places from where the template consumes input information. |
| 57 | + 1. The environment variables that are set before the template is used-- usually these are `export` shell environment variables. |
| 58 | + 2. The other is a `terraform.tfvars` file. |
| 59 | + |
| 60 | +Let's quickly explore each of these. |
| 61 | + |
| 62 | +#### Environment Variable |
| 63 | + |
| 64 | +You'll need to define a `.env` file in the root of the project. |
| 65 | + |
| 66 | +You can use our environment template file to start with: `cp .env.template .env` |
| 67 | + |
| 68 | +Provide values for the environment values in `.env` which are required to allow |
| 69 | +Terraform to provision resources within your subscription, and your Azure DevOps Organization. |
| 70 | + |
| 71 | +```bash |
| 72 | +ARM_SUBSCRIPTION_ID="<az-service-principal-subscription-id>" |
| 73 | +ARM_CLIENT_ID="<az-service-principal-client-id>" |
| 74 | +ARM_CLIENT_SECRET="<az-service-principal-auth-secret>" |
| 75 | +ARM_TENANT_ID="<az-service-principal-tenant>" |
| 76 | +ARM_ACCESS_KEY="<remote-state-storage-account-primary-key>" |
| 77 | +AZDO_PERSONAL_ACCESS_TOKEN="<see, below>" |
| 78 | +AZDO_ORG_SERVICE_URL="<typically: https://dev.azure.com/__YOUR_ORG__/" |
| 79 | +``` |
| 80 | + |
| 81 | +_Note: you can find out more information about the Azure DevOps personal access token [here](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page)._ |
| 82 | + |
| 83 | +#### Terraform Variables |
| 84 | + |
| 85 | +The Environments and Azure Subscription details, for each, need to be provided in the `terraform.tfvars` file. |
| 86 | +It should take the form of: |
| 87 | + |
| 88 | +```hcl-terraform |
| 89 | +environments = [{ |
| 90 | + environment: "dev1", |
| 91 | + az_sub_id: "1aaaaaa1-bbbb-cccc-dddd-eeeeee" |
| 92 | +},{ |
| 93 | + environment: "dev2", |
| 94 | + az_sub_id: "4aaaaaa4-bbbb-cccc-dddd-eeeeee" |
| 95 | +},{ |
| 96 | + environment: "prod", |
| 97 | + az_sub_id: "1aaaaaa1-bbbb-cccc-dddd-eeeeee" |
| 98 | +}] |
| 99 | +``` |
| 100 | + |
| 101 | +In the structure above, you can indicate as many environments as needed. Each should have a unique `environment` value, |
| 102 | +and each will need an accurate `az_sub_id`. For each environment, `az_sub_id` is the Azure Subscription ID that the |
| 103 | +Infrastructure Template will eventually provision resources into. |
| 104 | + |
| 105 | +You'll also need to specify the name of the Project (_The ADO Project that the Bootstrap Template will add pipeline resources to_). |
| 106 | +The Bootstrap Template can be used with an existing Azure DevOps Project ("the Project"), or it can create |
| 107 | +a new project. Take a look at the `azdo.tf` file, around lines 12-34 to see examples of how to use the template |
| 108 | +for either choice. |
| 109 | + |
| 110 | +In either case, you'll need to provide the name of the Project to use with a line like this: |
| 111 | +```hcl-terraform |
| 112 | +project_name = "My CICD Project Name" |
| 113 | +``` |
| 114 | + |
| 115 | +### Provisioned Resources |
| 116 | + |
| 117 | +The Bootstrap Template creates a few resources in the Azure Subscription(s) and the Azure DevOps Organization that you |
| 118 | +pointed to with the inputs, above. These resources establish the CI/CD pipelines in ADO, and map those pipelines to |
| 119 | +the appropriate Azure Subscriptions. |
| 120 | + |
| 121 | +Here's a list of the resources that are provisioned including what a typical name might be, what kind of resource it is, |
| 122 | +and some extra detail about the purpose of the resource. |
| 123 | + |
| 124 | +| Name (typical) | Resource Type | Description | |
| 125 | +| --- | --- | --- | |
| 126 | +| Infrastructure Pipeline Variables | azuredevops_variable_group | Common values that all pipelines share | |
| 127 | +| Infrastructure Pipeline Variables - *prod* | azuredevops_variable_group | Environment-specific values, per pipeline | |
| 128 | +| Infrastructure Pipeline Secrets - *prod* | azuredevops_variable_group | Environment-specific secret values, per pipeline | |
| 129 | +| Infrastructure Repository | azuredevops_git_repository | The Git repository where Infrastructure templates are stored and watched for changes | |
| 130 | +| Infrastructure CICD | azuredevops_build_definition | The Build Definition that watches for IaC Template changes, and then triggers the Infrastructure pipelines | |
| 131 | +| Infrastructure Deployment Service Connection | azuredevops_serviceendpoint_azurerm | The service connection that the Infrastructure Template will run under | |
| 132 | +| bootstrap-deploy-app | azuread_application | app | |
| 133 | +| n/a | azuread_service_principal | The Azure Service Principal that the Infrastructure Template will run under | |
| 134 | +| n/a | azurerm_role_assignment | The Azure Service Principal role assignment that the Infrastructure Template will run under | |
| 135 | +| bootstrap-iac-tf-workspaces | azurerm_resource_group | The Azure Resource Group which houses the remote state containers for Terraform | |
| 136 | +| iactf*prod* | azurerm_storage_account | An Azure Storage Account to house remote state containers, for a given environment | |
| 137 | +| tfstate | azurerm_storage_container | The Azure Blob Storage Container, within an environment-specific Azure Storage Account that will house remote state containers for Terraform | |
| 138 | +| *project_name* | The ADO Project that the Bootstrap Template will add pipeline resources to | |
| 139 | + |
| 140 | +### Outputs |
| 141 | + |
| 142 | +After the Bootstrap Template has been applied (via `terraforma apply`) you will see some output reflecting the |
| 143 | + |
| 144 | +| name | description | |
| 145 | +| ---- | ----------- | |
| 146 | +| project_id | The ID of the Project that was provisioned | |
| 147 | +| project_name | The name of the Project that was provisioned | |
| 148 | +| repo_clone_url | The *https* (not the *ssh*) URL of the IaC Repository that was created | |
| 149 | + |
| 150 | +## Running the Bootstrap Template |
| 151 | + |
| 152 | +### Set-up |
| 153 | + |
| 154 | +For the purposes of this Bootstrap Template, you'll need a few tools to be ready on your workstation. |
| 155 | + |
| 156 | +Please see the quick start guide's list of prerequisites: *[quick-start guide prerequisites](https://github.com/microsoft/cobalt/blob/master/docs/2_QUICK_START_GUIDE.md#2.3-prerequisites).* |
| 157 | + |
| 158 | +## Example Usage |
| 159 | + |
| 160 | +For this example, we'll be using `bash` running on a unix, such as OSX, Ubuntu or Windows Subsystem for Linux. |
| 161 | + |
| 162 | +1. Navigate to the directory where the `ado-bootstrap-iac-pipeline` template (where this `README.md` file is at). |
| 163 | + |
| 164 | +1. Execute the following commands to set the environment variables that the template will require |
| 165 | +for Terraform, Azure and Azure DevOps. This will take the values in an .env file and export them in the local |
| 166 | +shell: |
| 167 | + |
| 168 | +```bash |
| 169 | +# these commands `export` the environment variables needed to run this template into the local shell |
| 170 | +DOT_ENV=<path to your .env file -- see .env.template for more information> |
| 171 | +export $(cat $DOT_ENV | xargs) |
| 172 | +``` |
| 173 | + |
| 174 | +1. Execute the following commands to set up your terraform workspace. |
| 175 | + |
| 176 | +_Note: the Bootstrap Template creates the Azure resources needed for handling Terraform remote state, but |
| 177 | +we're not using remote state configuration for this._ |
| 178 | + |
| 179 | +```bash |
| 180 | +# This command configures terraform to use a local workspace unique to you. |
| 181 | +terraform init |
| 182 | +``` |
| 183 | + |
| 184 | +5. Execute the following commands to orchestrate a deployment. |
| 185 | + |
| 186 | +```bash |
| 187 | +# See what terraform will try to deploy without actually deploying |
| 188 | +terraform plan |
| 189 | +``` |
| 190 | + |
| 191 | +Review the output from the `plan` and make sure everything looks right. |
| 192 | + |
| 193 | +For example, you might want the names of environments, or Resource Groups to be slightly different. |
| 194 | + |
| 195 | +Also, if you happen to be running the Bootstrap Template |
| 196 | +a second time (after a previous `apply` operation), then you'll want to be careful that Terraform isn't removing or |
| 197 | +replacing any resources that you didn't intend it to. |
| 198 | + |
| 199 | +When everything looks good, you can let Terraform create everything in the plan: |
| 200 | + |
| 201 | +```bash |
| 202 | +# Execute a deployment |
| 203 | +terraform apply |
| 204 | +``` |
| 205 | + |
| 206 | +6. Contribute your Cobalt IaC repo to the newly provisioned Infrastructure Repository. |
| 207 | + |
| 208 | +7. Trigger the Infrastructure CICD pipeline that was provisioned into the Project. |
| 209 | + |
| 210 | +8. Authorize the Infrastructure CICD pipeline to access the variable groups that were provisioned by the into the Project. |
0 commit comments