|
2 | 2 |
|
3 | 3 | All CI/CD automation in this project is executed via GitHub Actions, whose workflow files live in this directory. |
4 | 4 |
|
| 5 | +Some of these workflows use hologit or invoke. See the READMEs in [.holo](../../.holo) and [ci](../../ci) for documentation regarding hologit and invoke, respectively. |
| 6 | + |
| 7 | + |
| 8 | +### build-\*.yml workflows |
| 9 | + |
| 10 | +Workflows prefixed with `build-` generally lint, test, and (usually) publish either a Python package or a Docker image. |
| 11 | + |
| 12 | + |
| 13 | +### preview-\*.yml workflows |
| 14 | + |
| 15 | +Workflows prefixed with `preview-` deal with generating previews for pull request changes. |
| 16 | + |
| 17 | + |
| 18 | +## composer-plan-files.yml (TODO: change name to preview-composer) |
| 19 | + |
| 20 | +When creating a PR that changes any file in the `airflow` directory, this workflow runs `Terraform plan` to create a comment with an execution plan to preview the changes that will be made to Staging Airflow DAGs. |
| 21 | + |
| 22 | + |
| 23 | +## composer-apply-files.yml (TODO: change name to deploy-composer) |
| 24 | + |
| 25 | +When merging a PR into the `main` branch with changes to any file in the `airflow` directory, this workflow runs `Terraform apply` to execute the actions proposed in the `Terraform plan` to Staging Airflow DAGs. |
| 26 | + |
| 27 | + |
5 | 28 | ## deploy-airflow.yml |
6 | 29 |
|
7 | 30 | While we're using GCP Composer, "deployment" of Airflow consists of two parts: |
8 | 31 |
|
9 | 32 | 1. Calling `gcloud composer environments update ...` to update the Composer environment with new (or specific versions of) packages |
10 | 33 | 2. Copying the `dags` and `plugins` folders to a GCS bucket that Composer reads (this is specified in the Composer Environment) |
11 | 34 |
|
| 35 | + |
| 36 | +## deploy-airflow-requirements.yml |
| 37 | + |
| 38 | +When merging `airflow/requirements.txt` changes into the `main` branch, this workflow updates composer dependencies. |
| 39 | + |
| 40 | + |
12 | 41 | ## deploy-apps-maps.yml |
13 | 42 |
|
14 | 43 | This workflow builds a static website from the Svelte app and deploys it to Netlify. |
15 | 44 |
|
| 45 | + |
| 46 | +## deploy-dbt.yml |
| 47 | + |
| 48 | +This workflow compiles dbt models and docs, syncs Metabase, uploads dbt artifacts, builds a visual comment about model changes. |
| 49 | + |
| 50 | + |
| 51 | +## deploy-dbt-docs.yml |
| 52 | + |
| 53 | +This workflow generates latest dbt artifacts with extracted questions (cards) and dashboards from Metabase using dbt-metabase exposures, and deploys the dbt documentation. |
| 54 | + |
| 55 | +Developers and analysts can view all dbt documentation and easily verify if particular models are in use by searching inside the `warehouse\models\metabase` folder or on [dbt docs site](https://dbt-docs.calitp.org/#!/overview). |
| 56 | + |
| 57 | +It is scheduled to run every Monday at 8am UTC or when any model changes. |
| 58 | + |
| 59 | + |
16 | 60 | ## deploy-kubernetes.yml |
17 | 61 |
|
18 | 62 | This workflow deploys changes to the production Kubernetes cluster when they get merged into the `main` branch. |
19 | 63 |
|
20 | | -## build-\*.yml workflows |
21 | 64 |
|
22 | | -Workflows prefixed with `build-` generally lint, test, and (usually) publish either a Python package or a Docker image. |
| 65 | +## preview-kubernetes.yml |
23 | 66 |
|
24 | | -## preview-\*.yml workflows |
| 67 | +This workflow renders kubectl diffs on PRs changing cluster content. |
25 | 68 |
|
26 | | -Workflows prefixed with `preview-` deal with generating previews for pull request changes |
27 | 69 |
|
28 | | -- `preview-kubernetes.yml` renders kubectl diffs on PRs changing cluster content |
| 70 | +## publish-docs.yml |
29 | 71 |
|
30 | | -Some of these workflows use hologit or invoke. See the READMEs in [.holo](../../.holo) and [ci](../../ci) for documentation regarding hologit and invoke, respectively. |
| 72 | +This workflow builds jupyter book docs and publishes docs to GitHub Pages. |
| 73 | + |
| 74 | + |
| 75 | +## sentry-release.yml |
| 76 | + |
| 77 | +When merging any changes into the `main` branch, this workflow notify the release to Sentry. |
| 78 | + |
| 79 | + |
| 80 | +## terraform-plan.yml (TODO: change name to preview-terraform) |
| 81 | + |
| 82 | +When creating a PR that changes any file in the `iac` directory (Infrastructure as Code), this workflow runs `Terraform plan` to create a comment with an execution plan to preview the changes that will be made to the infrastructure. |
| 83 | + |
| 84 | + |
| 85 | +## terraform-apply.yml (TODO: change name to deploy-terraform) |
| 86 | + |
| 87 | +When merging a PR into the `main` branch with changes to any file in the `iac` directory (Infrastructure as Code), this workflow runs `Terraform apply` to execute the actions proposed in the `Terraform plan` to the infrastructure. |
0 commit comments