terraform-google-composer

This module makes it easy to create a Cloud Composer Environment. As the module develops, this README should be updated.

The resources/services/activations/deletions that this module will create/trigger are:

• Create a GCP Composer Environment

Compatibility

This module is meant for use with Terraform 1.3+ and tested using Terraform 1.3+. If you find incompatibilities using Terraform >=1.3, please open an issue.

Version

Current version is 4.0. Upgrade guides:

• 3.X -> 4.0.
• 4.X -> 5.0.

Usage

Cloud Composer currently has two versions: V2 has greater flexibility in the Airflow core services (scheduler, webserver, worker) and has a more practical and scalable infrastructure. Therefore, we recommend prioritizing the use of V2 for new environments.

You can find an overview of the product here and the list of major differences. Plans for Cloud Composer V1 end of support is documented here.

Simple usage is as follows:

module "composer" {
  source  = "terraform-google-modules/composer/google"
  version = "~> 5.2"

  project_id        = "<PROJECT ID>"
  region            = "us-central1"
  composer_env_name = "composer-env-test"
  network           = "test-network"
  subnetwork        = "composer-subnet"
  enable_private_endpoint = false
}

Functional examples are included in the examples directory.

Inputs

Name	Description	Type	Default	Required
composer_env_name	Name of Cloud Composer Environment	string	n/a	yes
enable_private_endpoint	Configure public access to the cluster endpoint.	bool	false	no
network	Network where Cloud Composer is created.	string	n/a	yes
project_id	Project ID where Cloud Composer Environment is created.	string	n/a	yes
region	Region where the Cloud Composer Environment is created.	string	n/a	yes
subnetwork	Subetwork where Cloud Composer is created.	string	n/a	yes
zone	Zone where the Cloud Composer Environment is created.	string	n/a	yes

Outputs

Name	Description
airflow_uri	URI of the Apache Airflow Web UI hosted within the Cloud Composer Environment.
composer_env_id	ID of Cloud Composer Environment.
composer_env_name	The name of the Cloud Composer Environment.
gcs_bucket	Google Cloud Storage bucket which hosts DAGs for the Cloud Composer Environment.
gke_cluster	Google Kubernetes Engine cluster used to run the Cloud Composer Environment.

Requirements

These sections describe requirements for using this module.

Software

The following dependencies must be available:

• Terraform v1.3+
• Terraform Provider for GCP plugin v5.3+

Service Account

A service account with the following roles must be used to provision the resources of this module:

• Project Editor: roles/editor
• Network Admin: roles/compute.networkAdmin
• Instance Admin: roles/compute.instanceAdmin.v1
• Service Account User: roles/iam.serviceAccountUser
• Composer Worker: roles/composer.worker

The Project Factory module and the IAM module may be used in combination to provision a service account with the necessary roles applied.

APIs

A project with the following APIs enabled must be used to host the resources of this module:

• Cloud Composer API: composer.googleapis.com

The Project Factory module can be used to provision a project with the necessary APIs enabled.

Contributing

Refer to the contribution guidelines for information on contributing to this module.