Add a Terraform configuration to deploy lnt.llvm.org #128

ldionne · 2025-10-31T16:18:57Z

This patch adds a Terraform configuration file that should allow deploying to an EC2 instance. It requires a few secrets to be made available to Github Actions.

ldionne · 2025-10-31T16:19:43Z

docker/lnt.llvm.org/main.tf

+}
+
+resource "aws_instance" "docker_server" {
+  ami           = "ami-0c97bd51d598d45e4" # Amazon Linux 2023 kernel-6.12 AMI in us-west-2


@boomanaiden154 Are we OK with hardcoding the AMI? What do you folks usually do?

Not familiar with how AWS does things. Hard coding it doesn't seem like a big deal. But we want to be able to change it, which would probably force instance recreation. I think we should do what I suggested above where the instance is a clean slate on every boot but mounts a persistent volume that has the DB info.

ldionne · 2025-10-31T16:20:11Z

docker/lnt.llvm.org/main.tf

+resource "aws_instance" "docker_server" {
+  ami           = "ami-0c97bd51d598d45e4" # Amazon Linux 2023 kernel-6.12 AMI in us-west-2
+  instance_type = "t2.micro"
+  key_name      = "test-key-name" # TODO


I'm not sure what to put here, I presume this needs to match a key in the LLVM Foundation's actual AWS account.

Any keys should be specified in the provider. I think this is a different type of key.

ldionne · 2025-10-31T16:21:00Z

.github/workflows/deploy-lnt.llvm.org.yaml

+
+on:
+  push:
+    tags:


I am deploying on tags at the moment: I don't think we want to re-deploy at every commit since we risk bringing down the instance. Actually, I even wonder whether that should be a manually triggered job. WDYT?

We want to tag images by commit SHA, but explicitly version them in the terraform. That means we get a new image per commit, but only redeploy when we explicitly bump the commit of the images we're running.

ldionne · 2025-10-31T16:23:11Z

.github/workflows/deploy-lnt.llvm.org.yaml

+    - name: Initialize Terraform
+      run: terraform init
+
+    - name: Apply Terraform changes


If the instance already exists and then we re-deploy it, what's going to happen? My understanding is that we'd start over from scratch with an empty EC2 instance, which means we would lose all of the existing data stored on the instance. Is that not the case?

Do you understand the mechanism by which the data we store in VOLUMES in the Docker container end up being persisted across re-deployments of the EC2 instance? I don't.

I'm pretty sure it just calls whatever AWS API calls it needs to update the instance to match your terraform file, it won't get destroyed. Terraform holds state about this type of stuff.

The volume will just be stored on the root block device since we haven't attached any EBS storage or anything.

I'm pretty sure it just calls whatever AWS API calls it needs to update the instance to match your terraform file, it won't get destroyed. Terraform holds state about this type of stuff.

I see. But the Terraform state is not kept across invocations of the Github Action, so I don't really understand how Terraform can tell that we even already have an instance.

ldionne · 2025-10-31T16:24:15Z

CC @lukel97 I went ahead and gave this a shot, I was curious to understand the whole pipeline

ldionne · 2025-10-31T16:26:44Z

@boomanaiden154 I also created the appropriately-named secrets in the Github Actions of this repository, however they all have fake values at the moment.

lukel97

Thanks for fleshing this out, I'm not sure if you've tried deploying this to a test AWS account yet but it looks like it's missing a security group/ingress rules etc., so the web server won't be reachable by any public traffic IIUC.

I've also got some terraform files written here, it would be good to collaborate on this. I don't want us to step on each others toes so I'll just leave review comments for now but let me know if you'd rather have me just commit directly to the branch.

lukel97 · 2025-10-31T16:30:42Z

.github/workflows/deploy-lnt.llvm.org.yaml

+    - name: Initialize Terraform
+      run: terraform init
+
+    - name: Apply Terraform changes


I'm pretty sure it just calls whatever AWS API calls it needs to update the instance to match your terraform file, it won't get destroyed. Terraform holds state about this type of stuff.

The volume will just be stored on the root block device since we haven't attached any EBS storage or anything.

docker/lnt.llvm.org/ec2-startup.sh.tpl

ldionne · 2025-10-31T16:42:36Z

I've also got some terraform files written here, it would be good to collaborate on this. I don't want us to step on each others toes so I'll just leave review comments for now but let me know if you'd rather have me just commit directly to the branch.

Please feel free to commit directly to the branch. Sorry, I didn't know you had started on this already.

I did try to deploy an EC2 instance in my personal account, however that account is blocked right now (IDK why) so I haven't gotten very far. This was intended to be a starting point.

Feel free to push whatever changes you have to the branch.

lukel97 · 2025-10-31T16:39:37Z

docker/lnt.llvm.org/main.tf

+
+resource "aws_instance" "docker_server" {
+  ami           = "ami-0c97bd51d598d45e4" # Amazon Linux 2023 kernel-6.12 AMI in us-west-2
+  instance_type = "t2.micro"


The default block storage on these devices are tiny (~8GB IIRC?), you probably want to expand it to a few GB more

Suggested change

instance_type = "t2.micro"

instance_type = "t2.micro"

root_block_device {

volume_size = 64

volume_type = "gp3"

}

A couple GB boot disk should be fine, but slightly bigger might be good. The DB should probably be on a separate volume.

lukel97 · 2025-10-31T16:40:42Z

docker/lnt.llvm.org/ec2-startup.sh.tpl

+
+LNT_DB_PASSWORD=${__db_password__}
+LNT_AUTH_TOKEN=${__auth_token__}
+docker compose --file compose.yaml up


I think we need to daemonize this otherwise cloudinit will never finish

Suggested change

docker compose --file compose.yaml up

docker compose --file compose.yaml up -d

lukel97 · 2025-10-31T16:42:40Z

docker/lnt.llvm.org/ec2-startup.sh.tpl

+
+LNT_DB_PASSWORD=${__db_password__}
+LNT_AUTH_TOKEN=${__auth_token__}
+docker compose --file compose.yaml up


IIUC these user data scripts are only called when the instance is first initialized, but not e.g. rebooted. So we probably want to change the docker-compose restart policy to be unless-stopped so the containers get relaunched on a reboot

This depends upon how we set it up. I was thinking it might be better to setup the machine to be a clean slate on every boot, and mount a persistent volume that actually contains the DB. That makes it super easy to change system software inside TF.

lukel97 · 2025-10-31T16:45:14Z

I did try to deploy an EC2 instance in my personal account, however that account is blocked right now (IDK why) so I haven't gotten very far. This was intended to be a starting point.

Hah, my AWS account was also blocked, I'm currently waiting for AWS support to verify my identity. I feel your pain :)

boomanaiden154 · 2025-10-31T20:16:26Z

.github/workflows/deploy-lnt.llvm.org.yaml

+
+on:
+  push:
+    tags:


We want to tag images by commit SHA, but explicitly version them in the terraform. That means we get a new image per commit, but only redeploy when we explicitly bump the commit of the images we're running.

boomanaiden154 · 2025-10-31T20:17:30Z

docker/lnt.llvm.org/ec2-startup.sh.tpl

+sudo usermod -a -G docker ec2-user
+sudo chkconfig docker on
+
+LNT_DB_PASSWORD=${__db_password__}


Where are these env variables coming from?

boomanaiden154 · 2025-10-31T20:18:14Z

docker/lnt.llvm.org/ec2-startup.sh.tpl

+
+LNT_DB_PASSWORD=${__db_password__}
+LNT_AUTH_TOKEN=${__auth_token__}
+docker compose --file compose.yaml up


This depends upon how we set it up. I was thinking it might be better to setup the machine to be a clean slate on every boot, and mount a persistent volume that actually contains the DB. That makes it super easy to change system software inside TF.

boomanaiden154 · 2025-10-31T20:19:22Z

docker/lnt.llvm.org/main.tf

+provider "aws" {
+  region = "us-west-2"
+}
+


We also need a way to set the terraform state. We use a GCS bucket in the premerge cluster to do this. https://github.com/llvm/llvm-zorg/blob/87d07e600970abf419046d2ab6083b2d64240bce/premerge/main.tf#L31

Otherwise state isn't saved across checkouts, which means things won't work.

boomanaiden154 · 2025-10-31T20:20:06Z

docker/lnt.llvm.org/main.tf

+  region = "us-west-2"
+}
+
+variable "lnt_db_password" {


These should probably be data resources that reference secrets stored inside AWS's secret manager.

https://github.com/llvm/llvm-zorg/blob/87d07e600970abf419046d2ab6083b2d64240bce/premerge/main.tf#L113 is how we set this up for premerge. Not sure exactly how to do this for AWS.

boomanaiden154 · 2025-10-31T20:20:56Z

docker/lnt.llvm.org/main.tf

+}
+
+resource "aws_instance" "docker_server" {
+  ami           = "ami-0c97bd51d598d45e4" # Amazon Linux 2023 kernel-6.12 AMI in us-west-2


Not familiar with how AWS does things. Hard coding it doesn't seem like a big deal. But we want to be able to change it, which would probably force instance recreation. I think we should do what I suggested above where the instance is a clean slate on every boot but mounts a persistent volume that has the DB info.

boomanaiden154 · 2025-10-31T20:21:15Z

docker/lnt.llvm.org/main.tf

+
+resource "aws_instance" "docker_server" {
+  ami           = "ami-0c97bd51d598d45e4" # Amazon Linux 2023 kernel-6.12 AMI in us-west-2
+  instance_type = "t2.micro"


A couple GB boot disk should be fine, but slightly bigger might be good. The DB should probably be on a separate volume.

boomanaiden154 · 2025-10-31T20:21:41Z

docker/lnt.llvm.org/main.tf

+resource "aws_instance" "docker_server" {
+  ami           = "ami-0c97bd51d598d45e4" # Amazon Linux 2023 kernel-6.12 AMI in us-west-2
+  instance_type = "t2.micro"
+  key_name      = "test-key-name" # TODO


Any keys should be specified in the provider. I think this is a different type of key.

Add a Terraform configuration to deploy lnt.llvm.org

5139b61

This patch adds a Terraform configuration file that should allow deploying to an EC2 instance. It requires a few secrets to be made available to Github Actions.

ldionne requested a review from boomanaiden154 October 31, 2025 16:18

ldionne commented Oct 31, 2025

View reviewed changes

Try to fix issue with docker compose file

fbc9cb4

lukel97 reviewed Oct 31, 2025

View reviewed changes

Install docker compose plugin

c4577c9

boomanaiden154 reviewed Oct 31, 2025

View reviewed changes

-  instance_type = "t2.micro"
+  instance_type = "t2.micro"
+  root_block_device {
+    volume_size = 64
+    volume_type = "gp3"
+  }

	docker compose --file compose.yaml up
	docker compose --file compose.yaml up -d

Add a Terraform configuration to deploy lnt.llvm.org #128

Are you sure you want to change the base?

Add a Terraform configuration to deploy lnt.llvm.org #128

Uh oh!

Conversation

ldionne commented Oct 31, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ldionne commented Oct 31, 2025

Uh oh!

ldionne commented Oct 31, 2025

Uh oh!

lukel97 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ldionne commented Oct 31, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lukel97 commented Oct 31, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants