-
Notifications
You must be signed in to change notification settings - Fork 91
Add an autoscaling group for the docs-rs-builder #243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,57 @@ | ||
| // The instance profile the builder will assume when communicating with | ||
| // other AWS services. | ||
| // The autoscaling group for the builder | ||
|
|
||
| resource "aws_autoscaling_group" "builder" { | ||
| name = "docs-rs-builder" | ||
| vpc_zone_identifier = var.cluster_config.subnet_ids | ||
| max_size = var.max_num_builder_instances | ||
| min_size = var.min_num_builder_instances | ||
| # Let the instances get warm | ||
| default_instance_warmup = 60 | ||
|
|
||
| launch_template { | ||
| id = aws_launch_template.builder.id | ||
| version = "$Latest" | ||
| } | ||
| } | ||
|
|
||
| resource "aws_launch_template" "builder" { | ||
| name_prefix = "builder" | ||
| image_id = data.aws_ami.builder.id | ||
| instance_type = "t2.large" | ||
|
|
||
| network_interfaces { | ||
| associate_public_ip_address = true | ||
| security_groups = [aws_security_group.builder.id] | ||
| } | ||
|
|
||
| iam_instance_profile { | ||
| arn = aws_iam_instance_profile.builder.arn | ||
| } | ||
|
|
||
| block_device_mappings { | ||
| device_name = "/dev/sda1" | ||
|
|
||
| ebs { | ||
| volume_size = 64 | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Seeing the current filesystem usage (~100 GB) I would prefer something at least double this size. ( of course the current usage also includes the database & some web cache)
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be nice if we didn't need so much storage. I believe a large part of that storage is only needed during a single crate's build and afterwards can be deleted, no? Would it be possible to add some clean up to the builder process so that the filesystem usage doesn't grow so large?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hm.. we are cleaning up after the build. Plus the cleanup tasks for docker images, which are in cron right now. Only looking at the above I could totally imagine to just try with the current definition above, let the builder build, and watch how much space is used. ( assuming the big docker image is configured?) But, we're also planning on adding some build artifact caching: rust-lang/docs.rs#1757
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep, we just have a daily cronjob (systemd-timer) running
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where is the cronjob currently configured? I can add this to the Ansible configuration (though I wouldn't block merging this).
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's in |
||
| delete_on_termination = true | ||
| } | ||
| } | ||
|
|
||
| tag_specifications { | ||
| resource_type = "instance" | ||
|
|
||
| tags = { | ||
| Name = "docs-rs-builder" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| data "aws_ami" "builder" { | ||
| most_recent = true | ||
| name_regex = "^docs-rs-builder-*" | ||
| } | ||
|
|
||
| // The instance profile the builder will assume when communicating with s3 | ||
|
|
||
| resource "aws_iam_instance_profile" "builder" { | ||
| name = "builder" | ||
|
|
@@ -47,3 +99,41 @@ resource "aws_iam_role_policy" "builder_s3" { | |
| ] | ||
| }) | ||
| } | ||
|
|
||
| // Security group allowing all egress and ssh ingress from the bastion instance | ||
| resource "aws_security_group" "builder" { | ||
| vpc_id = var.cluster_config.vpc_id | ||
| name = "docs-rs-builder" | ||
| description = "Access rules for the docs-rs builder." | ||
|
|
||
| // SSH access from the bastion instance | ||
| ingress { | ||
| from_port = 22 | ||
| to_port = 22 | ||
| protocol = "tcp" | ||
| description = "SSH access from bastion" | ||
| security_groups = [aws_security_group.web.id] | ||
| } | ||
|
|
||
| // Allow outgoing connections | ||
|
|
||
| egress { | ||
| from_port = 0 | ||
| to_port = 0 | ||
| protocol = -1 | ||
| cidr_blocks = ["0.0.0.0/0"] | ||
| description = "Allow all IPv4 egress traffic." | ||
| } | ||
|
|
||
| egress { | ||
| from_port = 0 | ||
| to_port = 0 | ||
| protocol = -1 | ||
| ipv6_cidr_blocks = ["::/0"] | ||
| description = "Allow all IPv6 egress traffic." | ||
| } | ||
|
|
||
| tags = { | ||
| Name = "docs-rs-builder" | ||
| } | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm confused what the autoscaling does when you've pinned it to always be at one instance?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It assures that there's one healthy instance. So if one instance stops or gets terminated a new one boots.