Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support autoscaling runners #117

Merged
merged 74 commits into from
Sep 6, 2024
Merged
Changes from 1 commit
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
dffa68c
feat: support autoscaling runners
Racer159 Aug 14, 2024
f9b4713
fix spacing
Racer159 Aug 14, 2024
7ccae3f
fix version
Racer159 Aug 14, 2024
089f625
add more images
Racer159 Aug 14, 2024
6ea7dcc
fix values:
Racer159 Aug 14, 2024
7e063d3
update fleeting
Racer159 Aug 15, 2024
72fd96d
update values
Racer159 Aug 15, 2024
e5b20da
mostly working example
Racer159 Aug 27, 2024
85b874b
add arch to docker file
Racer159 Aug 27, 2024
305c790
Merge branch 'main' into 79-explore-fleeting-plugin
Racer159 Aug 27, 2024
8773692
update docs
Racer159 Aug 27, 2024
8366f1e
add initial iac for nightly testing fleeting
Racer159 Sep 4, 2024
3dc09e1
cleanup iac
Racer159 Sep 4, 2024
8ac942c
Merge branch 'main' into 79-explore-fleeting-plugin
Racer159 Sep 4, 2024
e365990
cleanup values
Racer159 Sep 4, 2024
4c2fc4b
fixup env
Racer159 Sep 4, 2024
8ae3c1f
fixup env
Racer159 Sep 4, 2024
4738548
lint
Racer159 Sep 4, 2024
6c7d739
cleanup env
Racer159 Sep 4, 2024
87272cd
add variable defaults
Racer159 Sep 4, 2024
5ec049a
fix permissions boundary
Racer159 Sep 4, 2024
b7cbd50
rebuild the arn
Racer159 Sep 4, 2024
30d9080
add ami_id
Racer159 Sep 4, 2024
6375f75
temporary testing for glr fleeting
Racer159 Sep 5, 2024
56bfb04
add cidr variable
Racer159 Sep 5, 2024
dd78c12
fix version and cidr
Racer159 Sep 5, 2024
a06c48b
push out subnet
Racer159 Sep 5, 2024
6b4f75a
fix runner size
Racer159 Sep 5, 2024
07041c3
add the role arn
Racer159 Sep 5, 2024
550bba0
add path to deploy
Racer159 Sep 5, 2024
35b51ed
add jest
Racer159 Sep 5, 2024
528510f
add debug logs to glr nightly
Racer159 Sep 5, 2024
2db1636
test with full arn
Racer159 Sep 5, 2024
daf8af7
add extraenv for region
Racer159 Sep 5, 2024
e5c560d
fix env
Racer159 Sep 5, 2024
7db1323
add irsa to bundle
Racer159 Sep 5, 2024
07bab1a
add workspace
Racer159 Sep 5, 2024
f323436
Update nightly-testing.yaml
corang Sep 5, 2024
ccb0324
fixup args
Racer159 Sep 5, 2024
da39708
remove newlines
Racer159 Sep 5, 2024
01ad2a5
remove volume filter
Racer159 Sep 5, 2024
177362d
remove double quotes
Racer159 Sep 5, 2024
5d46f4f
remove new lines
Racer159 Sep 5, 2024
bbb7d68
try no quotes
corang Sep 5, 2024
12067ff
fix arn output subshell
corang Sep 5, 2024
e20b2bf
change timeout/retry timing
corang Sep 5, 2024
ea8f3c0
more time
corang Sep 5, 2024
2f421f0
add a jumpbox
Racer159 Sep 5, 2024
081dcf6
fix subnet and vpc_id
Racer159 Sep 5, 2024
f862d82
create unique vpcs per run
Racer159 Sep 5, 2024
6c20235
associate public ip with jumpbox
Racer159 Sep 5, 2024
fb7c1e6
change username
Racer159 Sep 5, 2024
dbc55a8
break out testing
Racer159 Sep 5, 2024
b98eb30
aws secrets manager
Racer159 Sep 5, 2024
93ddc74
add local ips
Racer159 Sep 5, 2024
bab900f
fix ingress rule
Racer159 Sep 5, 2024
77fadf8
reuseaddr
Racer159 Sep 5, 2024
cba0c47
retire test workflow
Racer159 Sep 5, 2024
e8c642f
test ssh
Racer159 Sep 6, 2024
078d812
remove secrets manager key
Racer159 Sep 6, 2024
cc20ca8
revert ssh
Racer159 Sep 6, 2024
f02ba27
tune socat
Racer159 Sep 6, 2024
546910c
test sshing after k3s startup
Racer159 Sep 6, 2024
8e22484
move back ssh - new ami
Racer159 Sep 6, 2024
a992a69
add debug logging and more ssh options
Racer159 Sep 6, 2024
f2051ec
simplify the run check
Racer159 Sep 6, 2024
58d7d50
remove testing workflow
Racer159 Sep 6, 2024
b3a67f4
cleanup docs
Racer159 Sep 6, 2024
633c1db
Update chart/templates/plugin-registry.yaml
Racer159 Sep 6, 2024
d6701f4
Update tests/journey/pipeline-run.test.ts
Racer159 Sep 6, 2024
c3dc0fa
Update tests/journey/pipeline-run.test.ts
Racer159 Sep 6, 2024
e31fb0d
Update tests/journey/pipeline-run.test.ts
Racer159 Sep 6, 2024
98e42ca
Update tests/journey/pipeline-run.test.ts
Racer159 Sep 6, 2024
65e8a09
more feedback and final test enablement
Racer159 Sep 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
reuseaddr
Racer159 committed Sep 5, 2024
commit 77fadf8aa615b9288e5837723cad483fdda9f2a6
2 changes: 1 addition & 1 deletion .github/workflows/nightly-testing.yaml
Original file line number Diff line number Diff line change
@@ -86,7 +86,7 @@ jobs:
chmod 600 ~/.ssh/id_rsa_jump
ssh -o StrictHostKeyChecking=no -i ~/.ssh/id_rsa_jump \
-R 8080:localhost:443 "ubuntu@$(tofu output -raw jumpbox_public_ip)" \
'sudo socat TCP-LISTEN:443,fork TCP:localhost:8080' &
'sudo socat TCP-LISTEN:443,reuseaddr,fork TCP:localhost:8080' &
popd

UDS_GITLAB_RUNNER_AUTOSCALING_ROLE_ARN=$(cd .github/test-infra/asg-iac && tofu output -raw asg_role_arn) uds run test-fleeting --set FLAVOR=${{ matrix.flavor }} --no-progress
12 changes: 6 additions & 6 deletions .github/workflows/testing.yaml
Original file line number Diff line number Diff line change
@@ -85,7 +85,7 @@ jobs:
# chmod 600 ~/.ssh/id_rsa_jump
# ssh -o StrictHostKeyChecking=no -i ~/.ssh/id_rsa_jump \
# -R 8080:localhost:443 "ubuntu@$(tofu output -raw jumpbox_public_ip)" \
# 'sudo socat TCP-LISTEN:443,fork TCP:localhost:8080' &
# 'sudo socat TCP-LISTEN:443,reuseaddr,fork TCP:localhost:8080' &
# popd

# UDS_GITLAB_RUNNER_AUTOSCALING_ROLE_ARN=$(cd .github/test-infra/asg-iac && tofu output -raw asg_role_arn) uds run test-fleeting --set FLAVOR=${{ matrix.flavor }} --no-progress
@@ -103,8 +103,8 @@ jobs:
# with:
# suffix: ${{ matrix.type }}-${{ matrix.flavor }}-${{ github.run_id }}-${{ github.run_attempt }}

# - name: Destroy IAC
# if: always()
# run: uds run -f tasks/iac.yaml destroy-iac --no-progress
# timeout-minutes: 10
# continue-on-error: true
- name: Destroy IAC
if: always()
run: uds run -f tasks/iac.yaml destroy-iac --no-progress
timeout-minutes: 10
continue-on-error: true
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -22,3 +22,6 @@ overlay-values-*

# Dependency files
uds-config-gitlab.yaml

# CI files
irsa