Skip to content

Commit 24fe89f

Browse files
KayakinCoderKayakinCoder
authored
Post setup modifications (#1)
* basic file setup * update lambda script * follow stac-server install steps correctly * more fixes to update lambda script * set up workflows * fix path to zips * Workflow setup * Built stac-server version
1 parent 4ab82ff commit 24fe89f

File tree

20 files changed

+671
-25
lines changed

20 files changed

+671
-25
lines changed

‎.github/pull_request_template.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
## Related issue(s)
2+
3+
-
4+
5+
## Proposed Changes
6+
7+
1.
8+
9+
## Testing
10+
11+
This change was validated by the following observations:
12+
13+
1.
14+
15+
## Checklist
16+
17+
- [ ] I have deployed and validated this change
18+
- [ ] Changelog
19+
- [ ] I have added my changes to CHANGELOG.md
20+
- [ ] No changelog entry is necessary
21+
- [ ] stac-server Version
22+
- [ ] I have noted a version bump in the stac-server lambdas in CHANGELOG.md, and updated the `STAC_SERVER_TAG` throughout `/.github/workflows`
23+
- [ ] No stac-server version bump was performed

‎.github/workflows-WIP/release-tests.yml‎

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
name: Main Branch and Release Testing
2+
3+
on:
4+
push:
5+
branches: ["main"]
6+
tags: ["v*.*.*"]
7+
# Allows manual testing
8+
workflow_dispatch:
9+
10+
# TODO:
11+
# - Github env vars/secrets made available to this repo
12+
# - CI.tfvars
13+
# - Prepararing Environment statefile (can be simplified to use s3 state locking)
14+
# - After full testing, uncomment slack channel status...? Or we don't want slack channel posts?
15+
16+
jobs:
17+
pre-commit:
18+
uses: ./.github/workflows/reusable-precommit.yml
19+
20+
release-tests:
21+
permissions:
22+
id-token: write
23+
contents: read
24+
runs-on: ubuntu-latest
25+
env:
26+
CI: true
27+
STAC_SERVER_TAG: v3.10.0
28+
steps:
29+
- uses: actions/checkout@v5
30+
31+
- uses: actions/setup-node@v5
32+
with:
33+
node-version: "18"
34+
35+
# Here we read the terraform version from the .terraform-version file, and then install that version
36+
- name: Get Terraform version
37+
id: tf_version
38+
run: |
39+
echo "value=$(cat .terraform-version)" >> $GITHUB_OUTPUT
40+
- uses: hashicorp/setup-terraform@v3
41+
with:
42+
terraform_version: ${{ steps.tf_version.outputs.value }}
43+
44+
- name: Setting Pre-Requisites
45+
id: prereqs
46+
run: |
47+
echo "REPOSITORY_NAME=`echo \"${{ github.ref_name }}\" | tr -d '.' | cut -c1-8`" >> $GITHUB_ENV
48+
49+
- name: Prepararing Environment
50+
id: prep_env
51+
run: |
52+
echo "environment = \"git\"" >> ci.tfvars
53+
echo "project_name = \"${REPOSITORY_NAME}\"" >> ci.tfvars
54+
cat ci.tfvars
55+
echo "Creating terraform backend file ..."
56+
echo 'terraform {' >> test.s3.backend.tf
57+
echo ' backend "s3" {' >> test.s3.backend.tf
58+
echo ' encrypt = true' >> test.s3.backend.tf
59+
echo " bucket = \"${{ secrets.TF_STATE_BUCKET }}\"" >> test.s3.backend.tf
60+
echo " dynamodb_table = \"${{ secrets.TF_STATE_LOCK_TABLE }}\"" >> test.s3.backend.tf
61+
echo " key = \"${REPOSITORY_NAME}-github-test.tfstate\"" >> test.s3.backend.tf
62+
echo " region = \"${{ secrets.AWS_REGION }}\"" >> test.s3.backend.tf
63+
echo ' }' >> test.s3.backend.tf
64+
echo '}' >> test.s3.backend.tf
65+
cat test.s3.backend.tf
66+
67+
- name: Update stac-server lambdas
68+
id: update_stac_lambdas
69+
run: ./scripts/update-lambdas.bash
70+
71+
- name: Configure Terraform Init Credentials
72+
id: init_creds
73+
uses: aws-actions/configure-aws-credentials@v4
74+
with:
75+
aws-region: ${{ secrets.AWS_REGION }}
76+
role-to-assume: ${{ secrets.AWS_ROLE }}
77+
role-session-name: GitHubReleaseInit
78+
79+
- name: Terraform Init
80+
id: tf_init
81+
run: terraform init
82+
83+
- name: Terraform Validate
84+
id: tf_validate
85+
run: terraform validate
86+
87+
- name: Configure Terraform Plan Credentials
88+
id: plan_creds
89+
uses: aws-actions/configure-aws-credentials@v4
90+
with:
91+
aws-region: ${{ secrets.AWS_REGION }}
92+
role-to-assume: ${{ secrets.AWS_ROLE }}
93+
role-session-name: GitHubReleasePlan
94+
95+
- name: Terraform Plan
96+
id: tf_plan
97+
run: terraform plan -var-file="ci.tfvars" -out test.tfplan -lock=false
98+
99+
- name: Configure Terraform Apply Credentials
100+
id: apply_creds
101+
uses: aws-actions/configure-aws-credentials@v4
102+
with:
103+
aws-region: ${{ secrets.AWS_REGION }}
104+
role-to-assume: ${{ secrets.AWS_ROLE }}
105+
role-session-name: GitHubReleaseApply
106+
107+
- name: Terraform Apply
108+
id: tf_apply
109+
continue-on-error: true
110+
run: terraform apply -lock=false -input=false test.tfplan
111+
112+
# - name: Post status to Slack channel
113+
# id: tf_apply_successs
114+
# if: steps.tf_apply.outcome == 'success'
115+
# continue-on-error: true
116+
# uses: slackapi/[email protected]
117+
# with:
118+
# channel-id: ${{ secrets.SLACK_CHANNEL_ID }}
119+
# slack-message: ":badger_dance: terraform-aws-stac-server - ${{ github.ref_name }} terraform apply job has succeeded!\n${{ github.event.pull_request.html_url || github.event.head_commit.url }}"
120+
# env:
121+
# SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
122+
123+
# - name: Post status to Slack channel
124+
# id: tf_apply_failure
125+
# if: steps.tf_apply.outcome != 'success'
126+
# continue-on-error: true
127+
# uses: slackapi/[email protected]
128+
# with:
129+
# channel-id: ${{ secrets.SLACK_CHANNEL_ID }}
130+
# slack-message: ":sadpanda: terraform-aws-stac-server -${{ github.ref_name }} terraform apply has failed!\n:alert: make sure cleanup job deletes all AWS resources!\n${{ github.event.pull_request.html_url || github.event.head_commit.url }}"
131+
# env:
132+
# SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
133+
134+
- name: Configure Terraform Cleanup Check Credentials
135+
id: cleanup_check_creds
136+
if: always()
137+
uses: aws-actions/configure-aws-credentials@v4
138+
with:
139+
aws-region: ${{ secrets.AWS_REGION }}
140+
role-to-assume: ${{ secrets.AWS_ROLE }}
141+
role-session-name: GitHubReleaseCleanupCheck
142+
143+
- name: Terraform Destroy Pre-Check
144+
id: tf_destroy_plan
145+
if: always()
146+
run: terraform plan -destroy -var-file="ci.tfvars" -out test-cleanup.tfplan -lock=false
147+
148+
- name: Configure Terraform Cleanup Credentials
149+
id: cleanup_creds
150+
if: always()
151+
uses: aws-actions/configure-aws-credentials@v4
152+
with:
153+
aws-region: ${{ secrets.AWS_REGION }}
154+
role-to-assume: ${{ secrets.AWS_ROLE }}
155+
role-session-name: GitHubReleaseCleanup
156+
157+
- name: Terraform Destroy
158+
id: tf_destroy_apply
159+
if: always()
160+
continue-on-error: true
161+
run: terraform apply -destroy -lock=false -input=false test-cleanup.tfplan
162+
163+
# - name: Post status to Slack channel
164+
# id: tf_destroy_apply_successs
165+
# if: steps.tf_destroy_apply.outcome == 'success'
166+
# continue-on-error: true
167+
# uses: slackapi/[email protected]
168+
# with:
169+
# channel-id: ${{ secrets.SLACK_CHANNEL_ID }}
170+
# slack-message: ":badger_dance: terraform-aws-stac-server - ${{ github.ref_name }} cleanup job has succeeded!\n${{ github.event.pull_request.html_url || github.event.head_commit.url }}"
171+
# env:
172+
# SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
173+
174+
# - name: Post status to Slack channel
175+
# id: tf_destroy_apply_failure
176+
# if: steps.tf_destroy_apply.outcome != 'success'
177+
# continue-on-error: true
178+
# uses: slackapi/[email protected]
179+
# with:
180+
# channel-id: ${{ secrets.SLACK_CHANNEL_ID }}
181+
# slack-message: ":sadpanda: terraform-aws-stac-server -${{ github.ref_name }} cleanup job has failed!\n:alert: make sure AWS resources are deleted!\n${{ github.event.pull_request.html_url || github.event.head_commit.url }}"
182+
# env:
183+
# SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}

‎.github/workflows/ci.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: Continuous integration
2+
3+
on:
4+
push:
5+
branches: ["main"]
6+
pull_request:
7+
branches: ["main"]
8+
9+
jobs:
10+
pre-commit:
11+
uses: ./.github/workflows/reusable-precommit.yml
12+
13+
update-lambdas:
14+
runs-on: ubuntu-latest
15+
env:
16+
CI: true
17+
STAC_SERVER_TAG: v3.10.0
18+
steps:
19+
- uses: actions/checkout@v5
20+
21+
# Here we read the terraform version from the .terraform-version file, and then install that version
22+
- name: Get Terraform version
23+
id: tf_version
24+
run: |
25+
echo "value=$(cat .terraform-version)" >> $GITHUB_OUTPUT
26+
- uses: hashicorp/setup-terraform@v3
27+
with:
28+
terraform_version: ${{ steps.tf_version.outputs.value }}
29+
30+
- name: Update stac-server lambdas
31+
id: update_stac_lambdas
32+
run: ./scripts/update-lambdas.bash
33+
34+
- name: Terraform Init
35+
id: tf_init
36+
run: terraform init
37+
38+
- name: Terraform Validate
39+
id: tf_validate
40+
run: terraform validate -no-color

‎.github/workflows/reusable-precommit.yml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
name: Reusable Pre-Commit Job
2+
3+
# Reusable pre-commit workflow. Note this does not run automatically, it must be invoked by another workflow
4+
on:
5+
workflow_call:
6+
7+
jobs:
8+
pre-commit:
9+
runs-on: ubuntu-latest
10+
steps:
11+
- uses: actions/checkout@v5
12+
# Here we read the terraform version from the .terraform-version file, and then install that version
13+
- name: Get Terraform version
14+
id: tf_version
15+
run: |
16+
echo "value=$(cat .terraform-version)" >> $GITHUB_OUTPUT
17+
- uses: hashicorp/setup-terraform@v3
18+
with:
19+
terraform_version: ${{ steps.tf_version.outputs.value }}
20+
- uses: terraform-linters/setup-tflint@v6
21+
- uses: actions/setup-python@v5
22+
with:
23+
python-version: "3.11"
24+
- name: Install pre-commit
25+
run: pip install pre-commit
26+
- name: Run pre-commit
27+
run: pre-commit run --all-files

‎.github/workflows/snyk-scan.yml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# This workflow sets up Snyk scans
2+
3+
name: Snyk Scan
4+
5+
on:
6+
push:
7+
branches: ["main" ]
8+
pull_request:
9+
branches: ["main"]
10+
schedule: # Run snyk scan daily at midnight
11+
- cron: '0 0 * * *'
12+
13+
permissions:
14+
contents: read
15+
16+
jobs:
17+
snyk:
18+
permissions:
19+
contents: read # for actions/checkout to fetch code
20+
security-events: write # for github/codeql-action/upload-sarif to upload SARIF results
21+
runs-on: ubuntu-latest
22+
steps:
23+
- uses: actions/checkout@v5
24+
25+
- name: Snyk IaC report vulnerabilities
26+
uses: snyk/actions/[email protected]
27+
continue-on-error: true # To make sure that SARIF upload gets called
28+
env:
29+
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
30+
with:
31+
args: --sarif-file-output=snyk.sarif
32+
33+
# Push the Snyk Code results into GitHub Code Scanning tab
34+
- name: Upload result to GitHub Code Scanning
35+
uses: github/codeql-action/upload-sarif@v3
36+
with:
37+
sarif_file: snyk.sarif
38+
39+
- name: Snyk IaC gatekeeper
40+
uses: snyk/actions/[email protected]
41+
env:
42+
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
43+
with:
44+
args:
45+
--sarif-file-output=snyk.sarif
46+
--policy-path=.snyk
47+
--severity-threshold=high # Forces fail on high-severity vulnerabilities

‎.gitignore‎

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,10 +18,6 @@
1818
.terraform/
1919
__pycache__
2020
bin/terraform
21-
modules/jupyterhub-dask-eks/cluster.yaml
22-
modules/jupyterhub-dask-eks/daskhub.yaml
23-
modules/jupyterhub-dask-eks/spec.yaml
24-
modules/jupyterhub-dask-eks/storageclass.yaml
2521
node_modules
2622
package-lock.json
2723
plan.json
@@ -31,5 +27,4 @@ stac-server-*
3127
stac-server.tgz
3228
terraform-visual-report
3329
terraform.tgz
34-
modules/stac-server/historical-ingest/lambda/package/*
35-
!modules/cirrus/cirrus-lambda-dist.zip
30+
historical-ingest/lambda/package/*

‎.markdownlint.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
MD024: false
2+
MD013:
3+
line_length: 100

‎.pre-commit-config.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
repos:
2+
- repo: https://github.com/antonbabenko/pre-commit-terraform
3+
rev: v1.85.0
4+
hooks:
5+
- id: terraform_fmt
6+
- id: terraform_tflint
7+
- repo: https://github.com/astral-sh/ruff-pre-commit
8+
rev: v0.1.8
9+
hooks:
10+
- id: ruff
11+
- id: ruff-format

‎.snyk‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Snyk (https://snyk.io) policy file, patches or ignores known vulnerabilities.
2+
version: v1.25.0
3+
# ignores vulnerabilities until expiry date; change duration by modifying expiry date
4+
ignore:
5+
SNYK-CC-TF-99:
6+
- 'api.tf > resource > aws_api_gateway_method[stac_server_api_gateway_proxy_resource_method] > authorization':
7+
reason: Open API - no auth required
8+
created: 2023-09-14T14:35:23.783Z
9+
- 'api.tf > resource > aws_api_gateway_method[stac_server_api_gateway_root_method] > authorization':
10+
reason: Open API - no auth required
11+
created: 2023-09-14T14:35:23.783Z
12+
SNYK-CC-00250:
13+
- 'api.tf > *':
14+
reason: Open API - no auth required
15+
created: 2023-12-14T14:35:23.783Z
16+
- 'api.tf > *':
17+
reason: Open API - no auth required
18+
created: 2023-12-14T14:35:23.783Z
19+
- 'api.tf > *':
20+
reason: Open API - no auth required
21+
created: 2024-03-21T14:35:23.783Z

‎.terraform-version‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
1.13.4

0 commit comments

Comments
 (0)