ci(test-infra): deploy StrandsTestInfraStack from GitHub Actions

[strandly-the-agent]

Description

test-infra/ provisions the IAM role the integration tests assume, but nothing deployed it — the stack only moved when someone remembered to run npx cdk deploy. That is how #3691 broke CI: the new integ test's bedrock-mantle:ListModels grant merged, the live role never got it, and the failure looked like a missing IAM permission rather than a missing deployment.

This adds a workflow that deploys the stack, the two identities it uses — defined in the stack, so their trust and permissions are reviewable rather than hand-made in the console — and, at mkmeral's request, a pull-request path: the resolved cdk diff is posted to the PR, and the PR's own version of the stack can be deployed before merge behind a manual approval.

Path	What runs	Identity	Waits for a human?
Push to main under test-infra/**, or Run workflow from main	cdk diff → cdk deploy	StrandsTestInfraDeployRole in the test-infra-deploy environment	No
Pull request	Type-check, unit tests, and a read-only cdk diff posted to the PR	StrandsTestInfraDiffRole (read-only) in auto-approve / manual-approval	Only if the author lacks write access
Pull request, deploy job	cdk diff → cdk deploy of the PR's code	StrandsTestInfraDeployRole in the test-infra-deploy-approval environment	Always

Files

File	What it does
.github/workflows/test-infra-deploy.yml	The three paths above. pull_request_target, not pull_request: a fork PR gets no secrets on pull_request, so the synthesized template would not match the live one — and pull_request_target runs main's copy of the workflow, so a PR cannot edit the steps that handle it.
test-infra/lib/constructs/github-oidc.ts	Shared OIDC principal + the environment names both roles trust.
test-infra/lib/constructs/github-deploy-role.ts	StrandsTestInfraDeployRole — may assume the four CDK bootstrap roles, which is all cdk deploy needs. Powerful in effect (the bootstrap deploy role can pass an AdministratorAccess execution role), so the trust policy is what contains it.
test-infra/lib/constructs/github-diff-role.ts	New. StrandsTestInfraDiffRole — may assume the bootstrap lookup role only (ReadOnlyAccess). This is the role a pull request's own code holds.
test-infra/lib/constructs/integ-test-role.ts	Comma-separated internal env lists are trimmed with blank entries dropped. GitHub passes an unset secret as "", which previously produced a trust principal with an empty role name.
test-infra/package.json + lockfile	aws-cdk CLI 2.1125.0 → 2.1135.1 — a prerequisite, not a drive-by. With the pinned CLI any cdk synth of this stack fails today: Cloud assembly schema version mismatch: Maximum schema version supported is 53.x.x, but found 54.0.0. Only aws-cdk moves; the lockfile diff is 8 lines.
test-infra/test/*	20 → 45 tests.
test-infra/README.md, test-infra/AGENTS.md, AGENTS.md	The secrets, the environments, the one-time bootstrap, the approval flow, and the security model.

The trust model, and why it changed

The deploy role used to pin the OIDC subject repo:strands-agents/harness-sdk:ref:refs/heads/main. That cannot survive a pull-request path: GitHub sets GITHUB_REF to the default branch for pull_request_target, so a PR job with no environment: mints exactly the push-to-main subject. The ref cannot tell reviewed code from a pull request's — so both roles now pin, as StringEquals:

• job_workflow_ref = this workflow file @refs/heads/main — no other workflow can assume either role, and a workflow_dispatch from another branch is refused;
• sub = a list of repo:strands-agents/harness-sdk:environment:<name>, and the two roles' environment sets are disjoint. GitHub mints that subject only for a job declaring that environment, so a protected environment's required reviewers are enforced by IAM as well as by GitHub — and a token minted in the diff job (which runs unreviewed PR code) does not satisfy the deploy role's trust policy. A unit test asserts the disjointness.

Before this can work (needs a maintainer with account access)

1. Create two GitHub environments: test-infra-deploy with no protection rules (a merge must deploy unattended) and test-infra-deploy-approval with required reviewers. Neither may be reused by another workflow — their names are what the deploy role trusts. auto-approve / manual-approval already exist for the rest of the repo and are reused as-is.
2. Add the repository secrets listed in test-infra/README.md. Each must match what the live role currently has — an empty list does not fail the deploy, it deploys the role with those entries removed, so the workflow refuses to start if any is unset. The diff role needs no secret of its own: its ARN is derived from the account secret plus its fixed name.
3. Run one manual internal cdk deploy (the stack defines the roles that deploy it) and set STRANDS_TEST_INFRA_DEPLOY_ROLE to the created ARN. Commands are in test-infra/README.md → Automated deployment → One-time setup.

Worth knowing before approving a PR deploy

Approving test-infra-deploy-approval authorizes that pull request's TypeScript to run with credentials that can change the account — cdk deploy executes bin/test-infra.ts and everything it imports, from the PR, including a fork's. The approval is the only gate; read the code, not just the diff. The account then holds an unmerged stack until main deploys again, which the job says in its summary.

Deliberately not in this PR

• No test-infra lane in ci.yml. test-infra/ has no enforced CI today, so the new validate job is advisory only. Wiring it into detect-changes + alls-green touches the merge gate and belongs in its own PR.
• /test-infra is missing from .github/dependabot.yml — which is how the CDK CLI pin drifted far enough to break synth. Also a follow-up.

Related Issues

No tracking issue; requested directly by mkmeral. Context for the outage: #3691.

Documentation PR

Included — test-infra/README.md, test-infra/AGENTS.md, root AGENTS.md. No site/ changes (internal infrastructure, not user-facing docs).

Type of Change

Other: CI / test infrastructure.

Testing

Not verified live. My sandbox has no AWS credentials, so no real synth, diff, or deploy ran, and neither role's permission set has ever been exercised. The first real run needs watching. Note also that pull_request_target runs main's copy of the workflow, so this PR cannot exercise its own new workflow — that only takes effect once merged.

Verbatim gate output

$ npx tsc --noEmit
(no output, exit 0)

$ npx jest
PASS test/test-infra.test.ts
PASS test/deploy-workflow.test.ts
Test Suites: 2 passed, 2 total
Tests:       45 passed, 45 total
Snapshots:   0 total
Time:        2.08 s
Ran all test suites.

$ actionlint .github/workflows/test-infra-deploy.yml   # v1.7.7
.github/workflows/test-infra-deploy.yml:127:7: unexpected key "deployment" for "environment" section. expected one of "name", "url" [syntax-check]
.github/workflows/test-infra-deploy.yml:413:7: unexpected key "deployment" for "environment" section. expected one of "name", "url" [syntax-check]

Those two actionlint errors are its schema being behind, not a bug: environment.deployment is used by this repo's python-integration-test.yml and typescript-integration-test.yml, actionlint flags those files identically, and that workflow's latest pull_request_target run succeeded. Kept for consistency with them.

Also verified, without AWS:

• cdk diff --method=template is required, not cosmetic. cdk diff --help on the pinned CLI: the default method (auto) creates a change set, which uses the deploy role; template "compares templates directly and uses the lookup role". A read-only diff has to ask for it. --no-fail parses and exits 0 with differences present (checked against a local template), so a diff does not false-fail the job.
• The shell blocks, executed with stubs. Both preflights (all set / unset / empty string); the mask loop (each list entry registered separately, blanks dropped); the redactor against hostile input — a ::stop-commands:: line, a ``` fence, secret values, and a mid-line :: that is correctly left alone — with no secret-derived value surviving into the redacted diff or the job summary; `cdk diff` failing (status propagates, fence still closed); `cdk deploy` failing through the pipe (exit 7 propagates); and the deploy preflight's ref check for `pull_request_target` / `workflow_dispatch` off main / `push`.
• 14 mutations, each killed by a test: dropping --method=template; overlapping the diff and deploy environments; reverting to the bare-ref subject; leaking STRANDS_TEST_INFRA_DEPLOY_ROLE into the diff job; switching to pull_request; dropping the redactor call in either job; dropping the cdk deploy pipe; dropping only its 2>&1; removing the :: neutralization; drifting one redactor copy's sort order; keying the deploy job's concurrency per-PR; renaming the workflow file; dropping RUNNER_ROLES from the preflight; reverting the env-list trim.
• OIDC claim shapes against GitHub's docs, including that pull_request_target reports the default branch in GITHUB_REF (the reason the trust moved to environment subjects) and that this repo predates the July 2026 immutable-subject change, so the legacy sub format is correct.

Review-loop ledger — 4 rounds, independent fresh-context reviewers

Rounds 1–2 covered the original merge-to-main-only design: 8 findings, 7 fixed (RUNNER_ROLES missing from the preflight; no account pin; trust not pinned to the workflow; docs missing required env vars; test gaps; opaque non-main dispatch failure; stale test count), then APPROVE. One finding was deferred with reasoning — "cdk diff is advisory, and the trust pin forecloses an environment: approval gate" — and this round is that finding being implemented, so it is no longer disputed.

Round 3 (independent reviewer, this round's design) — CHANGES REQUESTED, 2 findings, both fixed:

#	Sev	Finding	Outcome
1	blocker	The deploy job held the same STRANDS_TEST_INFRA_* lists but had none of the diff job's masking or redaction, so every unattended post-merge run wrote the resolved IAM diff — private repos, bucket and secret names, the account id — into a job summary on a public repo. Job summaries are files, so ::add-mask:: never covered them, and nobody approves a push to main. My README also claimed the unredacted diff only appears behind an approval, which was false.	Fixed (e7e3c791) — the deploy job masks and redacts exactly as the diff job does, cdk deploy is piped through the redactor as it streams with 2>&1, README corrected, and the mask/redact test now runs over both jobs
2	medium	CDK output is a rendering of the PR's own code and was cat unescaped, so a resource named ::stop-commands::… would be executed as a workflow command	Fixed (e7e3c791) — the redactor neutralizes line-leading ::; it also became a line filter so one implementation serves a finished file and a live stream

Round 4 (new reviewer, given round-3's findings and fixes) — APPROVE, no blockers; full write-up posted in this thread. It re-derived both fixes from source rather than re-reading them (including the pipefail exit-code claim with a live repro, and the diff-job-skips-on-push control flow from GitHub's success()/needs semantics), and found one gap round 3 missed:

#	Sev	Finding	Outcome
3	medium	The redactor is inlined twice and the copies were only pattern-tested, not compared — the reviewer mutated only the deploy copy's sort order, losing the longest-match-first property, and all tests still passed	Fixed (0153004d) — the copies are byte-identical and a test asserts it
4	nit	A value split across a line break by CDK's table renderer would defeat a line filter. Traced as unreachable here (stream.columns is undefined for non-TTY output) but an undocumented third-party dependency	Fixed (0153004d) — recorded as a comment in the redactor, flagged to recheck on a CDK bump

One self-found fix, outside the review rounds: the deploy job now serializes on a constant concurrency group (d54325c4). The workflow-level group is keyed per pull request, so a PR deploy and a main deploy would have landed in different groups and could have run at once against the one stack — the thing the workflow's own comment says CloudFormation rejects.

Open / residual: nothing live has been exercised, and the environments plus secrets above must exist first. A fifth adversarial pass (attacking the OIDC trust model specifically) timed out twice without reporting, so that angle has had two reviewers' attention but not a dedicated one.

Checklist

• I have read the CONTRIBUTING document
• I have reviewed and understand every line of code in this PR
• My change is focused and reasonably small; unrelated work is called out as follow-ups rather than bundled
• I have added tests that prove the change works (45 tests, mutation-checked)
• I have updated the documentation accordingly
• My changes generate no new warnings
• hatch run prepare — N/A, no Python touched

---

🤖 Opened by strandly-the-agent at mkmeral's request. Four independent review passes ran across two design iterations, but a human should still verify the IAM and OIDC details and watch the first real run — the parts that cannot be checked from a sandbox.

[strandly-the-agent]

@mkmeral this is the test-infra deploy workflow you asked for. I could not add you as a requested reviewer — my token only has read access here, so POST /repos/strands-agents/harness-sdk/pulls/3713/requested_reviewers returns 404 — hence flagging it in a comment instead.

Two things need someone with account access before the workflow can do anything (both in the PR body and test-infra/README.md): the seven repository secrets, and one manual internal cdk deploy to create StrandsTestInfraDeployRole — the stack defines the role that deploys it.

Worth your eyes specifically: the OIDC trust conditions on that role, and whether STRANDS_TEST_INFRA_RUNNER_ROLES should be required the way I made it (an empty list quietly removes those principals from the live integ role's trust policy, which is the failure class this PR exists to prevent). Nothing was exercised against AWS — no credentials in my sandbox — so the first real run needs watching.

[mkmeral]

@strandly-the-agent can we also run this in PRs with manual approval? we have examples in other CIs. I'd like to be able to see the diff in roles and stuff before approving though ideally (I can do through pr, but for more programmatic changes I'd like to see what it resolved to too)

[strandly-the-agent]

Note

Posted by a review subagent on its own initiative — my fault, I didn't scope its tools to exclude GitHub writes. Kept rather than deleted because the content is accurate and independent. Both items it raised are now closed: the redactor copies are byte-identical with a test asserting it, and the CDK line-wrap caveat is recorded as a comment in the redactor — both in 0153004d. Ledger in the PR body.

Round 2 review — APPROVE (posted as a comment, not a formal review: GitHub won't let an author self-approve). Verified all round-1 fixes independently at e7e3c791 (matches HEAD); no regressions, one test-coverage gap worth closing before merge, no blockers.

✅ Checked: tsc --noEmit clean, jest 44/44, actionlint → only the known deployment false-positive (same as the live python-integration-test.yml). Re-derived (not just re-read) both fixes from source, plus the control-flow question about the diff job auto-skipping on push.

Fix 1 (masking/redaction in deploy) — confirmed complete

Step order in deploy is right: preflight (no secrets printed) → mask → write redactor → (non-sensitive) record step → checkout → creds → cdk diff (raw file → redact.py → only the redacted file is cat'd/summarized) → cdk deploy 2>&1 | redact.py. No cat of the raw diff file anywhere, and deploy has no artifact-upload step so there's no third leak surface to worry about there.

Re-derived the exit-code claim rather than trusting it: set -uo pipefail; cdk deploy ... 2>&1 | python3 redact.py with no trailing exit $? — confirmed with a (exit 7) | python3 redact.py; echo $? repro that pipefail correctly reports 7, because python3 itself exits 0 and pipefail picks the rightmost non-zero. No -e needed since the pipeline is the script's last statement.

Fix 2 (:: neutralization + line filter) — confirmed correct, with one caveat

Repro'd the redactor standalone: leading ::stop-commands::... → escaped, foo::bar mid-line → left alone, as intended.

Caveat (not a blocker): a secret-derived value split across a line break by CDK's own table renderer would survive the line filter — confirmed by feeding a wrapped value through the redactor and watching both halves print raw. Traced why this isn't actually reachable today: aws-cdk's formatTable only applies a column width (and thus wraps cell text) when stream.columns is defined, and that getter (toolkit-lib/api/streams.ts) falls back through process.stdout.columns (undefined — this workflow always redirects/pipes, never a TTY) to process.env.COLUMNS (unset in a GH Actions run: step). So no wrap happens in this workflow's actual invocations. It's an implicit dependency on an undocumented third-party behavior, though — worth a one-line comment in redact.py noting why line-based redaction is safe here, so the next CDK major bump doesn't quietly reopen this.

🟡 Should-fix: the two redactor copies aren't equivalence-tested, only pattern-tested

Diff'd the diff/deploy copies mechanically — byte-identical except comments, as intended. But test.each(['diff','deploy'])('...masks and redacts...') only checks that each copy independently contains certain substrings (::add-mask::, each env var name near redact.py, the two re.sub calls) — it doesn't check the two copies match.

Demonstrated the gap: changed only the deploy copy's sorted(..., key=len, reverse=True) to sorted(..., key=len) (silently breaking the "longest-first, so a short entry can't clobber a longer one containing it" property) — full deploy-workflow.test.ts suite still passes, 16/16. Reverted before finishing.

Suggested fix: extract each job's redact.py heredoc body via jobBlock() (already have the helper) and assert the two strings are ===. That turns any future one-sided edit into an immediate, obvious test failure instead of a silent drift.

Control-flow re-derivation: does diff correctly no-op on push?

Worth stating since it's non-obvious: diff's own if: github.repository == '...' has no always()/success()/etc., so GitHub ANDs it with the implicit success() — and success() is documented to return false when a needs job (authorization-check, skipped on push) was skipped. So diff is correctly skipped on push, matching the inline comment and the deploy job's needs.diff.result == 'skipped' branch. Also matches this repo's own established convention in python-integration-test.yml, which explicitly reaches for always() && needs.X.result == 'success' precisely to override that same default. No bug here — just re-derived it since it's the crux of "can PR code run unapproved."

Nothing else found. The false-comment garbling I initially flagged in github-oidc.ts was an artifact of my own file-reading tool, not the repo — confirmed via git show HEAD:..., disregard.

[strandly-the-agent]

@mkmeral both are in, at 0153004d — and building the first one turned up a leak in the design you were about to approve.

• Deploy from a PR behind manual approval. The deploy job now runs in a test-infra-deploy-approval environment (required reviewers) and deploys the PR's own commit. Post-merge deploys stay unattended in test-infra-deploy. Same pattern as python-integration-test.yml / docs-deploy-preview.yml, reusing the repo's authorization-check gate.
• See what it resolved to, before approving. A read-only cdk diff runs first and is posted to the PR, refreshed on each push. It uses --method=template deliberately: the default diff method creates a change set, which needs the deploy role, and this job must not have it.

Secret-derived names are redacted to *** — this repo is public, so the log, job summary, artifact and comment are all world-readable. Which is how the leak surfaced: the existing deploy job wrote the resolved diff — private repos, bucket and secret names, the account id — straight into a public job summary on every merge, with no human in the loop. ::add-mask:: never covered it because summaries are files. Fixed for both jobs now.

Needs you, and nothing works until then: create test-infra-deploy with no protection rules and test-infra-deploy-approval with required reviewers — those two names are literally what the deploy role's trust policy pins — plus the repository secrets, plus one manual internal cdk deploy, which now also creates StrandsTestInfraDiffRole. That role needs no secret of its own; its ARN derives from the account secret.

Before you approve a PR deploy: it runs that pull request's TypeScript with credentials that can change the account — cdk deploy executes bin/test-infra.ts and everything it imports, from the PR, fork included. The approval is the only gate. The diff tells you what it says it will change; only the code tells you what it will do.

The trust policy changed shape — worth your eyes, since roles are what you wanted to review

It used to pin the OIDC subject repo:strands-agents/harness-sdk:ref:refs/heads/main. That does not survive a PR path: GitHub sets GITHUB_REF to the default branch for pull_request_target, so a PR job with no environment: mints exactly the push-to-main subject. A ref pin cannot tell reviewed code from a pull request's.

So both roles now pin job_workflow_ref (this file, on main) plus a list of …:environment:<name> subjects, and the two roles' environment sets are disjoint — that disjointness is the only thing stopping a token minted in the diff job, which runs unreviewed PR code, from satisfying the deploy role's trust. A unit test asserts it, and the mutation that overlaps them fails three tests.

Two consequences to know: the diff role can assume only the bootstrap lookup role (ReadOnlyAccess), never deploy; and approving the diff for an outside contributor hands that PR's code the STRANDS_TEST_INFRA_* lists, which is why an author without write access waits for a reviewer there too.

Two caveats on the first run

• This PR cannot exercise its own workflow. pull_request_target runs main's copy, so the new jobs only start behaving once this is merged. The first post-merge run is the one to watch.
• Nothing has run against real AWS — no credentials in my sandbox — so neither role's permission set has ever been exercised. In particular, if cdk diff --method=template turns out to need something the bootstrap lookup role lacks, that is where it will show up.

Full detail, the gate output and the four-round review ledger are in the PR body; round 4's independent write-up is above.