Repository navigation
feat(lakehouse-agent): flag-routed Okta + OBO alongside Cognito - #1933
Draft
nweispfenning wants to merge 105 commits into
Draft
nweispfenning wants to merge 105 commits into
nweispfenning wants to merge 105 commits into
Conversation
added 30 commits
August 7, 2026 10:39
flag helper (validate/set/get, .env->SSM /app/lakehouse-agent/idp-provider, fail-fast); .env.example default cognito (preserves upstream); README IdP section + flag-map skeleton + [COGNITO]/[OKTA] guard convention; offline unit test 15/15. No behavior change to existing notebooks/deploy.
…naming ported from validated fork; two-app OBO topology; project-name + SSM-prefix normalized to lakehouse-agent (OBO feature terms preserved); ruff-formatted to repo style (46 F541 cleared); py_compile clean, zero -obo-project survivors.
net-new; search_claim_notes + owner_user_sub per-user RLS; least-priv AOSS role; Okta JWT authorizer + Authorization-header allowlist; normalized to lakehouse-agent; ruff-formatted (27 F541); hygiene-clean; generated .bedrock_agentcore.yaml excluded.
net-new; AOSS collection provisioning + CustomOauth2 OBO provider (TOKEN_EXCHANGE, RFC 8693) + OBO gateway/target (no interceptor by design) + optional agent-IAM (direct self-mint path only; gateway path needs none - Finding 15) + cleanup; normalized to lakehouse-agent; ruff (57 F541); hygiene-clean.
…Phase 1) DR-8 flag-branch convention; authorizer/config/claim-extraction branched Cognito|Okta (Cognito byte-preserved); RESPONSE interceptor attached (A1, both paths); in-place idempotent role/gateway (Flag-3); IDP_PROVIDER plumbed to interceptor Lambdas via deploy env (Flag-2); GW1 keeps upstream gateway-* keys (Flag-1); fixed latent Okta config-print AttributeError; branched token_exchange/tool_validation group-claim. ruff/py_compile/hygiene clean.
thin notes REQUEST interceptor injects caller sub on body-context; server reads context.user_id (Cognito) vs bearer (Okta), fail-closed both; match-by-construction seed (cognito-user-*-sub); 5b/04 GW2 branch (Cognito interceptor gateway + M2M provider + OpenSearch target / Okta OBO verbatim); OpenSearch-runtime authorizer branch; dedicated least-priv notes-interceptor role. ruff/py_compile/hygiene clean.
two-client wiring (claims/->GW1 gateway-arn, notes/->GW2 obo-gateway-url), same bearer to both, union catalog, finally-teardown; IdP-agnostic (GW2 keys unified by 5b/04); agent role net-zero OBO (Finding 15); fail-soft claims-only if GW2 absent (now logged).
…n paths Paired with 4b/5b; all cross-refs fixed (notebooks, both READMEs, ported dangling refs now valid); advanced-interceptor add-on hard-coded paths patched (pre-deploy.sh + policy-stack.ts + README) — paths only, no logic/Cedar; repo-wide grep for old names = zero; add-on pre-deploy.sh bash -n + path-existence verified.
obo-gateway-* SSM keys → notes-gateway-*; resource → lakehouse-notes-gateway; derived role → agentcore-lakehouse-notes-gateway-role; writer(04)/cleanup(06)/reader(agent); Okta-only obo tokens preserved (lakehouse-obo-target, lakehouse-obo-okta-provider, okta-obo-client-*, obo-credential-provider-arn); 05_update_agent_iam uncoupled (patches agent role, not gateway); grep obo-gateway=zero except the kept 5b-obo-gateway-setup dir; py_compile OK; ruff format --check clean.
… chunk 3) detect_gateway_idp (discoveryUrl-host primary + credential-shape fallback, fail-closed) + assert_gateway_idp_matches; guard inserted before the converge/reuse in create_gateway.py (GW1) and 5b/04 (GW2, both branches) — where the old code would silently keep the wrong-IdP authorizer; fail-fast 'run teardown first' message; 10/10 offline unit test. DR-11. (Pre-existing ruff nit at idp_config.py:65 is NOT ours — verified append-only diff; consistent with repo baseline.)
…chunk 4) Rename 01-deploy-cognito→01-deploy-idp; Step-0 set+persist IDP_PROVIDER (set_idp_provider, default cognito, R1.4); ## [COGNITO]/## [OKTA] headers + if-guards on setup/audit/verify (R11.2/.4); Cognito cells byte-preserved (DR-8); Okta verify reuses verify_okta_setup.py; dropped unused cognito_client from agnostic init; stale-ref + rename cross-refs fixed, grep-zero; 18 cells valid nbformat, code compiles. Satisfies R1/R2/R9.1/R9.2/R11.
…unk 5a) git-rename 05-deploy-gateway->05a-deploy-claims-gateway; retitle GW1/claims (both paths); removed stale DynamoDB-tenant-map prose (cells 1+13) -> REQUEST+RESPONSE interceptor design; agnostic-ified Step-3 ARNs cell (dropped hard-coded Cognito pool-ARN, reads interceptor+runtime ARNs only); narrative ## [COGNITO]/## [OKTA] notes only (no functional guards -- branching stays in create_gateway.py); cross-refs fixed (04/06/README), grep-zero; 14 cells nbformat-valid, code compiles.
…rch (Phase 6 chunk 5b) net-new notebook wiring collection->runtime->[COGNITO]seed->load-sample->verify->auth-layer([OKTA]OBO-provider / [COGNITO]interceptor)->GW2 (04, branches internally, DR-11 guard fires there); flag read once via get_idp_provider; 05_update_agent_iam omitted per Finding 15 (note-only cell); seed [COGNITO]-only; seed ordered BEFORE load-sample -- the loader tags sample notes with owner subs and aborts if absent (corrects initial design ordering); 22 cells nbformat-valid, code compiles, cwd paths verified, hygiene clean.
…FINDING-1) Okta notes path had no producer for okta-user-*-sub before 05b load_sample (the fork's inline notebook-07 fixture was left behind in consolidation) -> Okta 05b Step 4 sys.exit(1). Fix: setup_okta.create_test_users now writes /app/lakehouse-agent/okta-user-<label>-sub = u["login"] (EMAIL = access-token sub claim) for every test user, with a fail-fast @-guard; created.id deliberately NOT used (would vacuous-pass, fork Finding 16). Reconciled stale "notebook 07" refs (loader docstring/error/module-doc, design section 6) -> setup_okta/notebook 01. Cognito seed unchanged (asymmetry justified).
…k-5 gate) FINDING-2 (MAJOR) forward-trail 05a->05b->06 + 06 lists 05b as prereq (GW2 no longer orphaned); FINDING-3 (MINOR, Option a) IdP toggle stays the in-notebook Step-0 cell (explicit value=), removed the misleading .env IDP_PROVIDER advertisement (.env.example + idp_config docstrings) that created a silent-override trap -- DR-12 UPHELD+clarified; FINDING-4 (NIT) loader comment opensearch/->notes/; plus ruff format idp_config.py (our net-new file, now formatter-clean). R1.4 preserved.
…h-PKCE) + IdP badge (Phase 6 chunk 6a-i) Flag read once (get_idp_provider), fail-soft banner→nb01 if unset; load_config branches cognito_*/okta_* + shared runtime_arn/gateway-url(GW1)/region → single st.session_state.idp_config (replaces cognito_config/auth_config); login branches — [COGNITO] upstream password-grant BYTE-PRESERVED, [OKTA] ported Auth-Code+PKCE (OAuth2Component, REDIRECT_URI+coupling note); shared contract access_token/id_token/user_email; IdP badge; enrichments ported SHARED (invoke_agent (10,300), fetch_interceptor_tools soft-fail via GW1 gateway-url, nest_asyncio, PATH_LABELS, last_tools_used, env-gated save-token); requirements += streamlit-oauth/PyJWT/mcp/nest_asyncio + boto3 bump; dropped unused Cognito client-secret from config. Ruff: format-clean; check=21 baseline-class (no new class) — left per convention.
…08-streamlit + README reconcile (Phase 6 chunk 6a-ii) Atomic renumber (git mv cleanup→09 first to avoid an 08 collision, then streamlit→08); 08-streamlit-ui [COGNITO]/[OKTA] guards (Cognito M2M pre-flight token + invoke test guarded, Okta→launch app, launch cell agnostic, branch-aware summary), flag read once (c2 get_idp_provider); README reconciled to the final arc 07-iso-test / 08-streamlit / 09-cleanup (guide-range, notebook-table, cleanup §, structure-tree, 06 next-links) + added the 07-iso-test row (file lands in 6c); resolves audit FINDING-5 (cleanup is now 09). grep old names = 0, no double-08, nbformat valid + code compiles.
…der (Phase 6 chunk 6b-i) NEW interceptor-notes/cleanup.sh — idempotent teardown of the notes-interceptor (Lambda lakehouse-notes-interceptor, role lakehouse-notes-interceptor-role + policies, log group, SSM notes-interceptor-lambda-arn, local dist/+zip), mirrors deploy.sh, --keep-ssm flag, Cognito-only (all skip no-ops on Okta). 06_cleanup_obo_gateway.py now explicitly deletes the Cognito M2M provider lakehouse-notes-cognito-oauth-provider (delete_oauth_provider generalized to a name param, called for both the Okta OBO + Cognito M2M providers, safe-if-absent both ways) — removes reliance on cleanup_gateway.py's incidental substring match. Fills the two teardown gaps found by the 6b recon. Verified bash -n / py_compile / ruff (18 baseline-class, no new).
Restructured to reverse-deploy order + dual-IdP; flag read once (c1). NEW Step 2 GW2 teardown (06_cleanup_obo_gateway both-paths + [COGNITO] notes-interceptor cleanup.sh); NEW 4b OpenSearch-runtime cleanup (Step 4); Step 7 IdP branch ([COGNITO] cleanup_cognito / [OKTA] cleanup_okta — guarded; cleanup_okta never unconditional, safe against its sys.exit-without-OKTA-env); DELETE_S3_BUCKET flipped True->False; title/summary refreshed to 01-09 arc + full coverage. B17 PRESERVED — no put_data_lake_settings/LF-admin removal (deregister only; verified 0 in code cells). Closes audit Note-1. Verified nbformat(25)/compile(14)/all-cwds-exist/AST-guards.
… 6 chunk 6c) Ported the fork's isolation test (cross-user data isolation on both gateways). FINDING-1 closed (fixture READS <idp>-user-*-sub; put_parameter=0). Cell-19 token-validation branched (Okta aud+email / Cognito GUID no-aud). Cell-25 log-scan advisory fail-soft on Cognito. SSM prefix normalized + keys renamed (notes-gateway-url / gateway-url). Cleared all outputs (DR-6); dropped empty cell (42 cells). requirements.txt += opensearch-py/aws-requests-auth/PyJWT/nest_asyncio. Cells 28-30 (tool-gating) [OKTA]-guarded pending TRM-GAP; core scenarios 1-6 IdP-agnostic + independent. Verified nbformat/compile/grep.
…-gating (MAJOR) (a) setup_dynamodb_tenant_role_maps.py branches claim_name on IDP_PROVIDER (cognito:groups/groups; read once via get_idp_provider) — the Okta interceptor lookup now hits (was Cognito-only → Okta claims tool-gating returned empty/403); claim_value (IdP-invariant) + allowed_tools (admin=query_login_audit/text_to_sql, correct for FINAL) unchanged. (b) 07 cells 29-30: fork-only get_claims_analytics -> query_login_audit + promoted cells 28-30 to dual-IdP (removed [OKTA] guard; legit cell-6/cell-19 branches remain). get_claims_analytics NOT re-added (DR-2). Live group-claim delivery = Phase-7 validation. Closes MAJOR finding TRM-GAP. Verified py_compile/ruff(14 baseline, 0 new)/nbformat/grep.
Flag-map 'How to set it' -> DR-12 Step-0-cell mechanism (dropped the .env contradiction); 06 row claims/+opensearch/ -> claims/+notes/; dropped stale build-progress meta; 09-cleanup row -> dual-IdP branched; notebook-table + structure-tree add 05b-deploy-notes-gateway + missing dirs (1-okta-setup/4b/5b/interceptor-notes); overview + streamlit prose -> dual-IdP; test_cognito_login dir-prefix fix. 05a DynamoDB-prose CORRECTED (claims interceptor DOES use lakehouse_tenant_role_map; the earlier 'no DynamoDB' note was wrong; DR-9 thin-notes counterpart preserved). Closes tracked doc items (05b-listing, opensearch/->notes/, 05a-prose). Verified nbformat/grep.
… (Phase 6 chunk 7b-i) Added 'python -m utils.idp_config <cognito|okta>' entrypoint (argparse choices, explicit value = DR-12-consistent, boto3 imported after arg-parse so --help works offline; library API unchanged). Rewrote the Option-B DEPLOY walkthrough dual-IdP (Step-0 flag-select + [COGNITO]/[OKTA] Step-1 + net-new Step-5b GW2/OpenSearch mirroring the notebook arc). Rewrote the CLEANUP block dual-IdP reverse order (mirrors notebook 09). DANGLING-REF test_cognito_login.py -> decode_user_token.py. Verified py_compile/--help/ruff(0 new)/grep/fences-balanced.
… chunk 7b-ii)
Rewrote the Cognito-only deployment guide to dual-IdP (508→813 lines):
NEW Step 0 (python -m utils.idp_config flag-select, DR-12); Step 1 branched
[COGNITO] setup_cognito / [OKTA] setup_okta (okta-* keys, seeds
okta-user-*-sub=email); NEW Step 7 Notes Gateway GW2 + OpenSearch (7.1-7.6
mirroring notebook 05b); renumbered 0-10 + all cross-refs; interceptor
sub-steps relabeled 5.1/5.2 (avoid notebook-05b collision); Architecture
dual-IdP subsection (DR-1/DR-9); [OKTA] prereqs; B17 LF-admin union-append
note (§3a) + teardown-preserve note; GW1 SSM keys corrected to
{id,arn,url,name} (verified vs create_gateway.py); dual-IdP reverse-order
Cleanup (mirrors notebook 09; LF admins preserved); Quick Reference +
Directory tree regenerated; fixed stale L63 path.
Verified fences / step-consistency 0-10 / B17-grep / 9 cleanup paths on disk.
… (Phase 6 chunk 7b-iii) Added a [both IdPs] intro (RBAC is IdP-agnostic; only the group-claim source differs — cognito:groups/groups, seeder branches on IDP_PROVIDER). Naming reconcile: lakehouse-role-mappings→lakehouse_tenant_role_map, lakehouse-session-logs→lakehouse_user_login_audit, interceptor filename fixed. Mechanism accuracy: rewrote 'How the Interceptor Works' to the real group→role STS + RESPONSE tool-filter model (replaced X-User-Identity/X-User-Scopes header framing in prose; box-art diagrams left intact + a one-line 'simplified' caption under each). Tool accuracy: query_assigned_claims (nonexistent) → real adjuster set; added text_to_sql to the admin mapping row + Summary (admin = query_login_audit + text_to_sql; get_claims_analytics correctly absent). IdP-tagged identity-priority + group claim. Added a concise notes-RLS callout (OpenSearch owner_user_sub, DR-9) cross-referencing README Step 7 + notebook 05b. Verified stale-tokens=0 / tool-names in FINAL set / X-User-* only in box-art / fences even / diagnostics none.
…nt (Phase 6 chunk 8a) Added Application+Purpose tags at creation on net-new notes-side resources, correct shape per API: GW2 gateway (both branches, map) + role (list) + Cognito M2M provider (map); OBO provider (map); 4b runtime role (list) + runtime (post-launch tag_resource, fail-soft — the starter toolkit exposes no tags= kwarg as-written); AOSS collection (lowercase key/value; the 3 security/access policies are not taggable — API gap); notes-interceptor Lambda + role (CLI --tags). First installment of tag-all (DR-13); claims-side + shared infra follow in 8b. Verified py_compile / bash -n / ruff format --check / grep.
… — completes tag-all (Phase 6 chunk 8b) Tagged all remaining deployed AWS resources with Application+Purpose (completes DR-13 tag-all across 8a+8b): claims gateway/role/provider, shared interceptor role, request/response Lambdas, 4a runtime role + runtime (post-launch tag_resource, fail-soft), Cognito user pool (UserPoolTags), post-auth Lambda/role + login-audit DynamoDB, agent role + runtime (post-launch tag_resource), S3 Tables table-bucket + 2 tables + query-results bucket (fail-soft; needs s3tables:TagResource — Phase-7 perm check). Normalized the one stray Project tag (login-audit DynamoDB) -> Application/Purpose per DR-13. Gaps (no tags API): Cognito app-client/domain, S3 Tables namespace, Glue LF table, AOSS policies. DR-8: 9 upstream/shared files touched, ADDITIVE tags only. Verified py_compile / bash -n / ruff format --check / grep (Project=0).
…audit B1+B2) Replace the positional 12-template/4-slice pool (crashed at slice [12:15] for the 5th seeded user -> notes path couldn't deploy on the standard 5-user seed) with a canonical NOTES_BY_LABEL map (5 labels x 3 = 15 notes); assign by LABEL not sort-position (B2); add assert_label_coverage fail-fast pre-check (prevents B1 recurring if the seed changes). Notebook 07 prose corrected to label-keyed; count+disjoint answer key unchanged. Verified py_compile / ruff format --check / nbformat-structural + 22 cells compile / logic-check (5 labels, 15 distinct claim_ids, coverage pass + fail-fast). Real bulk_load = Phase-7 live check. Resolves audit B1+B2.
The agent runtime deployer read only Cognito keys -> omitted authorizer_configuration on the Okta path -> toolkit defaulted to IAM -> Okta bearer token couldn't authenticate. Now reads get_idp_provider once; Cognito branch keeps allowedClients=[cognito-app-client-id] (USER app client, NOT M2M); NEW Okta branch builds customJWTAuthorizer{allowedAudience:[okta-resource-server-audience], discoveryUrl:okta-discovery-url}; request_header_configuration hoisted to unconditional (both paths); IdP-aware status prints; pure-IAM fallback only for genuinely-unconfigured. Verified py_compile + AST structural (2 authorizer branches, header depth-0 unconditional, allowedClients x1 / allowedAudience x1, m2m-count=0) + grep. ruff deferred to pre-PR sweep (carry item 7 - no ruff in reachable env). Resolves audit B3.
…terceptor (R3 · audit M1) RESPONSE interceptor returned the FULL catalog on missing/invalid token, missing claim, or exception (fail-open, inherited from upstream). Now returns an empty catalog (deny-all) on every failure path; the exception handler rebuilds a minimal tools/list response with result.tools=[] from event (not partial locals). Reuses the existing no-mapping empty-list idiom. Dual-IdP branches preserved. Deliberate deviation from upstream fail-open (DR-14) - disclosed in PR notes. Verified py_compile/ruff-format/grep (venv).
added 7 commits
August 10, 2026 12:28
The header note read 'data-cell filters ship uninvoked -- a documented tutorial limitation', which invites the misreading that row isolation is untested. It is not: row isolation is enforced by a predicate bound to the caller's verified identity and is validated 12/12 on both IdP paths. Reword to give the actual rationale: a data-cell filter's row expression is statically defined, so per-user scope would require provisioning one filter per user. _apply_row_filter is retained and now documented as a hook for static, group-level row policies rather than reading as dead code. Verified by AST (not grep, which has undercounted this codebase 5x): 0 call sites pass row_filter by keyword and 0 originate a positional value across 75 .py files and 89 notebook code cells; the only 2 positional sites that reach the slot are the guarded internal forwards.
The previous note stated the rationale on authority. Cite it instead, so a reader can check it rather than take our word for it. A data filter's row expression is a subset of the PartiQL WHERE grammar that compares columns with constants, permits no PartiQL functions, and is fixed at filter-creation time -- so no caller-derived value can appear in it. That is why per-user row scope would need one filter per user, and why the bound identity predicate is the mechanism instead. Links the PartiQL support reference and the row-level access control tutorial, which scopes a reviews table by marketplace with one filter per value granted to a separate IAM principal. Also names the mechanism explicitly: both approaches get called 'user-level permissions', so the old wording read as if a workable feature had been skipped.
The row-scope claim was restated in eleven places across nine files. Last cycle corrected exactly one of them, so the sample shipped a corrected note next to ten stale ones -- and the next correction would have needed applying eleven times again. Canonical now: setup_lakeformation_permissions.py holds the full reasoning and the AWS references for code readers, and README.md gains a reader-facing section for people who never open the deployment scripts. Every other site carries one short consistent sentence pointing at the script. Applies the accurate mechanism throughout: a data filter's row expression takes constants only and permits no PartiQL functions, so it cannot reference the caller -- per-user scope would need one filter per user. Drops the 'documented tutorial limitation' and 'machinery exists but is uninvoked' framing, which invited the reading that row isolation is untested. README also notes that both approaches get called 'user-level permissions', for readers arriving expecting Lake Formation row filters. Citations follow existing repo convention: 'Reference:' blocks in docstrings, inline links in markdown.
…down Fills the three documentation gaps in the Okta path (worklist item 23) and discloses one code-level hazard found while doing it. API-token permission scope -- the one real gap. Every doc named the token and where to create it; none said what it must be able to do. An Okta API token inherits the role of the admin who creates it, and setup_okta.py writes to four admin surfaces: authorization servers, applications, groups, users. There is no pre-flight permission check, so a narrower token provisions part of the tenant and then fails with 403, leaving a half-built setup. Now stated as a table of the four surfaces, with Super Admin (or an equivalent custom role) as the instruction and verify_okta_setup.py as the recovery path. Plan-name currency -- "Okta Integrator Free Plan", matching the sibling AWS-operations-agent/okta-auth reference and Okta's current branding, with the former "Developer Edition" name in parentheses so either search term lands. What teardown removes, and the one thing it cannot. Cleanup is a full reset on both IdP paths: 09-optional-cleanup Step 8 runs cleanup_okta.py, which deletes both apps, the authorization server, three groups, five test users and twelve okta-* SSM keys. That was undocumented, so a reader had no way to know the teardown reaches into their Okta tenant, and no way to know that skipping that one step is how you keep it. Documented in three places, with the direction of the surprise reversed at the end: the reader's OKTA_API_TOKEN is NOT removed and cannot be -- they minted it by hand, so the sample never owned it and deleting the okta-api-token SSM key removes a copy, not the credential. Revoking it is now an explicit final step in the notebook's summary rather than an omission. Idempotency is adopt-by-name, and teardown deletes what it adopted. setup_okta.py reuses any object whose label matches and prints an informational "already exists" line; cleanup_okta.py deletes by those same constants, and no provenance is recorded anywhere durable, so the two cannot be told apart at teardown. The app and server names are distinctive and @example.com is reserved, but administrators / adjusters / policyholders are ordinary group names in a sample aimed at insurance teams. Disclosed in all three places with the mitigation (rename, or use a dedicated free tenant); the code fix is deliberately left to its own change since it alters the idempotency contract. .env destination made explicit. .env.example ships at the sample root; the loader also accepts deployment/.env and prints which file it read. Verified against utils/env_file.py find_env_file() rather than assumed -- both layouts are supported by design, so the docs now name the sample root without implying the other is wrong. Files: README.md, deployment/README.md, deployment/1-okta-setup/README.md, 09-optional-cleanup.ipynb (3 cells: intro, Step 8, final summary), .env.example. No behavioural change -- documentation and notebook prose only.
Findings from the first sweep of the shipped reader-facing docs against the
implementation (31 behavioural assertions examined: 20 confirmed with
file-and-line citations, 6 contradicted across 14 sites, 5 unverifiable).
Fixed in severity order. No code changes -- in every case the code was right
and the prose had gone stale.
1. The RESPONSE interceptor was described as doing "row/field filtering ...
+ result scoping" (05a cells 1 and 13). It does neither. lambda_function.py
returns early on `if method != "tools/list"` and the only field it ever
rewrites is result.tools. Corrected to match cell 5, which was already right
three cells away, and cell 5 is now the canonical statement: it says
explicitly that nothing else is touched, names where row and column scope
actually live (the caller-bound WHERE predicate and the Lake Formation grant
on the assumed tenant role), and records the fail-closed empty-catalog
behaviour. This one mattered most: a reader could have concluded row scope is
enforced twice when it rests entirely on the bound predicate.
2. Cleanup claimed to "revert the agent IAM OBO patch" (09, three sites -- two
markdown, one summary print). There is no such step in FINAL and there never
should be: the agent deliberately holds no OBO grant, so there is nothing to
undo. 06_cleanup_obo_gateway.py's docstring already said so. Claim removed
rather than replaced, with a short note pointing at 05b's canonical
statement so the absence reads as deliberate instead of missing.
3. The adjuster row predicate was documented as assignment only --
`WHERE adjuster_user_id = '{...}'` (scenarios.md). The code is a disjunction:
the caller's own claims OR, guarded by a role_exp CTE, the claims assigned to
them. The code is correct and the doc was wrong: the first disjunct is the
same row-scope predicate every caller gets, and dropping it would stop an
adjuster who is also a customer from seeing their own claims. Documented in
full, including why the branch exists (policyholders are not granted
adjuster_user_id, so referencing it raises COLUMN_NOT_FOUND) and that the
disjunct is latent with the shipped fixture, where all nine claims belong to
the two policyholders. The adjacent "cannot see" list read as exhaustive and
has been re-scoped.
4. scenarios.md drew column filtering as masking -- five table cells of block
characters standing in for filtered columns. The code produces absence, not
redaction: ColumnWildcard.ExcludedColumnNames for adjusters and a 15-name
column grant for policyholders, so SELECT * cannot return a blanked value.
Notebook 03 already said "filtering, not masking. Nothing is redacted or
blanked." The terminology pass that renamed 62 sites never reached the
pictures, and the picture is what a tester compares their output against.
Both tables now omit the filtered column and say what its absence means.
Also fixed in the same block: Scenario 1's "How" line described Lake
Formation column filtering as "(to be added to Claims table)". It ships and
is live; the parenthetical said the control did not exist yet.
5. Notebook 07 framed a dual-IdP test as Okta-only in three places, including
its headline invariant ("any two distinct Okta users ... the same Okta_App").
It runs on both paths and its own cell 29 says so. Restated in IdP-neutral
terms with the GW2 mechanism difference called out where it belongs.
6. Notebook 01's Okta prerequisites were the stale copy -- no API-token
admin-role requirement, superseded plan name. That requirement had landed in
four other files and missed the notebook that actually runs setup_okta.py,
which is the closest prerequisite statement to the 403 it prevents. Added,
with a pointer to the 1-okta-setup deep-dive as the canonical home.
Two prose corrections in the same pass:
- Notebook 02 stated the IAM principal-validation window as "the measured gap
was four seconds". A second clean-slate deploy needed two backoffs and six
seconds. Restated as a measured range with the retry budget, and says why the
2/4/8/16 schedule is necessary rather than generous -- one measurement would
have justified collapsing it and reintroduced the failure intermittently.
- Notebook 07 asserted "ROPC is blocked on the IE tenant" as though general. It
is a property of a specific tenant, named in a way a reader cannot check.
Scoped to per-tenant sign-on policy, with the reason PKCE is used regardless.
Where a claim appeared in several places one is now canonical and the others
point at it: 05a cell 5 for the interceptor's scope, 05b cell 20 for the agent
holding no OBO grant, 03 cell 11 for filtering-not-masking, and
1-okta-setup/README.md for the Okta prerequisites.
Verified: nbformat valid on all five notebooks; cell counts and type sequences
unchanged; every code cell parses except 01's pre-existing `%pip` magic, which
fails identically at HEAD; no ragged markdown tables in scenarios.md; the six
corrected strings return zero matches; hygiene scan clean.
…rming
load_env_credentials() warned on three branches that are all supported
configurations. The first is the one a tester meets on step one:
WARNING .env file not found at .env
Contract, established before editing: .env is ONE OPTIONAL RUNG of the
credential chain, not a requirement. init_aws() calls this function and
discards its return value, then calls get_aws_session() unconditionally, which
resolves credentials from container IAM role, then environment variables, then
an SSO profile, then default credentials -- the priority order both this
module's docstring and init_aws()'s own docstring already spell out. A reader
authenticating with `aws configure`, AWS SSO, or an instance/container role
never needs the file, on either IdP path.
So the answer to "is .env needed here?" is: optional, and its absence is fully
covered. That makes the old text misleading rather than merely noisy -- it
reported a non-event as a problem at the first step of the tutorial.
Three branches corrected, all to informational:
- File absent: now says it is optional, names what .env would supply, and names
the alternatives that work instead.
- File present with no AWS keys: this one bit hardest. A .env holding only
OKTA_* -- exactly what the Okta path asks the reader to create, and what a
Cognito reader inherits -- landed here and was told to add four AWS keys it
does not need. Prescribing an unnecessary fix is worse than a bare warning
because the reader may act on it.
- File present but setting nothing: same treatment.
Not suppressed. Each branch still reports what it found and what will be used
instead, so someone who INTENDED to supply credentials via .env still gets the
breadcrumb. Wording matches the shape utils/env_file.py already uses for the
same non-event, keeping the two loaders consistent.
Deliberately unchanged: the Optional/Tuple annotations and import order in this
file. ruff reports 12 findings here, byte-identical before and after this change
(9x UP045, 1x I001, 1x UP006, 1x UP035) -- pre-existing and tracked separately;
not folded into a message fix. `ruff format --check` passes, as it did before.
Verified: py_compile clean; ruff check identical to the pre-change baseline;
ruff format --check passes; all three branches exercised live and their output
inspected, including the OKTA_*-only case.
Two items.
1. nb07 asserted nothing about columns (item 20). The isolation scenarios prove
ROW scope; a Lake Formation column grant that drifted would have sailed
through a green ISOLATION TEST PASSED, and the 15/20/21 figures existed only
because they had been checked by hand, twice, out of band.
Approach: for each tenant role, assume the role, call glue:GetTable on
`claims` under those credentials, and count the columns that come back. Lake
Formation filters the table DEFINITION per role, so the returned count is the
grant -- the same mechanism that makes SELECT * expand to a different width
per persona.
Why this shape rather than the obvious alternatives:
- Counts, not names. Enumerating the expected column list would duplicate the
grant and leave two copies to keep in step. One number per role detects
drift in either direction and fails loudly without restating the grant.
This is the distinction from the SELECT * decision: naming columns in a
QUERY turns a grant change into a total outage, whereas naming a count in a
TEST turns it into a loud, specific failure. Different drift modes,
opposite right answers.
- Reads the grant rather than a query result, which is what lets it cover all
three roles. Only policyholder and admin tokens are captured by the
harness, so a query-based check could not have reached adjusters at all
without adding another browser login to a run that already has to fit in
one ~1h Okta token window.
- No IdP branch, and none possible. Grants attach to the IAM tenant roles,
which are identical on both paths; the IdP only decides how a caller maps
onto one of those roles. So the assertion cannot silently cover one path
only. It also needs no persona tokens, so it still runs when token capture
has failed -- which is when you most want to know the data layer is intact.
Failure-mode separation is the load-bearing part, because this ships without
ever having run live. A count that comes back and disagrees reports
"policyholders returned 14 columns, expected 15" under a COLUMN GRANT DRIFT
banner. A count that could not be obtained reports "could not determine
column count -- <reason>" under an ASSERTION COULD NOT DETERMINE COLUMN
COUNTS banner, which says in as many words that it is not a drift result.
Both fail the run -- neither is ever a silent pass, matching the existing
INCONCLUSIVE discipline -- but they call for opposite next actions, so they
must be distinguishable without reading code. An absent or empty Glue column
list is deliberately treated as unknown rather than as a count of zero, so a
response-shape change cannot masquerade as total revocation.
2. opensearch_tools.py documented search_claim_notes' parameter as the "Okta
sub" in a function that serves both paths (server.py branches Cognito/Okta).
Corrected to name both sources. Same defect one layer down from the notebook
framing fixed in the previous commit; filed as its own item so it was not
lost, since docstrings were scoped out of that audit.
Verified offline, since the stack is down: nbformat valid; 45 cells, every code
cell parses; ruff check and format clean on the touched module. The new cell's
logic was exercised against fakes across all seven branches -- happy path, drift
in either direction, SSM unavailable, assume-role denied, glue:GetTable error,
and an empty column list -- confirming each produces the right tag, the right
banner, and that the drift and could-not-determine paths are mutually exclusive.
Unproven until the next Okta cycle: the live AWS calls themselves. Specifically
the CatalogId form ({account}:s3tablescatalog/{bucket}, copied from the MCP
server's working call), that the tenant roles are assumable by the notebook's
caller, and that Glue returns LF-filtered columns to an assumed tenant role as
expected. Any of those being wrong surfaces as COULD NOT DETERMINE with a named
reason, not as a spurious drift report -- which is exactly what that separation
is for.
nweispfenning
marked this pull request as ready for review
August 11, 2026 12:38
nweispfenning
marked this pull request as draft
August 11, 2026 12:53
…ader The runtime validates the inbound Authorization header and forwards it into the container, and the header allowlist that enables this is already configured on both identity-provider paths. So the agent no longer needs the token duplicated in the request body. - Read the token from the request context's forwarded headers, preferring it over the body field. Both the canonical and lowercase header names are accepted, with or without the "Bearer " scheme prefix. - Check the value's shape (three dot-separated segments) before using it. On a runtime configured without a JWT authorizer, Authorization carries a SigV4 signature instead, and nothing upstream distinguishes the two: the header is not treated as restricted and is normalised into the request context before any allowlist rule is consulted. Without this check a signing credential would be forwarded to the gateways as if it were a user bearer. - Keep the body field as a transitional fallback that logs a warning when it is used, so a caller still sending the token that way becomes observable instead of assumed. - Stop sending the token in the request body from the Streamlit UI and from the two runtime test scripts. All three already send the Authorization header, which is required to reach the runtime at all. The entrypoint's second parameter must be named `context`. The runtime SDK inspects the handler's parameter names and passes the request context only under that name; with any other name it invokes the handler with the payload alone and no request headers are available, with no error raised. That requirement is documented in the docstring next to the signature, and the code logs an explicit diagnostic naming it if the context is ever absent. Also sorts the import block in the two test scripts. The lint rule flagged this before this change, but editing the files brings them into the lint gate's changed-file set, so it is fixed here rather than surfacing as a new failure.
Replaces the third-party OAuth component on the Okta path. That component drives the flow through a popup and polls the popup's location from inside its own iframe; when the browser refuses that cross-origin read it discards the security error and keeps polling, so the sign-in never completes. The behaviour is in the component's own shipped bundle and is identical across the Streamlit versions tested, so there is no version to pin. The app is already a web server, so it can run the flow itself and no popup is involved. Its own origin is the redirect target, which is the URL the Okta setup script already registers, so no identity-provider configuration changes and existing test organisations keep working. What the flow does now: - Builds the authorization URL with PKCE S256: a high-entropy verifier is kept server-side and only its SHA-256 challenge is sent. `prompt=login` forces a fresh credential prompt so a reader can switch test users, and an optional email is passed as a login hint to pre-fill the username. - Receives the authorization code as a query parameter on the app's own page, then exchanges it at the token endpoint using HTTP Basic client authentication, as a confidential client. Three details that are load-bearing rather than incidental: - The verifier and state must outlive a full page navigation, and that navigation starts a new session, so session state is gone by the time the code arrives. They live in a server-side store keyed by state instead. - That store is treated as security-relevant. The returned state must resolve to an entry we issued, entries are single-use so a replayed code cannot be paired with the same verifier twice, and entries expire so an abandoned attempt does not leave a verifier resident. The attempt is minted once per session rather than once per rerun, so keystrokes in the hint field do not accumulate entries. - The query parameters are cleared after a successful exchange. Reruns are frequent, and a code left in the URL would be resubmitted against a code the provider has already consumed. Scope of the change: confined to the Okta branch, and the session contract both branches converge on (access token, id token, display identity) is unchanged. The Cognito password-grant path is untouched. The OAuth component is dropped from the UI requirements; the flow now needs only the standard library and the HTTP client already in use.
The sign-in link was rendered with st.link_button, which always opens a new tab. The flow itself worked, but the tab the reader started in was left showing a logged-out login form while the signed-in session lived in a different tab. That looks like a failure even though the sign-in succeeded, and it is a regression against the previous behaviour, where the sign-in window closed and the original tab became signed in. Rendered as an anchor with target="_self" instead, so the current tab navigates to the identity provider and returns signed in. The mechanism is unchanged: the return trip is still a full page navigation, so the verifier and state still come from the server-side store rather than from session state.
The hosted-domain step could report success while leaving the user pool with no domain at all, producing a deployment that completes with every step green and cannot serve a single request: with no domain there is no /oauth2/token endpoint, so the machine-to-machine client-credentials grant fails, so both gateway targets fail to authenticate, and the agent answers nothing. No error appeared anywhere -- the only signal was an informational line reading as though idempotency had kicked in. Three separate causes, each addressed here: Name derivation. The domain suffix came from a fixed-length slice of the whole user pool id, which begins with the region, so "us-east-1_UkB4mHbEr" yielded "useast1u" -- seven characters of region and one of identity. That is a 36-wide space in a namespace shared by every AWS account in the region, so unrelated deployments collide readily. The suffix now comes from the pool id's unique segment, the part after the underscore. Error handling. The except block treated any error whose text contained the word "domain" as benign, which covers nearly everything an API whose own parameter is named Domain can raise. In particular it swallowed "Domain already associated with another user pool" -- the case where the name is taken and the pool ends up with no domain. Only a domain that already belongs to this pool is benign now; anything else fails the step with the underlying message and a note about what breaks downstream. Hostname confirmation. The stored hostname was composed from the name the script intended to use, which asserts success rather than establishing it. It is now read back from the API after creation, so what reaches SSM is what the service actually holds. cleanup_cognito.py already reads the domain from the API before deleting it; this applies the same pattern on the way in. Verified: the derivation is covered by an offline test that lifts the three derivation lines out of this file and exercises them against sample pool ids, including the specific id that collided and two pools that share a first character. Formatting, lint and compile checks pass.
Running notebook 07 could make the Streamlit walkthrough impossible to
complete, and reported success while doing it.
set_test_user_passwords.py resolved its password as
os.environ.get("LAKEHOUSE_TEST_PASSWORD") or _generate_password()
so with no environment variable set -- the default, since nothing instructs
the reader to set one -- it generated a random password, applied it
permanently to all five personas, printed "Set permanent password for 5/5
test personas", and deliberately never printed or stored the value. Notebook
07's prep cell calls this helper, so simply running the isolation test
replaced every persona's credential with one the reader could not learn.
Notebook 08 then could not be completed, and nothing said why. Meanwhile the
sign-in panel advertised TempPass123!, which was by then wrong.
Three changes, and the coupling between them is the actual fix:
The fallback is now the documented default. With no environment variable the
helper uses the same value setup_cognito.py already assigns as the initial
TemporaryPassword and that notebook 08 already documents. This restates a
published demo credential rather than introducing one, and it keeps the
default reader's path working with no prerequisite. Refusing outright was
considered and rejected: the helper is called from notebook 07, so refusing
would stop the isolation test running at all for the default reader, which
moves the breakage earlier and wider rather than fixing it. The random-value
path is removed rather than left unused.
The helper now names its source. It prints whether the password came from the
caller, from the environment, or from the documented default -- never the
value. This is the fix for the failure mode rather than for the immediate
bug: the value used and the value documented can legitimately differ, and
when they do, the only way to sign in is to know which one won. That
information previously existed nowhere a reader could reach it.
The sign-in panel no longer displays a password, and the notebook documents
it instead, immediately before the reader is told to sign in -- including the
override rule, so the instruction is true whether or not the environment
variable is set. The app cannot know which value the setup scripts actually
applied, so a credential hint rendered there can silently disagree with
reality; a hint that is confidently wrong is worse than none, because it is
the one value a reader will try.
The three places that state this password -- setup_cognito.py, the helper and
notebook 08 -- must agree, and a comment on the constant says so.
Verified offline, 27 checks, no AWS: resolution order (caller > environment >
documented default) exercised through the shipped function with fake clients;
the generated-password path proven absent rather than merely unreachable; the
source named and the value never printed on all three paths; and the four
files that mention the password asserted to agree, since nothing was
individually wrong before -- the pieces simply disagreed. Formatting, lint,
compile and notebook validation pass.
Also in this commit, because it touches the same panel and serves the same
goal: the active identity provider is now emphasised in the sidebar rather
than captioned. It already rendered from shared code above the login widget,
so both paths showed it, but as de-emphasised grey text. Its job is to answer
"which path am I on?" at a glance -- the two login experiences differ by
design, a password form on one and a redirect to the provider on the other,
and without a legible indicator someone reporting a problem cannot say which
they were looking at. That happened during testing: the panel was described
as looking completely different and it took a source read to establish which
branch had rendered.
The two identity-provider paths should match rather than merely resemble each
other. Both UI changes here follow from that: a credential hint that exists
on one path only is asymmetric by construction, and an indicator that is
present but hard to read does not let a reader tell the paths apart.
setup_cognito.py printed both the user app client secret and the machine-to-machine app client secret in the clear at the end of a run. The exposure is durable rather than transient. The deploy notebook runs this script with subprocess.run(...) and no capture_output, so the child's stdout is inherited by the Jupyter kernel, captured as that cell's output, and saved into the .ipynb file. A reader who runs the notebook and commits their work commits a live OAuth client secret into their own repository. Console output also gets pasted into issues and chat threads. Nothing is lost by removing the values. Both secrets are already written to SSM Parameter Store as SecureStrings earlier in the same run (cognito-app-client-secret, cognito-m2m-client-secret), and every component that consumes them reads them from there. No notebook, README or script asks the reader to copy the printed value. Both lines now print a fixed-width mask, matching the convention already used in create_gateway.py, which prints a mask when it loads a secret rather than the value. The two CodeQL clear-text-logging suppression comments went with the lines they existed to silence.
The isolation notebook loaded three persona tokens and asserted nothing about how long they had left to live or who issued them. The two policyholder tokens were decoded and checked for distinct subjects and per-provider claim shape; the admin token was read off disk and never decoded at all. An expired token therefore surfaced much later, at the runtime authorizer, as an empty tool list or a missing row. In an isolation test an empty result looks exactly like the access controls working correctly, so a missing prerequisite presented as a passing security property or as an isolation regression, depending on which cell hit it first. The column-count assertions need no tokens and would still have passed, which made the run look half-green. All three loads now go through one helper that asserts the token decodes, that exp is present and more than five minutes in the future, and that iss matches the issuer of the active identity provider. Five minutes rather than zero because the scenarios take several minutes and a token dying part-way through is the failure being prevented. An absent exp claim is treated as unknown and fails, never as acceptable. The issuer forms are taken from the request interceptor rather than rebuilt here, so this assertion cannot disagree with the component that actually validates the token at runtime. Each token now reports its remaining lifetime and issuer as it loads, so freshness is visible before anything can fail. Every failure names the persona, says to sign in again and how, and states that no isolation control has been exercised yet, so a stale credential cannot be mistaken for a test result.
…eployment for RunTimes" This reverts commit 9812327.
Adds an AgentCore CLI (0.30.0) project at lakehouse-agent/agentcore/ that deploys the claims MCP server, the notes MCP server and the agent as CodeZip runtimes from their existing source directories, plus a pyproject.toml per runtime for the CLI packager. agentcore.json is committed with placeholders only; agentcore/deploy_runtimes.py fills the IdP authorizer, environment variables and role ARNs from SSM at deploy time, writes aws-targets.json from the caller's identity, restores the placeholders, and stores the runtime ARNs where the gateway steps read them. Deployer-specific files are gitignored.
…cution roles Creates and deletes the three AgentCore Runtime execution roles (lakehouse-mcp, opensearch-mcp, lakehouse-agent) in one script, so role management no longer depends on the starter-toolkit deploy and cleanup scripts. Trust policy, permission policies, descriptions and tags are unchanged from those scripts. Create is idempotent (in-place update of the script's own trust and inline policy); delete skips missing roles. --role limits either action to named roles.
Each runtime directory now holds its own CLI project (deployment/<runtime>/agentcore/), so each runtime deploys as its own stack at the notebook step that needs it. codeLocation is the runtime directory itself; runtime source is unchanged. The single sample-level project is removed; the deploy wrapper stays in place for now. .gitignore rules follow the new paths.
The deploy wrapper moves to deployment/agentcore_cli_deploy.py and takes the runtime to deploy (lakehouse-mcp, opensearch-mcp or lakehouse-agent). It checks that runtime's execution role, fails naming the notebook that produces any missing input, injects and restores that project's agentcore.json, and stores the runtime ARN and ID under the SSM names the later steps read. --destroy deletes that project's stack and SSM keys only.
…ooks Notebooks 04, 05b and 06 now create each runtime's execution role and then deploy that runtime's CLI project with deployment/agentcore_cli_deploy.py. Notebook 09 destroys each runtime's stack and deletes its role only after the stack is gone. The deploy script now finds the npm AgentCore CLI past the starter toolkit's agentcore inside the virtual environment, so the notebooks work from an activated venv.
READMEs: AgentCore CLI prerequisites (Node.js 20+, @aws/agentcore@0.30.0 exact, uv; Docker no longer needed), the two-agentcore-tools clash, per-runtime role creation, aws-targets.json handling, and why gateways and interceptors stay on scripts. Runtime deploy and teardown steps now call setup_runtime_roles.py and agentcore_cli_deploy.py. Notebook 09 intro and summary describe stack + role teardown. Utility scripts point at the new deploy commands.
The three runtimes now deploy through the AgentCore CLI (deployment/agentcore_cli_deploy.py) with execution roles from setup_runtime_roles.py, so the old toolkit scripts are removed: 4a and 4b deploy_runtime.py / cleanup_runtime.py, and 6 deploy_lakehouse_agent.py / cleanup_agent.py. The starter toolkit is dropped from the root, 4a and 4b requirements.txt; the bedrock-agentcore SDK stays. References to the removed scripts in comments, docstrings, printed next-step hints and the advanced-interceptor cleanup block now name the CLI commands. The 'two tools called agentcore' notes and the wrapper's clash message now describe a previously installed toolkit.
… failures The Okta token-exchange troubleshooting section now says where to look when an exchange fails (System Log, event type app.oauth2.as.token.grant) and adds the DPoP condition: both Okta apps must have DPoP disabled or the exchange fails with invalid_dpop_proof. It links the AWS post on on-behalf-of token exchange with AgentCore Gateway, whose Common pitfalls section covers the same gates. The top README gains a Further reading section with that post.
…ts and docs Comments, docstrings, Markdown and notebook prose carried internal design-record identifiers that a reader cannot resolve, since the design document they point to is not part of the sample. The identifiers are removed; the surrounding text is kept, and reworded only where a sentence depended on the identifier (for example, the pre-flight check is now named as the pre-flight IdP-mismatch guard). No executable code changes: every edited Python file parses to the same AST once docstrings are set aside, and every notebook code cell is unchanged apart from comment lines. Printed strings and API description arguments were left as they are.
Finishes removing internal tracking references that a reader cannot resolve: requirement numbers, finding numbers and task-list pointers in comments, docstrings, Markdown and notebook prose, plus the design-record tags left in four printed or API strings (the notes gateway description, two test banners and the cleanup summary). Surrounding text is kept and reworded only where a sentence depended on the pointer. Only string literals and comments change: every edited Python file parses to the same AST with string constants masked, and each changed string is one of these removals.
Comments, docstrings, one README sentence and two printed or API strings pointed readers at numbered sections of a design document that is not part of the sample. The pointers are removed and the surrounding text kept, reworded only where a sentence began with the pointer. Only string literals and comments change: every edited Python file parses to the same AST with string constants masked.
… comments Eight comments in the gateway setup scripts and interceptor Lambdas pointed at a numbered section of a design document that is not part of the sample. The pointers are removed; where a comment named an SSM key or key family, that name is kept. Only comments change: every edited Python file parses to the same AST.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(lakehouse-agent): flag-routed Okta + OBO alongside Cognito
What this does
Adds Okta as a supported identity provider for the existing
lakehouse-agentuse case, selected at deploy time via
IDP_PROVIDER∈ {cognito,okta}(SSM:
/app/lakehouse-agent/idp-provider). Cognito behavior is unchanged.The Okta path additionally demonstrates on-behalf-of token exchange per
RFC 8693: the agent exchanges the caller's access token for a downstream
token, so the second gateway enforces the end user's identity rather than the
agent's. This complements the existing inbound-auth sample at
01-features/05-authenticate-and-authorize/01-inbound-auth/03-inbound-auth-okta— that one establishes inbound JWTauthorization; this one carries caller identity through to a second hop inside
an end-to-end use case.
This design also follows AWS guidance against forwarding an inbound JWT as-is
to downstream services, exchanging it for a least-privileged scoped token
instead.
All changes are confined to
02-use-cases/01-conversational-agents/lakehouse-agent/. No repo-level filesare modified.
Architecture
CUSTOM_JWTauthorizer plus REQUEST/RESPONSE Lambdainterceptors. REQUEST gates tools per tenant group and exchanges the JWT for
scoped IAM credentials; RESPONSE filters the advertised tool catalog. Both
fail closed.
claim_name, so onetable shape serves both IdPs.
WHERE user_id = ?).Caller values are escaped as SQL literals before being passed as Athena
execution parameters, so malformed input fails closed rather than widening
scope. Lake Formation provides column filtering and table grants; data-cell
filters are not used.
01-deploy-cognito→01-deploy-idp;4-→4a-.mcp<2,opensearch-py<4.Validation
Both IdP paths were validated against live deployments, 12/12 assertions each,
against an identical answer key:
(4 and 5 rows); cross-tenant reads return nothing.
adjuster_user_iddenied.Scope note: changes after that run are limited to lint compliance,
documentation and notebook restorations, and a cleanup-script fix. A
confirmatory regression pass is pending, which is why this is opened as a draft.
A note on the lint gate
This PR adds a sample-local
ruff.tomlthat extends the root config and ignoresseven rule families the sample deliberately uses (
except Exceptionas thefail-closed idiom, naive datetimes load-bearing for fixtures, vanilla exceptions
required by a hard-fail contract, and others), each annotated inline.
Context: the root
pyproject.tomldeclares noselect, so it inherits ruff'sdefault rule set, and CI installs ruff unpinned — so the gate tightens on its own
as ruff releases. It currently reports ~1,000 violations across already-merged
peer files in this directory. Pinning ruff or declaring an explicit
selectatthe repo root would help every contributor; happy to open that separately.
Known gaps — contributions welcome
None affect the validated paths; all fail closed where they fail.
UI
streamlit-oauthis unmaintained and its popup flow can hang. Likely fix:adopt the in-repo pattern at
03-integrations/ux-examples/streamlit-chat.Cleanup / teardown
cleanup_runtime.pydeletes its CodeBuild project but not its own SDK role,leaving orphans.
revoke_permissionsfor Lake Formation grants. Tenant role names aredeterministic, so stale grants can re-attach to recreated roles on a later
deploy — worth fixing.
(redeploy idempotency).
Auth / IdP
flow can't be driven by an automated end-to-end login. Token-paste mode
(
--token) is provided for the debug helpers.payload. Header-based propagation is the cleaner approach and was not pursued:
gateway targets here have no
metadataConfiguration.allowedRequestHeadersset, which is the likely original blocker.
query_login_auditis offered to Okta administrators, but the login-audittable is created only on the Cognito path. The call fails closed with no data
exposure, but the error text points at a Cognito-only script and should be
made IdP-aware.
Should be parameterized.
tightened, for tutorial clarity.
Verification scope
induction: the gateway interceptor returns 403 before those paths are reached.
client-credentials check is not wired up.
RUF013is suppressed in bothserver.pyfiles: widening tool parameter typeschanges the JSON
inputSchemathe gateway advertises, so it needs acoordinated runtime and gateway-target redeploy.
contexts (local venv, container base, Lambda runtime).
Future work
MIGRATION.md.Notes for reviewers
main; zero file overlap with upstream changes.%pipand subprocess calls usesys.executable, so both target the kernel's interpreter rather than PATH.sys.executablematches the repo's dominant convention;%pipfollows 46 other files here, though the contributing template still shows!pip.