Repository navigation
fix(aws-startup-advisor): default to bedrock-runtime in llm-to-bedrock and gcp/azure AI guidance, add a Runtime Anthropic Messages lane - #585
Open
herosjourney wants to merge 32 commits into
Conversation
added 5 commits
November 6, 2026 23:17
…tadata from the public model cards The plugin recorded GPT-5.4 and GPT-5.5 as bedrock-mantle-only (no CRIS, no inference profile, no bedrock-runtime surface) with Chat Completions "unconfirmed", a 272K context window and no long-context price tier, and the canonical catalog cited "internal reference" probes as provenance. The public model cards no longer say any of that. This PR corrects the metadata only; it does not change endpoint selection, dispatch, or add new code paths. Sources (read 2026-11-06 via the AWS Documentation MCP tools): - https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-54.html - https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-55.html What the cards state, now recorded verbatim in the catalog and prose: - Endpoints: bedrock-runtime supported (Responses, Chat Completions, Converse; not Messages/Invoke) and bedrock-mantle supported (Responses, Chat Completions). Runtime has no in-region form; requests name us.openai.gpt-5.4 / global.openai.gpt-5.4 and us.openai.gpt-5.5 / global.openai.gpt-5.5. Mantle in-region: 5.4 = us-east-1, us-east-2, us-west-2, us-gov-west-1; 5.5 = us-east-1, us-east-2. US Geo CRIS from 6 regions, Global CRIS from 33. - Context 1,050,000 tokens shared, 128,000 max output. - Pricing (Standard, In-Region only; Priority/Flex unsupported): 5.4 short 2.75 / 0.275 cache read / 16.50, long 5.50 / 0.55 / 24.75, GovCloud short 3.375 / 0.3375 / 20.25; 5.5 short 5.50 / 0.55 / 33.00, long 11.00 / 1.10 / 49.50. No Geo/Global price rows, so prose keeps the ~10% figure for these two on every inference option and never claims parity. - Runtime features: invocation logs, streaming, Guardrails and application inference profiles (Converse only), implicit prompt caching and structured outputs (Responses only); no server-side tool use. Changes: - skills/shared/ai/models/openai-bedrock.json: GPT-5.4/5.5 rows re-sourced to the public cards (no "internal reference"), mantle_openai_chat available, runtime_converse path with CRIS, inference_profiles (us/global), region_availability, pricing tiers, context/output limits; path-level constraints corrected. - Canonical shared docs: ai-openai-to-bedrock.md (Tier 0 table, region gate, caching, Bedrock-native section, volume guidance), ai-migration-guardrails.md, schema-design-aws-ai.md (ban is on BARE ids paired with converse/runtime, not on the models), bedrock-pricing-cache.md (rows, long-context table, caching note, GovCloud rate). - gcp-to-aws and azure-to-aws: references/shared/openai-on-bedrock.md, shared/schema-design-aws-ai.md (azure), design-ai.md, design.md/generate.md _assert lines (azure), generate-artifacts-ai.md (azure), generate-ai.md, discover-preview.md and clarify-ai-only.md (gcp), clarify-ai-only.md (azure). - bedrock-migration-assessment clarify-ai-only.md; agent-advisor model-recommend.md (documents the engine's current GPT-5.x reshape as an engine limitation, not a model fact). - llm-to-bedrock: resolve-bedrock-model-id.md (Case A/B), behavior-delta openai-to-bedrock.md, SKILL.md (two id-form sentences); docstrings/comments in preflight_bedrock.py, iam_policy.py, bedrock_pricing.py and the preflight 404 remedy string. The is_mantle_model / is_mantle_gpt ID-form logic is unchanged. - agents: llm2bedrock-code-rewriter.md (Converse rule is id-form-dependent), llm2bedrock-report-generator.md (cost text), llm2bedrock-prompt-evaluator.md. - openai_model_recommendation.py: only the chat_completions_to_responses block message text (dropped the "GPT-5.x on Mantle rejects Chat Completions" claim). - Tests: tests/test_gpt6_review_round2.py, test_gpt6_review_round5.py, test_gpt6_round4_findings.py and agent-advisor scripts/test_openai_model_recommendation.py now assert the corrected public facts (CRIS ids present, "mantle-only"/"no CRIS"/"unconfirmed" absent, catalog provenance is the public card). New regression tests in test_preflight_bedrock.py, test_iam_policy.py, test_bedrock_pricing.py prove us./global.openai.gpt-5.4|5.5 classify as runtime and bare ids as mantle in all three helpers. - Vendored copies under agent-advisor, gcp-to-aws, azure-to-aws re-synced with tools/sync-vendored.py. Validation (all run from the worktree with mise Python 3.12): - mise -C plugins/aws-startup-advisor run ci: PASS - lint:md 0 errors; lint:model-id OK (19 retired IDs enforced); lint:model-staleness fresh (4d old, 45d window); lint:model-catalog-drift OK (3 pre-existing warnings: sonnet-5-5, haiku-5-5, gpt-6.1-terra not in catalog; unrelated); lint:startup-model-pricing OK; lint:artifacts PASS (99 checked, 0 errors); lint:vendored-parity PASS (62 files byte-identical); lint:skill-refs PASS (1482 checked, 0 missing); lint:consent-disclosure OK; test: 1395 passed, 222 subtests passed; test:telemetry: 311 passed; test:heroku-source-review: 44 pass, 0 fail - agent-advisor pytest (skills/agent-advisor/scripts): 564 passed, 0 failed (562 at base + 2 new) - llm-to-bedrock pytest (skills/llm-to-bedrock/scripts): 418 passed, 0 failed (415 at base + 3 new; needs --with httpx --with aws-bedrock-token-generator) - root: python3 tools/validate.py "All validations passed"; mise run lint:md 0 issues / 2763 files; lint:manifests, lint:skill-parity, lint:spec all pass; test:tools 6 OK. mise run build's security step fails on a pre-existing gitleaks false positive in upstream commit f8d09fd (tests/test_bedrock_migration_assessment_discover.py:118, "claude-3.5-sonnet"); gitleaks protect over this change set: no leaks found. - grep over plugins/ for mantle-only / "no CRIS" / "no inference profile" / "no runtime path" / "unconfirmed" / "internal reference" / "no 1M tier" near GPT-5.4/5.5: no remaining claims (only negative test assertions and the bare-id rule "bare proprietary openai.gpt-* IDs use Mantle only"). Base: 0a218f8 (upstream/main)
…ift caused by the GPT-5.4/5.5 catalog facts Review round 1 (CHANGES_REQUESTED, 2 blockers) on c1944a5. Finding 1: recording the public GPT-5.5/GPT-5.4 facts in skills/shared/ai/models/openai-bedrock.json (mantle_openai_chat available, region_availability) changes agent-advisor recommendations through the unchanged catalog-order fallback in openai_model_recommendation.py (_mantle_candidate_order -> _catalog_model_for_path): GPT-5.5 and GPT-5.4 precede GPT-6 Astra in catalog key order, so a GPT-6 Sol / 6.1 Sol Chat Completions source outside us-east-1 now lands on - openai.gpt-5.5 via mantle_openai_chat in us-east-2 (base: decision_required, mantle_region_unavailable, options mantle chat / runtime_converse) - openai.gpt-5.4 via mantle_openai_chat in us-west-2 (base: openai.gpt-6-astra) and a GPT-5.6 Sol Responses source in us-west-2 lands on openai.gpt-5.6-terra (base: openai.gpt-5.5, whose cataloged footprint is us-east-1/us-east-2 only). The previous commit message called this PR "no behavior change"; that was wrong for these cells. The catalog facts are kept (they are what the public cards state); the engine ordering is NOT changed in this PR. Instead: - skills/agent-advisor/scripts/test_openai_model_recommendation.py: new regression tests pin all five cells so the shift is deliberate and visible (test_gpt6_chat_outside_own_footprint_falls_back_to_in_region_gpt5_chat x4, test_gpt56_sol_responses_outside_us_east_skips_gpt55_for_in_region_terra). - skills/agent-advisor/references/phases/model-recommend/model-recommend.md: the engine-limitation paragraph now states that Chat fallbacks walk catalog order, that GPT-5.5/5.4 precede GPT-6 Astra, and that a GPT-6 Chat source outside us-east-1 lands on GPT-5.5 (us-east-2) / GPT-5.4 (us-west-2); a newer-generation source falling back to an older-generation model is an engine ordering limitation to resolve in the follow-up selection PR (PR 2). Finding 2: skills/agent-advisor/scripts/openai_model_recommendation.py:118-119 still said "the (Mantle-only) GPT-5.x family". Reworded to the engine fact (GPT-5.x sources not recognized as same-model runtime identities map to Claude tiers; GPT-5.5/5.4 do carry a cataloged runtime_converse path via us./global. CRIS ids). Grep over plugins/ for mantle-only / "no CRIS" / "no inference profile" / unconfirmed / "internal reference" near GPT-5.4/5.5/5.x is now clean apart from negative test assertions, the GovCloud Opus 5.5 pricing row, and the bare-id rule ("bare proprietary openai.gpt-* IDs use Mantle only"). Carry-forward for PR 2 (endpoint selection): reorder / tier the mantle_openai_chat candidate walk so a GPT-6 source does not fall back to GPT-5.4/5.5 ahead of GPT-6 Astra; add GPT-5.5/5.4 to _SOURCE_MODEL_KEYS so they are recognized as same-model runtime identities. No new behavior added; no dispatch or endpoint-selection code changed. No vendored files affected (none of the three files is in the sync manifest); lint:vendored-parity re-run and still 62/62 byte-identical. Sources (public model cards, read 2026-11-06 via the AWS Documentation MCP tools): - https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-54.html - https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-55.html Validation (run from the worktree with mise Python 3.12): - mise -C plugins/aws-startup-advisor run ci: PASS - lint:md 0 errors; lint:model-id OK (19 retired IDs enforced); lint:model-staleness fresh (4d old, 45d window); lint:model-catalog-drift OK (3 pre-existing warnings, unrelated); lint:startup-model-pricing OK; lint:artifacts PASS (99 checked, 0 errors); lint:vendored-parity PASS (62 files byte-identical); lint:skill-refs PASS (1482 checked, 0 missing); lint:consent-disclosure OK (6 checked); test: 1395 passed, 222 subtests passed; test:telemetry 311 passed; test:heroku-source-review 44 pass, 0 fail - agent-advisor pytest (skills/agent-advisor/scripts): 569 passed, 0 failed (564 at c1944a5 + 5 new) - llm-to-bedrock pytest (skills/llm-to-bedrock/scripts): 418 passed, 0 failed - root: python3 tools/validate.py "All validations passed"; mise run lint:md 0 issues / 2763 files; lint:manifests, lint:spec, lint:skill-parity pass; test:tools OK. mise run build's security step still fails only on the pre-existing gitleaks finding in upstream commit f8d09fd (tests/test_bedrock_migration_assessment_discover.py:118); gitleaks protect --staged over this change set: no leaks found. Base: 0a218f8 (upstream/main)
…re constraints per model for GPT-5.4/5.5
The GPT-5.4 and GPT-5.5 model cards publish only us. and global. CRIS ids;
the in. profile exists for GPT-5.6 only. The runtime_converse path strength
and both schema-design vocabulary/checklist lines attached us./in./global.
to all three models, contradicting the catalog's own next constraint and
openai-on-bedrock.md. Rewritten so in. stays on GPT-5.6 / GPT-6.
The GPT-5.6 runtime constraint ("does not support server-side tool use,
structured outputs, or application inference profiles") is now scoped to
GPT-5.6, and a separate GPT-5.4/5.5 constraint records what their cards
mark: server-side tools unsupported; structured outputs and implicit prompt
caching Responses-only; Guardrails and application inference profiles
Converse-only.
Vendored copies re-synced (62 checked, 3 synced). Regression test added.
… the rewriter and Design/Generate checklists bc51f0c scoped the in. prefix to GPT-5.6 in the catalog and schema, but three consumers of that rule still said us./in./global. for GPT-5.4/5.5: agents/llm2bedrock-code-rewriter.md, gcp-to-aws design-ai.md's checklist, and azure-to-aws generate-artifacts-ai.md's checklist. Their model cards publish only us. and global. ids. Each now reads: us./in./global. for GPT-5.6; us./global. only for GPT-6, GPT-5.5 and GPT-5.4. Adds tests/test_gpt54_55_cris_prefix_scope.py, which reads the catalog, the shared/Azure schemas, the guardrails guide, and the three consumer files and fails on any window that pairs GPT-5.4/5.5 with a grouped us./in./global. grant without scoping in. to GPT-5.6. Verified to fail against the previous wording of all three files.
…and Luna only, not Sol
The GPT-5.6 Sol card (read 2026-11-07) publishes us.openai.gpt-5.6-sol and
global.openai.gpt-5.6-sol only; Terra and Luna publish us./in./us-gov./global.
Prior wording granted in. to "GPT-5.6" family-wide in the catalog strength,
the rewriter, the GCP Design and Azure Generate checklists, both schema
copies, and ai-migration-guardrails.md ("GPT-5.6 Sol/Terra/Luna
(us./in./global.)"). Each now reads: us./in./global. for GPT-5.6 Terra and
Luna; us./global. only for GPT-5.6 Sol, GPT-6, GPT-5.5 and GPT-5.4.
tests/test_gpt54_55_cris_prefix_scope.py now also scans the catalog strength
text and rejects any grouped in. grant to GPT-5.6 that does not name
Terra/Luna and place Sol in the us./global.-only set. Verified to fail
against the dfd07bd wording of every file above.
…-project IAM grant, Converse for application profiles, In-Region-only pricing Three inline findings by leon1418 on 6b873f8, each verified against the public GPT-5.4 / GPT-5.5 model cards (read 2026-11-07): https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-54.html https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-55.html 1. IAM default-project grant (iam_policy.py:29). The cards require bedrock:InvokeModel on arn:aws:bedrock:{region}:{account-id}:project/default in addition to the inference profile. iam_policy.py gated that grant on is_gpt6_cris_model(). Added is_gpt54_55_cris_model() (exact us./global. openai.gpt-5.4 / -5.5 short ids and their inference-profile ARNs; bare mantle ids, GPT-5.6 CRIS ids and unrelated models do not match) and requires_default_project_grant(); generate_policy() now emits the third ARN for those ids. Updated the GPT-6-only sentence in gcp generate-ai.md, gcp/azure generate-artifacts-ai.md Step 3, and the report generator §6.4. 2. Application inference profiles are Converse-only (gcp design-ai.md:425). GCP and Azure Design now require required_client_api: "converse" when application inference profiles are the driving need on GPT-5.5 / GPT-5.4 (and any model whose card lists them Converse-only), forbid leaving the field absent, keep the responses default for a region-only move, and record a conflicting Responses-only need (structured outputs, implicit caching) as an explicit decision instead of a silent drop. Added checklist items to the GCP Validation Checklist and the Azure shared schema (required_client_api section and checklist). 3. Unpublished Geo/Global prices (ai-openai-to-bedrock.md:46). The cards publish commercial In-Region prices only. Scoped the verified ~10% premium to the In-Region option in the canonical guide (Tier 0 rows included), both openai-on-bedrock.md fact bases, the report generator and the pricing-cache rows; a CRIS plan needs a separately sourced Geo/Global price or reports the comparison unavailable, and an In-Region proxy is an unverified planning assumption with no verified premium or savings percentage. bedrock_pricing.lookup() already returns available:false for the CRIS ids; pinned with a test (short ids and profile ARNs, no PriceList call). Tests: test_iam_policy.py TestGpt54Gpt55CrisProjectArn (short ids, profile ARNs, bare/GPT-5.6/Anthropic controls, mixed run); test_bedrock_pricing.py test_gpt54_55_cris_ids_are_unpriced_not_inferred_from_in_region; new tests/test_gpt54_55_runtime_review_round3.py text contracts for all three findings. tools/sync-vendored.py re-run (5 copies synced). Validation (this commit, in the worktree): - mise -C plugins/aws-startup-advisor run ci: exit 0 (test 1407 passed + 222 subtests; test:telemetry 311 passed; heroku-source-review 44/44; lint:md 0 errors; vendored-parity PASS 62 files; skill-refs PASS 1482) - llm-to-bedrock: uv run --project scripts python -m pytest: 453 passed (bare mise python3 shows 10 ModuleNotFoundError: httpx in test_astra_execution_contract.py — environment, not the change) - agent-advisor: python3 -m pytest skills/agent-advisor: 570 passed - tests/test_gpt54_55_cris_prefix_scope.py: still green - python3 tools/validate.py: All validations passed (exit 0) - tools/sync-vendored.py --check: PASS - mise run security (gitleaks): 1 pre-existing upstream false positive at f8d09fd (tests/test_bedrock_migration_assessment_discover.py, generic-api-key), unrelated to this PR; nothing new flagged.
leon1418
previously requested changes
Nov 8, 2026
leon1418
left a comment
leon1418
left a comment
Contributor
There was a problem hiding this comment.
[🤖 AI review 🤖]
Reviewed all 76 changed files at 620467c0e07cd686145a90e421dc1eeda4bc9bc0 against 0a218f8cc13220013f72147e2e0fdbe350b34ae2 using OCR delegation, an isolated general reviewer, repository contract tracing, and independent candidate verification. Four P2 corrections remain:
- Propagate Runtime Responses default-project permissions through IAM and both cloud setup generators.
- Preserve each model’s API in mixed-target preflight.
- Block unsupported standalone Mantle Messages execution and make GCP Messages artifact routing exclusive. Completing the deferred execution lane is optional.
- Preserve the resolved API in run-context and define its invalidation behavior.
The inline comments give the affected consumers and acceptance checks. All registered plugin CI components passed locally: 1,408 plugin tests plus 222 subtests, 311 telemetry tests, 44 Node tests, and the configured linters/validators. The separate llm-to-bedrock and agent-advisor suites passed 435 and 570 tests. Offline SDK/dispatch/IAM/context probes found the gaps above; no live AWS inference was performed. git diff --check found one nonblocking extra EOF blank line.
herosjourney
force-pushed
the
fix/llm-to-bedrock-runtime-default
branch
from
November 8, 2026 03:11
620467c to
6ca403b
Compare
leon1418
previously requested changes
Nov 8, 2026
leon1418
left a comment
leon1418
left a comment
Contributor
There was a problem hiding this comment.
[🤖 AI review 🤖]
Verified the four original findings fixed at f3c01556baeb1fb39e845cd8e4c236c59803cd1b. The per-row preflight, default-project grants, deferred Mantle Messages stop, and saved API field all pass the focused checks.
Three P2 findings remain after the complete hybrid review:
- The new IAM report recipe reloads original design IDs instead of applying execution-time model overrides. This is introduced by the fix.
- Retained Responses retrieval lacks project-scoped GetInvoke permission. I missed this in earlier reviews.
- A saved API answer is reused before its design identity is checked. This identity gap also existed in the earlier guidance and was missed.
The inline comments give the bounded corrections and acceptance checks. Validation passed: 622 llm-to-bedrock tests, 570 agent-advisor tests, 1,431 plugin tests plus 222 subtests, 311 telemetry tests, both Node suites, all configured plugin linters/validators, root validation, and git diff --check. Probes were offline; no live AWS inference was performed.
9 of 13 tasks
leon1418
previously approved these changes
Nov 8, 2026
herosjourney
pushed a commit
to herosjourney/agent-toolkit-for-aws
that referenced
this pull request
Nov 8, 2026
…22b / Q10.5) and persist ai_constraints.mantle_requirements PR aws#585 defined the mantle_selector enum but nothing populated it from the user. Add one question block with four sub-questions (asynchronous/background inference; provider-run server-side or pre-configured tools incl. web search; customer-created Projects/Workspaces; strict single-Region processing + Region): - gcp-to-aws clarify-ai.md: Q22b (ESSENTIAL, Gate-2 AI batch, prefilled from workloads[].mantle_signals), Mantle-only signals line in the AI Context Summary, Preferences Output block, question budget 4 -> 5 globals - gcp-to-aws clarify-assemble.md: Q22b in the AI/agentic batch, its "use defaults" bullet and defaults-table row - gcp-to-aws / bedrock-migration-assessment clarify-ai-only.md: Q10.5 between Q10 and Q11 with identical table rows and Interpret line (BMA parity); BMA prefills from the validator summary's mantle_signals_detected; gcp fast path now presents Q10.5 (never dropped); Step 3 field row mantle_requirements - azure-to-aws clarify-ai.md: Q22b Category F bullet, Step 3 rows entry, Who-consumes row; azure clarify-ai-only.md: Q10.5 paragraph, fast path, Step 3 bullet, AI-only example block entry (shows web_search + region) - schema-preferences.md (gcp + BMA): mantle_requirements catalog rows (Q22b (AI-only: Q10.5) / ai_constraints Q10.5) - fixtures/bma-ai-only-clarify/preferences.json: mantle_requirements answer and Q10.5 in questions_asked - tests/test_mantle_requirements_clarify_contract.py: 13 new contract tests Wrapper: ai_constraints.mantle_requirements.value = [{ workload_id, selector, source, evidence?, region? }], selector in background_inference | server_side_tools | web_search | projects | workspaces | in_region_processing | none, source in detected | user | none; chosen_by user or default (source default:<Qid> + report caveat), never extracted. Verification (plugins/aws-startup-advisor): - pytest tests/test_mantle_requirements_clarify_contract.py tests/test_bedrock_migration_assessment_core.py tests/test_validate_artifacts.py tests/test_azure_clarify_fast_path_contracts.py tests/test_estimate_clarify_volume_resolution.py tests/test_live_cli_ai_metrics.py: 243 passed - pytest tests (full plugin suite): 1475 passed, 222 subtests, 1 failed = tests/test_model_id_lint.py::test_real_plugin_passes flagging the untracked skills/llm-to-bedrock/scripts/.venv (pre-existing local env, not a tracked file) - validate-artifacts.py --fixtures: PASS 100 artifacts, 0 errors, baseline unchanged - validate-artifacts.py --run-dir fixtures/bma-ai-only-clarify --skill bedrock-migration-assessment --no-baseline: PASS - markdownlint-cli2: 0 errors on 823 files; sync-vendored --check PASS (64); check-skill-refs --check PASS (1525); tools/validate.py passed; git diff --check clean
GPT-5.4 and GPT-5.5 publish an In-Region price only, so a Geo or Global plan stays unpublished. Co-authored-by: Cursor <cursoragent@cursor.com>
…policy and make Runtime the primary same-model OpenAI path in design guidance Base: 6b873f8 (PR 1 aws#583 tip). Public basis: Amazon Bedrock User Guide endpoints.html, models-endpoint-availability.html, inference-responses-api.html, inference-chat-completions.html, inference-messages-api.html, tool-use-server-side.html, web-search.html, projects.html, workspaces.html, cost-mgmt-application-inference-profiles.html, model-invocation-logging.html, monitoring-mantle-metrics.html, quotas-runtime.html, quotas-mantle.html, scaling-throughput-best-practices.html (all read 2026-11-07). - New canonical skills/shared/ai/bedrock-endpoint-policy.md (rule, mantle_selector enum, public matrix, lifecycle and quota wording), vendored into gcp-to-aws and azure-to-aws (README rows + sync). - ai-openai-to-bedrock.md / ai-openrouter-to-bedrock.md: runtime_openai_cris with the source app's required_client_api is the Tier 0 default; mantle_* only with a mantle_selector; Runtime steps shown first; "smallest change" phrasing removed. - Three schema-design-aws-ai.md copies: runtime_messages, messages, same_model_runtime, mantle_selector vocabulary; Runtime default stated. - gcp/azure design-ai.md, azure design.md/generate.md _assert, gcp/azure generate-artifacts-ai.md, gcp generate-ai.md, gcp estimate.md, azure estimate-assemble.md, both openai-on-bedrock.md Client setup: Runtime first and labelled default, Mantle selector-gated, same_model_runtime route added. - Fixtures: mantle_selector on the two mantle_openai_responses rows. - Tests: new tests/test_endpoint_policy_contract.py; extended test_generate_mantle_dispatch.py, test_ai_only_option_c_framework_route.py, test_gpt6_review_round2.py, test_gpt6_round4_findings.py. Verification (from plugins/aws-startup-advisor): - python3 tools/sync-vendored.py --check -> PASS, 64 files - uv run --no-project --with pytest python -m pytest tests -q -> 1406 passed (base 6b873f8: 1398) - uv run --project skills/llm-to-bedrock/scripts --with pytest python -m pytest skills/llm-to-bedrock/scripts -q -> 418 passed (unchanged) - uv run --project skills/agent-advisor/scripts --with pytest python -m pytest skills/agent-advisor/scripts -q -> 570 passed (unchanged) - mise run lint:md (0 errors / 823 files), lint:skill-refs (1508 refs, 0 missing), lint:artifacts (99/0), lint:vendored-parity (64), lint:model-id -> PASS - grep -rn -i -E 'primary path|smallest (possible )?change|remains the default| Mantle is the PRIMARY' skills | grep -v vendored -> every Mantle-related hit gone; remaining hits are pre-existing, unrelated to Bedrock endpoints (design-ref-harness.md:29, heroku-to-aws, gcp elastic-beanstalk.md, prompt-library bedrock-quota-manager.md, agent-advisor test_unit_grouping.py). agents/llm2bedrock-code-rewriter.md:260 is left for the execution-lane change.
…drock-runtime Messages endpoint in the shared guide and design/generate phases The canonical ai-anthropic-to-bedrock.md now keeps anthropic.Anthropic and repoints it at https://bedrock-runtime.\{region\}.amazonaws.com/anthropic with a string Bedrock token minted per client (provide_token), records migration_path "runtime_messages" + required_client_api "messages", moves the boto3 Converse rewrite under an explicit-alternative section, and gates Mantle Messages behind a mantle_selector. gcp/azure design-ai.md Anthropic lines, gcp/azure generate-artifacts-ai.md Step 0/Step 1, the estimate route lists, skills/shared/ai/README.md, gcp SKILL.md and the behavior-delta Anthropic reference carry the same wording; vendored copies re-synced. Public basis: inference-messages-api.html and inference-profiles-prereq.html (read 2026-11-07). Verification (plugins/aws-startup-advisor): - python3 tools/sync-vendored.py --check: PASS, 64 files byte-identical - uv run --no-project --with pytest python -m pytest tests -q: 1407 passed, 222 subtests - uv run --project skills/llm-to-bedrock/scripts --with pytest python -m pytest skills/llm-to-bedrock/scripts -q: 418 passed - mise run lint:md: 823 files, 0 errors; mise run lint:skill-refs: 1514 checked, 0 missing; mise run lint:vendored-parity: PASS 64 - grep -rn 'Converse client swap' skills agents | grep -v vendored: no matches
…thout an API and add the Runtime Anthropic Messages lane to llm-to-bedrock normalize_api_path now raises for a runtime_openai_cris plan that recorded no source API instead of silently resolving to Converse; SKILL.md resolves that case with the user (resolved_runtime_api) and only an explicit required_client_api "converse" reaches the Converse rewrite. Adds the runtime_messages execution lane: create_messages_client / probe_runtime_messages_model (Anthropic SDK on bedrock-runtime /anthropic with a string token minted per client), --runtime-api messages, image_input.messages_message, the BedrockGetInferenceProfile IAM statement for CRIS targets, the rewriter's Runtime Anthropic Messages lane (Mantle express lane is now selector-gated, Converse rewrite is for explicit converse plans), evaluator §6b/§9.5b and the §10 messages branch, report-generator §6.4 and resolver notes, plus the endpoint policy section in SKILL.md. Pins anthropic>=0.75,<1 in the scripts project. Also un-indents the Converse snippet in the shared Anthropic guide so the agent-advisor snippet test executes the right block. Public sources (read 2026-11-07): https://docs.aws.amazon.com/bedrock/latest/userguide/inference-messages-api.html https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html https://docs.aws.amazon.com/bedrock/latest/userguide/endpoints.html Verification (plugins/aws-startup-advisor): - uv run --project skills/llm-to-bedrock/scripts --with pytest python -m pytest skills/llm-to-bedrock/scripts -q -> 431 passed (418 + 13 new), 0 failed - uv run --project skills/agent-advisor/scripts --with pytest python -m pytest skills/agent-advisor/scripts -q -> 570 passed; git diff --stat 6b873f8..HEAD -- skills/agent-advisor is empty - python3 tools/sync-vendored.py --check -> PASS 64 files - mise run ci -> exit 0 (lint:md 0 errors; lint:skill-refs 1517/0 missing; lint:vendored-parity 64; lint:artifacts 99/0; test 1408 passed + 222 subtests; test:telemetry 311; test:heroku-source-review 44) - root: python3 tools/validate.py -> All validations passed; mise run validate -> exit 0 - root: mise run build -> lint + validate pass; security fails only on the pre-existing gitleaks finding in upstream f8d09fd (tests/test_bedrock_migration_assessment_discover.py:118); not from this change - gitleaks protect --staged -> no leaks found - residual Mantle-default grep over plugins/*.md|*.json -> no hits
… runtime_messages plan and align the Mantle Messages snippet with the public page Review iteration 1 findings (review.json CHANGES_REQUESTED): - major: preflight_bedrock.main() routed every non-Mantle id to the Messages probe when --runtime-api messages was set, so a runtime_messages plan whose $TARGET_MODELS also carried an embedding id (B1 collects every bedrock_models[].aws_model_id) failed preflight spuriously. Embedding ids now fall through to probe_model (InvokeModel) before the messages branch. New test_main_messages_plan_probes_embedding_ids_via_invoke_model covers a Claude profile + amazon.titan-embed-text-v2:0 list. - minor: agents/llm2bedrock-code-rewriter.md Mantle express lane Anthropic row and snippet used base_url .../anthropic/v1 with auth_token=; the SDK appends /v1/messages, so that resolved to /anthropic/v1/v1/messages. Now base_url .../anthropic with api_key=, matching the public Messages API page (inference-messages-api.html, read 2026-11-07) and the shared guide. - minor: evaluator §6b and §9.5b heredocs are now executed by test_runtime_messages_connectivity_snippet_uses_the_messages_client and test_runtime_messages_vision_snippet_sends_a_base64_source_block. - minor: preflight_bedrock.py module docstring now scopes in. to GPT-5.6 Terra and Luna, with GPT-5.6 Sol and GPT-6 Astra us./global. only. - minor: the mantle_selector enum paraphrase is removed from skills/shared/ai/ai-openai-to-bedrock.md and gcp design-ai.md; both now point at references/vendored/ai/bedrock-endpoint-policy.md. The llm-to-bedrock SKILL.md section stays as the one permitted restatement. Vendored copies re-synced (tools/sync-vendored.py). Verification (plugins/aws-startup-advisor, 2026-11-07): - sync-vendored --check: PASS 64 (2 re-synced) - llm-to-bedrock scripts: 435 passed (431 + 4 new) - agent-advisor scripts: 570 passed; no non-vendored agent-advisor change - mise run ci: exit 0 (lint:md 0 errors, lint:skill-refs 1520/0 missing, lint:vendored-parity PASS 64, lint:artifacts 99/0, test 1408 passed + 222 subtests, test:telemetry 311, test:heroku-source-review OK) - root python3 tools/validate.py: All validations passed; mise run validate: exit 0 - residual Mantle-default grep over plugins/aws-startup-advisor *.md/*.json: nothing - gitleaks protect --staged: no leaks found
…ject grant by id form/API, per-row preflight APIs, mantle_messages lane gate, resolved_runtime_api in resume context Re-checked each leon1418 finding against 6ca403b (post-rebase onto 8c37ef0): 1. Default-project grant (ai-openai-to-bedrock.md:339). Partly satisfied by the rebase: 8c37ef0 grants project/default for GPT-6 and the GPT-5.4/5.5 CRIS ids, but GPT-5.6 Sol/Terra/Luna were still excluded and the grant was model-family-based. The public Responses API page (https://docs.aws.amazon.com/bedrock/latest/userguide/inference-responses-api.html, "Permissions", read 2026-11-07) authorizes bedrock:InvokeModel on the default project for every Responses request on bedrock-runtime, and the GPT-5.6 Sol/Terra/Luna cards (read 2026-11-07) repeat it for their runtime CRIS ids (Chat tab included). iam_policy.py now classifies by id form (is_runtime_openai_cris_model: us./in./us-gov./global. openai.gpt-* ids and their inference-profile ARNs; _GEO_PREFIX gains in. and us-gov.) and decides the grant by the selected API (--runtime-api responses|chat_completions emit it, converse/messages do not, id-only keeps it). GCP generate-ai.md / generate-artifacts-ai.md, Azure generate-artifacts-ai.md, both openai-on-bedrock.md fact bases, the shared guide (+vendored copies) and the report-generator §6.4 recipe now state the rule and cite the page. 2. Mixed-model preflight (preflight_bedrock.py:417). Confirmed open: one --runtime-api was applied to every id. preflight gains --target ID=API pairs (per-row required_client_api; invoke for embeddings) and fails closed with reason unknown_runtime_api before any probe when a row's API is missing or unknown; --models/--runtime-api stays as the legacy single-API form. B1/B4 pass each row as its own pair. 3. mantle_messages stop (SKILL.md:936). Confirmed open. New lane gate `preflight_bedrock.py --check-lanes aws-design-ai.json` (unsupported_lane_stop) halts before B2/B4/Phase C with a structured unsupported_lane result naming the deferred Mantle Messages lane and the workspaces selector; iam_policy --migration-path mantle_messages refuses to emit runtime permissions for the bare Claude id; the context rule no longer labels it converse. GCP generate-artifacts-ai.md Step 0: the broad Mantle branch is scoped to the three OpenAI-SDK spellings so mantle_messages matches exactly one branch and runtime_messages keeps Messages. The lane is not implemented. 4. resolved_runtime_api (SKILL.md:913). Confirmed open. The field is now in the C0 current-context field list (always present), resolved in B4 before preflight (read back from run-context.json on resume), and has an invalidation row (EVAL, REWRITE, REPORT archived; ANALYSIS, INGESTION kept). Rebase-review non-blocking items: reworded the "keeps the `responses` default" checklist sentences (gcp design-ai.md, azure shared schema-design-aws-ai.md) to the source-API default rule; removed the EOF blank line in tests/test_ai_only_option_c_framework_route.py. Tests: test_iam_policy.py (id-form x API matrix incl. in./us-gov., Converse-only and Anthropic/oss controls, CLI flags, mantle_messages refusal), test_preflight_bedrock.py (offline fake SDK clients: Claude/Messages + GPT/Chat, Claude/Messages + Nova/Converse + embeddings, fail-closed unknown API, --check-lanes), test_validate_result.py (actual comparator: unchanged resume matches, dropped field reproduces the reviewer's MISMATCH, changed API named), tests/test_runtime_default_review_round1.py (text contracts + comparator). Existing GPT-5.6 "no grant" assertions in test_iam_policy.py were flipped to match the 2026-11-07 card read. Validation (worktree fix-llm-to-bedrock-runtime-default): - mise -C plugins/aws-startup-advisor run ci: EXIT 0 (lint:md 0 errors; vendored-parity 64 PASS; skill-refs 1521 PASS; artifacts 99 PASS; test 1429 passed + 222 subtests; test:telemetry 311 passed; heroku 44 pass) - llm-to-bedrock scripts pytest: 599 passed - agent-advisor scripts pytest: 570 passed - python3 tools/validate.py: All validations passed - tools/sync-vendored.py --check: PASS 64 byte-identical - git diff --check: clean - mise run build (gitleaks): fails ONLY on pre-existing upstream f8d09fd (tests/test_bedrock_migration_assessment_discover.py:118 generic-api-key, leaks found: 1) — not introduced by this branch
…s[] row in iam_policy (--target ID=API) so a GPT secondary under a Claude Messages primary keeps it Review-gate finding B1 (pr585-review-round-1): the §6.4 report-generator recipe passed the primary's single `Target API surface` as --runtime-api for every id, and generate_policy applied it to all ids, so a runtime_messages plan with a GPT Chat/Responses secondary emitted no arn:...:project/default for that secondary (a regression against 6ca403b's id-only call). - iam_policy.py: new repeatable `--target <aws_model_id>=<required_client_api>` form (parse_targets, fails closed with reason unknown_runtime_api before any policy is built) and a `target_apis` kwarg on generate_policy; each row's own API decides requires_default_project_grant for that id, overriding the plan-wide runtime_api. `--models --runtime-api` stays as the legacy single-API form; --models/--target are mutually exclusive; the mantle_messages lane stop still wins over every other check. - llm2bedrock-report-generator.md §6.4: recipe passes one --target pair per bedrock_models[] row read from aws-design-ai.json (primary row = Target API surface; `invoke` for embedding rows), explicitly forbids applying the primary's surface to other rows, reports unknown_runtime_api verbatim, and documents the legacy form as single-row only. - Shared guide IAM step, GCP generate-ai.md / generate-artifacts-ai.md and Azure generate-artifacts-ai.md recipes name the per-row form; both vendored copies of the shared guide re-synced (byte-identical). - Tests: test_iam_policy.py TestPerRowTargetApis (reviewer's spot-check as the regression; Claude/messages + GPT/chat -> project/default exactly once; Claude/messages + Nova/converse + embeddings/invoke -> absent; GPT chat row beside GPT converse row -> once; row API overrides plan-wide; fail-closed parse; CLI --target grant / unknown API exit 1 / mutual exclusion). tests/test_runtime_default_review_round1.py: text contract pinning the §6.4 recipe (--target per row, no --runtime-api in the command, per-row warning, unknown_runtime_api) and the four setup recipes; mixed plan driven through parse_targets + generate_policy. Validation (run in this worktree, mise python 3.12): - mise -C plugins/aws-startup-advisor run ci: exit 0 -- test 1431 passed + 222 subtests; test:telemetry 311 passed; vendored parity 64 PASS - uv run --project skills/llm-to-bedrock/scripts pytest skills/llm-to-bedrock: 614 passed (bare mise python without the uv project: 603 passed, 11 failed on ModuleNotFoundError httpx in test_astra_execution_contract -- environment, not code) - pytest skills/agent-advisor: 570 passed (no agent-advisor files touched) - pytest tests/test_gpt54_55_cris_prefix_scope.py: 3 passed - python3 tools/validate.py: All validations passed - tools/sync-vendored.py --check: PASS, 64 byte-identical - git diff --check: clean - mise run build: gitleaks fails only on pre-existing upstream commit f8d09fd (tests/test_bedrock_migration_assessment_discover.py:118 generic-api-key); unrelated to this branch
… one Bedrock id sits in several --target rows PR 585 review round 1, pass 2, blocker B2: `target_apis = dict(targets)` kept only the LAST row's API per id, so `--target us.openai.gpt-5.6-sol=chat_completions --target us.openai.gpt-5.6-sol=converse` emitted 0 project/default while the reverse order emitted 1. The §6.4 recipe emits one pair per bedrock_models[] row without deduplicating, and the schema lets a GPT CRIS id appear on a converse row (Guardrails / application-inference-profile call site) and a chat_completions/responses row for a second source model. - iam_policy.py: new pure `target_apis_by_id(pairs) -> dict[str, set[str]]` groups every API an id is called on; `generate_policy(target_apis=)` accepts a str or a set per id, validates each API in the set (fail closed on an unknown one), and grants `project/default` once when ANY of the id's rows is responses/chat_completions. CLI `--target` now builds the set instead of `dict(targets)`. - test_iam_policy.py::TestPerRowTargetApis: [gpt=chat,gpt=converse] and the reverse order and [gpt=converse,gpt=responses] all grant exactly once; [gpt=converse, gpt=converse] grants none; target_apis_by_id keeps every row's API; an unknown API inside a row set fails closed; CLI duplicate-id rows grant once in both orders (the reviewer's spot-check). - llm2bedrock-report-generator.md §6.4: tell the agent not to deduplicate rows and that a duplicated id is granted once if any row is responses/chat_completions. Validation (run in this worktree, 2026-11-07): - mise -C plugins/aws-startup-advisor run ci: exit 0 — [test] 1431 passed, 222 subtests passed; [test:telemetry] 311 passed; lint:md 0 errors; vendored-parity PASS 64; artifacts PASS 99; skill-refs PASS 1521. (A stale gitignored skills/llm-to-bedrock/scripts/.venv from the previous step broke lint:md on the first run; it was removed, not committed.) - pytest skills/llm-to-bedrock (uv project env): 622 passed (was 614; +8 new). - pytest skills/agent-advisor: 570 passed (no agent-advisor files touched). - pytest tests/test_gpt54_55_cris_prefix_scope.py: 3 passed. - python3 tools/validate.py: All validations passed. - tools/sync-vendored.py --check: PASS, 64 byte-identical. - git diff --check: clean. - mise run build: exit 1 ONLY on gitleaks' pre-existing upstream finding at f8d09fd (tests/test_bedrock_migration_assessment_discover.py:118, generic-api-key); not introduced by this branch. - CLI spot-check: sol=chat+converse -> 1 project/default; converse+chat -> 1; converse+converse -> 0.
…ds in the §6.4 IAM recipe, project-scoped GetInvoke for retained stored-response ops, identity-checked saved-API reuse in B4 Three P2 findings from leon1418 on f3c0155 (threads 4505069045, 4505069047, 4505069050). 1. Report §6.4 IAM recipe reloaded aws-design-ai.json's ORIGINAL ids, so a C4 target change (Sol → Terra, recorded in resolved_model_overrides) emitted Sol permissions and omitted the Terra profile. The recipe now joins each bedrock_models[] row's required_client_api to the validated analysis target_models (overrides applied) and never falls back to the row's aws_model_id; iam_policy.validated_target_pairs() does the join and fails closed on a row without a validated entry or an API. Regression: a C4 Sol → Terra change yields a C6 policy naming Terra (+ project/default) and no Sol resource; the pre-fix design-row path is kept as the failing control. 2. Retained client.responses.retrieve(id) / cancel / delete on bedrock-runtime authorize bedrock:GetInvoke / CancelInvoke / DeleteInvoke on project/default, separately from the InvokeModel creation needs (inference-responses-api.html "Permissions", read 2026-11-08). analysis.json gains optional stored_response_ops (enum retrieve|cancel|delete); the analyzer §8.3 scans for the call sites; iam_policy.py gains --stored-response-ops and emits a BedrockStoredResponseOps statement with ONLY the retained actions on project/default, and only when a --target row is a runtime GPT CRIS id on `responses` — Converse, Anthropic Messages, Chat Completions, embeddings, bare mantle ids and id-only legacy calls are refused (reason stored_response_ops_no_responses_row). The report recipe passes the field through, the How to Apply checklist gains a create-plus-retrieve permission check, and the GCP / Azure setup_bedrock contracts, GCP generate-ai and the shared OpenAI guide (vendored copies re-synced) name the grant, its controls and the create-plus-retrieve verification. 3. SKILL.md B4 read the saved resolved_runtime_api before C0's assess_design_sha256 check, so a cached `messages` answer could send a new GPT row to the Anthropic endpoint. B4 now runs preflight_bedrock.py --resume-runtime-api <run-context> --design <aws-design-ai.json>, which reuses the saved answer only when it exists AND the saved assess_design_sha256 and target_models (source, aws_model_id) identity match the current design; any mismatch, missing context or missing field forces re-resolution before any probe. The context-block recovery sentence points at the same gate. Validation (worktree fix-llm-to-bedrock-runtime-default): - mise -C plugins/aws-startup-advisor run ci: exit 0 — test 1441 passed (+222 subtests), test:telemetry 311 passed, lint:md 0 errors, vendored-parity PASS (64), skill-refs PASS, model-id OK, artifacts PASS, consent-disclosure OK, model-catalog-drift OK (3 pre-existing warns) - llm-to-bedrock scripts pytest (uv run --project .): 655 passed (622 baseline + 33 new) - agent-advisor scripts pytest: 570 passed (unchanged; no agent-advisor edits) - tests/test_gpt54_55_cris_prefix_scope.py and test_runtime_default_review_round1.py green - python3 tools/validate.py: All validations passed - tools/sync-vendored.py --check: PASS - git diff --check: clean - mise run build: exit 1 ONLY on the pre-existing gitleaks finding at upstream f8d09fd (tests/test_bedrock_migration_assessment_discover.py:118, generic-api-key on the string "claude-3.5-sonnet"); lint:manifests passed. Not introduced or touched by this commit.
…nd record them as mantle_signals Discover now records the five code-detectable bedrock-mantle selectors (background_inference, web_search, server_side_tools, projects, workspaces) with file:line evidence so Clarify can confirm them instead of guessing the endpoint from source code alone. mantle_only_model_or_api stays Design-derived and in_region_processing is never detected (always asked). - skills/shared/ai/bedrock-endpoint-policy.md: new "## How a selector is learned" section (positive/negative constructs per selector, cites inference-responses-api.html read 2026-11-07); vendored copies re-synced (gcp-to-aws, azure-to-aws references/vendored/ai). - gcp-to-aws discover-app-code.md: Step 5C "Mantle-Only Selector Signals" (table + workloads[].mantle_signals[] output), Step 5B example, checklist. - azure-to-aws discover-app-code.md: Step 5C (bullet style), checklist. - gcp/azure schema-discover-ai.md: workloads[].mantle_signals[] shape, CRITICAL field bullet / enum, validation rule. - discover-openai-api.md / discover-anthropic-api.md / discover-openrouter-api.md (gcp + azure): Step 4 merge adds a projects / workspaces / web_search (:online) signal with the <usage_api>/line 0 sentinel; OpenRouter workspace_id is explicitly NOT a workspaces signal. - bedrock-migration-assessment discover-app-code.md: Step 4.5 with top-level mantle_signals[] output; Step 5 example and closing rule allow the key. - scripts/contracts/ai-workload-profile.schema.json (+ byte-identical BMA copy): optional root mantle_signals[] and mantleSignal definition. - validate-ai-profile.py: _check_mantle_signals (unmatched_mantle_signal) and mantle_signals_detected in the summary. - fixtures/bma-scan-openai-mantle-signals: positives on app/llm.py:5,9,14,21; negatives (function tool named web_search, store=True) on lines 30 and 37. - tests/test_mantle_selector_discovery.py (15 tests); test_bedrock_migration_assessment_core.py summary key set updated. Verification (plugins/aws-startup-advisor): - python3 tools/sync-vendored.py && --check: PASS (64 files, 2 synced) - python3 scripts/validate-artifacts.py --fixtures: PASS, 100 artifacts, 0 errors, no new baseline entries - python3 tools/check-skill-refs.py --check: PASS (1525 refs) - markdownlint-cli2 (823 tracked files): 0 errors - uv run --no-project --with pytest python -m pytest tests -q: 1456 passed, 222 subtests passed, 1 failed — the one failure is test_model_id_lint::test_real_plugin_passes flagging a retired model id inside the gitignored, untracked skills/llm-to-bedrock/scripts/.venv/ (anthropic SDK constants); pre-existing local environment, not a tracked file. Acceptance subset (6 files): 253 passed. - python3 tools/validate.py (repo root): All validations passed - git diff --check: clean
…22b / Q10.5) and persist ai_constraints.mantle_requirements PR aws#585 defined the mantle_selector enum but nothing populated it from the user. Add one question block with four sub-questions (asynchronous/background inference; provider-run server-side or pre-configured tools incl. web search; customer-created Projects/Workspaces; strict single-Region processing + Region): - gcp-to-aws clarify-ai.md: Q22b (ESSENTIAL, Gate-2 AI batch, prefilled from workloads[].mantle_signals), Mantle-only signals line in the AI Context Summary, Preferences Output block, question budget 4 -> 5 globals - gcp-to-aws clarify-assemble.md: Q22b in the AI/agentic batch, its "use defaults" bullet and defaults-table row - gcp-to-aws / bedrock-migration-assessment clarify-ai-only.md: Q10.5 between Q10 and Q11 with identical table rows and Interpret line (BMA parity); BMA prefills from the validator summary's mantle_signals_detected; gcp fast path now presents Q10.5 (never dropped); Step 3 field row mantle_requirements - azure-to-aws clarify-ai.md: Q22b Category F bullet, Step 3 rows entry, Who-consumes row; azure clarify-ai-only.md: Q10.5 paragraph, fast path, Step 3 bullet, AI-only example block entry (shows web_search + region) - schema-preferences.md (gcp + BMA): mantle_requirements catalog rows (Q22b (AI-only: Q10.5) / ai_constraints Q10.5) - fixtures/bma-ai-only-clarify/preferences.json: mantle_requirements answer and Q10.5 in questions_asked - tests/test_mantle_requirements_clarify_contract.py: 13 new contract tests Wrapper: ai_constraints.mantle_requirements.value = [{ workload_id, selector, source, evidence?, region? }], selector in background_inference | server_side_tools | web_search | projects | workspaces | in_region_processing | none, source in detected | user | none; chosen_by user or default (source default:<Qid> + report caveat), never extracted. Verification (plugins/aws-startup-advisor): - pytest tests/test_mantle_requirements_clarify_contract.py tests/test_bedrock_migration_assessment_core.py tests/test_validate_artifacts.py tests/test_azure_clarify_fast_path_contracts.py tests/test_estimate_clarify_volume_resolution.py tests/test_live_cli_ai_metrics.py: 243 passed - pytest tests (full plugin suite): 1475 passed, 222 subtests, 1 failed = tests/test_model_id_lint.py::test_real_plugin_passes flagging the untracked skills/llm-to-bedrock/scripts/.venv (pre-existing local env, not a tracked file) - validate-artifacts.py --fixtures: PASS 100 artifacts, 0 errors, baseline unchanged - validate-artifacts.py --run-dir fixtures/bma-ai-only-clarify --skill bedrock-migration-assessment --no-baseline: PASS - markdownlint-cli2: 0 errors on 823 files; sync-vendored --check PASS (64); check-skill-refs --check PASS (1525); tools/validate.py passed; git diff --check clean
….mantle_requirements in Design and disclose the bedrock-runtime assumption
Design no longer infers the Bedrock endpoint from the source API, the model or a
guess: gcp-to-aws and azure-to-aws design-ai.md gain Step 0.8, which reads
preferences.json -> ai_constraints.mantle_requirements (Clarify Q22b / Q10.5),
derives bedrock_models[].mantle_selector only from a detected|user entry (plus
the catalog for mantle_only_model_or_api), records the cited entry as
bedrock_models[].mantle_requirement on every mantle_* row, and writes the
verbatim endpoint_assumption sentence ("No Mantle-only requirement was recorded
for <workload_id>; bedrock-runtime selected. ... re-run Clarify.") on every
runtime row. Generate echoes whichever field is present in the script/adapter
header; the llm-to-bedrock report renders an Endpoint line from a new
"Endpoint basis:" context line and cites the public Responses API page for the
Runtime background=true 400.
Design: skills/gcp-to-aws/references/phases/design/design-ai.md (Step 0.8,
Part 5, Part 6 table, Validation Checklist, Present Summary item 6)
skills/azure-to-aws/references/phases/design/design-ai.md (Step 0.8,
Parts 3/5, Completion gate, Present summary)
Schema: skills/shared/ai/schema-design-aws-ai.md (bedrock_models[] bullet,
new "mantle_requirement and endpoint_assumption" section, 2 checklist
items) + vendored copies in gcp-to-aws/azure-to-aws references/vendored/ai
skills/gcp-to-aws/references/shared/schema-design-aws-ai.md
skills/azure-to-aws/references/shared/schema-design-aws-ai.md
Generate: skills/gcp-to-aws/references/phases/generate/generate-artifacts-ai.md
skills/gcp-to-aws/references/phases/generate/generate-ai.md
skills/azure-to-aws/references/phases/generate/generate-artifacts-ai.md
Report: skills/llm-to-bedrock/SKILL.md (Endpoint policy paragraph, B1 tuple,
Endpoint basis: context line)
agents/llm2bedrock-report-generator.md (Executive Summary Endpoint
line, Model Mapping notes, Limitations)
Fixtures: fixtures/aws-design-ai/multi-provider-direct (detected web_search
Mantle row + runtime endpoint_assumption), openrouter-gateway-multi-
upstream (in_region_processing mantle_requirement + 2 assumptions),
new runtime-default-no-requirement (runtime-only design)
Tests: tests/test_mantle_requirements_design_contract.py (17 tests)
Verification (all run on this commit's tree):
- python3 tools/sync-vendored.py && --check: PASS, 64 files (2 re-synced)
- python3 scripts/validate-artifacts.py --fixtures: PASS, 101 artifacts, 0 errors,
baseline unchanged (MISSING_REQUIRED=6, UNKNOWN_KEY=44)
- mise -C plugins/aws-startup-advisor run ci: exit 0 — lint:md 0 errors/823
files, lint:artifacts PASS, lint:vendored-parity PASS, lint:skill-refs PASS
(1532 refs), model-id/staleness/catalog-drift/pricing/consent PASS,
test 1496 passed + 222 subtests, test:telemetry 311 passed,
test:heroku-source-review PASS
- skills/llm-to-bedrock/scripts: 655 passed (unchanged)
- skills/agent-advisor/scripts: 570 passed (no file under agent-advisor changed)
- python3 tools/validate.py: all validations passed; git diff --check clean
…cover, Clarify and Design Cross-FEAT integration pass over a6f9ef1 (mantle_signals), 73835b1 (ai_constraints.mantle_requirements) and 922687e (mantle_requirement / endpoint_assumption): - endpoint_assumption sentence: the shared schema-design-aws-ai.md and the gcp design-ai.md Step 0.8 rendered the placeholder as `<workload_id>` in backticks while the azure docs, the aws-design-ai fixtures and tests/test_mantle_requirements_design_contract.py expect a plain id. The placeholder is now unquoted in both and the "where <workload_id> is" note says the id is written plain (no backticks or quotes). - gcp clarify-ai.md Q22b Preferences Output example: the `evidence` of the detected `projects` entry now copies the Step 5B/5C construct verbatim (src/ai/openai_client.py:9 default_headers={"OpenAI-Project": ...}) so the example matches the "copy the signal's file:line construct" rule. - gcp and azure discover-app-code.md output checklists: the "real line that contains the construct" rule now names the `"<usage_api>"` / `line: 0` sentinel that the usage-API Step 4 merges append, matching schema-discover-ai.md. - bedrock-endpoint-policy.md "How a selector is learned": names the Clarify question ids (Q22b / Q10.5) and the Design outputs (mantle_requirement / endpoint_assumption) so the policy describes the whole chain. - Vendored copies re-synced (gcp-to-aws and azure-to-aws references/vendored/ai/bedrock-endpoint-policy.md and schema-design-aws-ai.md). Verification (worktree fix-llm-to-bedrock-runtime-default): - mise -C plugins/aws-startup-advisor run ci: exit 0 (lint:md 0 errors, lint:artifacts PASS 101 artifacts 0 errors, lint:vendored-parity PASS 64, lint:skill-refs PASS 1532, model-id / staleness / catalog-drift / startup-model-pricing / consent-disclosure PASS, test 1496 passed + 222 subtests, test:telemetry 311 passed, test:heroku-source-review PASS). The gitignored local skills/*/scripts/.venv dirs were parked outside the tree for the run (markdownlint and model-id-lint walk into them) and restored. - python3 tools/sync-vendored.py --check: PASS 64 - python3 scripts/validate-artifacts.py --fixtures: PASS 101 artifacts, 0 errors - skills/llm-to-bedrock/scripts: 655 passed - skills/agent-advisor/scripts: 570 passed (no file under agent-advisor changed) - python3 tools/validate.py: All validations passed - git diff --check: clean - mise run build (gitleaks) not run: it fails on pre-existing upstream commit f8d09fd, unrelated to this branch.
… a source with no Mantle path; widen the Anthropic and report-fallback wording Review round (verdict.json CHANGES_REQUESTED, 1 major + 2 minor): - major — Step 0.8 (gcp and azure design-ai.md) and the derivation section of skills/shared/ai/schema-design-aws-ai.md now define the outcome for a matched ai_constraints.mantle_requirements entry on a workload whose source API has no Mantle path (Gemini / Vertex AI via google-genai, or a workload whose target is a cross-family direct / converse / gpt-oss row): the row keeps bedrock-runtime and the verbatim endpoint_assumption sentence, and discloses the unhonored selector with its source and evidence in rationale and capability_gaps[] (a regional_warnings[] entry for in_region_processing); a Mantle path is never invented and the answer is never dropped. The gcp and azure shadow schema-design-aws-ai.md docs carry the same clause; the Present Summary endpoint line (gcp item 6, azure summary) names the gap. The Clarify copy (gcp Q22b, gcp/BMA Q10.5 identical sentence, azure Q22b bullet and Q10.5 paragraph) tells the user up front that a "yes" for such a workload records a capability gap rather than an endpoint change. tests/test_mantle_requirements_design_contract.py now asserts the Step 0.8 slice names the no-Mantle-path case and capability_gaps. - minor — gcp design-ai.md Step 0 Anthropic bullet and Part 5 Anthropic row, and azure design-ai.md Part 5: "(a `workspaces` entry)" -> "(any matched entry, typically `workspaces`)", matching Step 0.8 and the scanners (Anthropic web_search_20250305 -> web_search, code_execution_* -> server_side_tools). - minor — llm-to-bedrock SKILL.md emits the C6 "Endpoint basis:" line whenever any bedrock_models[] row carries migration_path mantle_*, mantle_selector, mantle_requirement or endpoint_assumption, rendering a Mantle row without mantle_requirement as "bedrock-mantle (<selector or 'selector unrecorded'>; source unknown — plan predates mantle_requirements)"; B1 keeps migration_path / mantle_selector for that purpose. The report generator's Endpoint line renders that form and falls back to the runtime wording only when the line is absent (no row carries any marker). - Vendored copies re-synced (gcp-to-aws and azure-to-aws references/vendored/ai/schema-design-aws-ai.md). Verification (worktree fix-llm-to-bedrock-runtime-default): - mise -C plugins/aws-startup-advisor run ci: exit 0 (lint:md 0 errors, lint:artifacts PASS 101 artifacts 0 errors, lint:vendored-parity PASS 64, lint:skill-refs PASS 1532, model-id / staleness / catalog-drift / startup-model-pricing / consent-disclosure PASS, test 1496 passed + 222 subtests, test:telemetry 311 passed, test:heroku-source-review PASS). The gitignored local skills/*/scripts/.venv dirs were parked outside the tree for the run and restored. - python3 tools/sync-vendored.py --check: PASS 64 - python3 scripts/validate-artifacts.py --fixtures: PASS 101 artifacts, 0 errors - skills/llm-to-bedrock/scripts: 655 passed - skills/agent-advisor/scripts: 570 passed (no file under agent-advisor changed) - python3 tools/validate.py: All validations passed - git diff --check: clean - mise run build (gitleaks) not run: it fails on pre-existing upstream commit f8d09fd, unrelated to this branch.
…edrock endpoints The GPT-5.4 and GPT-5.5 cards publish an In-Region rate only, so a missing Geo or Global row stays unpublished. Co-authored-by: Cursor <cursoragent@cursor.com>
added 5 commits
November 9, 2026 09:08
…s by endpoint and emits Web Search grants for a web_search row (PR 588 F12, F15)
F12 — `generate_policy` now splits `responses` rows by endpoint: a runtime GPT CRIS row
keeps `bedrock:GetInvoke` / `CancelInvoke` / `DeleteInvoke` on `project/default`; a bare
Mantle id on `responses` is covered by the `BedrockMantleInference` statement (Mantle
inference, stored-response retrieval at `GET /v1/responses/{id}` included, authorizes
`bedrock-mantle:CreateInference`) and no longer raises
`stored_response_ops_no_responses_row`. Converse / Messages / Chat / embedding / id-only
plans still fail closed. Public basis: inference-responses-api.html "Permissions",
read 2026-11-09.
F15 — new `--mantle-selector` (repeatable; `parse_mantle_selectors` fails closed with
reason `unknown_mantle_selector`) and `--external-web-access`. A `web_search` selector with
a Mantle row appends `BedrockWebSearch`: `bedrock-websearch:InvokeSearch` + `InvokeFetch`
on `*` with an `aws:RequestedRegion` condition (the service's only condition key; all three
actions take `*`), plus `bedrock-websearch:ExternalWebAccess` only when the user confirmed
external access. `web_search` without a Mantle row → `web_search_without_mantle_row`;
`--external-web-access` without `web_search` → `external_web_access_without_web_search`.
Other selectors add nothing. Public basis: web-search.html, security-web-search.html,
read 2026-11-09 (`external_web_access` defaults to true; without the action every Fetch
fails authorization while the request still returns 200).
Recipes: report §6.4 command block gains both flags; the stored-response paragraph keeps
its pinned sentences and adds the Mantle dispatch; new "Web Search needs its own grants"
paragraph; How-to-Apply gains the Mantle create-plus-retrieve variant and a conditional
"Tool-level permission check" bullet. GCP / Azure generate-artifacts-ai.md and the canonical
OpenAI guide (+ vendored copies via sync-vendored) carry the same facts and a Step-5
tool-level check.
Tests: test_iam_policy_round2.py — TestStoredResponseOps replaces the bare-Mantle raise test
with all-Mantle retained-retrieve (Mantle policy, no GetInvoke, legacy form too), runtime
control, mixed runtime+Mantle (GetInvoke once + Mantle sids), Mantle Chat-only still fails
closed, CLI all-Mantle exits 0; new TestWebSearchGrants (grant shape, external action,
parametrized absence, runtime-only raise, unknown selector, CLI emit / refuse / reasons).
New tests/test_runtime_default_review_round3.py pins the recipe strings.
Validation: skills/llm-to-bedrock/scripts `uv run --group dev pytest -q` 672 passed
(was 655); plugin `pytest tests -q` 1532 passed, 1 failed (known
test_model_id_lint::test_real_plugin_passes local .venv noise, pre-existing);
tools/sync-vendored.py --check PASS (64); markdownlint-cli2 0 issues.
…nt tools are not server_side_tools signals (PR 588 F13) The Sonnet 5 and Opus 5.5 model cards list `computer_20251124`, `computer_toolset_20260801`, `browser_toolset_20260801`, `bash_20250124`, `text_editor_20250728` and `memory_20250818` under "Computer use" and "Client tools and toolsets using `bedrock-runtime` and `bedrock-mantle` endpoints" (model-card-anthropic-claude-sonnet-5.html, model-card-anthropic-claude-opus-5-5.html, read 2026-11-09): they are client-executed and available on both endpoints, so they must not select Mantle. bedrock-endpoint-policy.md (canonical + 2 vendored mirrors via sync-vendored) and the gcp / azure / bedrock-migration-assessment discover-app-code.md scanners: the `server_side_tools` positive list keeps `"type": "mcp"`, `code_interpreter`, `file_search`, `computer_use_preview`, `image_generation` and Anthropic `code_execution_*` (provider-executed); `computer_*` / `text_editor_*` / `bash_*` (+ toolsets, `memory_*`) move to "Not a signal" with the card citation; both cards added to § Sources. New fixture fixtures/bma-scan-anthropic-client-tools (Anthropic SDK `claude-sonnet-5` with computer / text_editor / bash tools, profile with no `mantle_signals`). Tests (tests/test_mantle_selector_discovery.py): test_policy_and_scanners_keep_anthropic_client_tools_out_of_server_side_tools and test_client_tool_only_anthropic_fixture_has_no_mantle_signal_and_reaches_runtime_messages (validator ok, `mantle_signals_detected == []`, and iam_policy on `runtime_messages` emits `BedrockInvokeModelScoped` — the lane reaches IAM/preflight, not unsupported_lane). Validation: plugin `pytest tests -q` 1532 passed, 1 failed (known test_model_id_lint::test_real_plugin_passes .venv noise); scripts/validate-artifacts.py --fixtures PASS (103); tools/sync-vendored.py --check PASS (64).
…carry the per-model Guardrails API rule (PR 588 F3, Azure half) The blanket "`converse` / `chat_completions` / `invoke` (never `responses`) for 6.1 Sol and the other GPT-6/5.6 CRIS models" in skills/azure-to-aws design-ai.md (Part 2 Guardrails block) and references/shared/schema-design-aws-ai.md (§ required_client_api) is replaced with the per-model rule GCP already carries on aws#588: `converse` for GPT-6 Astra, GPT-5.6 Sol / Terra / Luna, GPT-5.5, GPT-5.4 and GPT-6.1 Terra (Converse-only); `responses` / `chat_completions` / `invoke` / `converse` for GPT-6 Sol and GPT-6 Luna; `chat_completions` / `invoke` / `converse` (never `responses`) for GPT-6.1 Sol; the nine card URLs are listed and "Do not leave the client on `responses` when Guardrails is required on a model whose card excludes Responses." closes the block. agents/llm2bedrock-code-rewriter.md: "note Guardrails are Converse-only and prompt caching Responses-only on runtime" becomes the per-card scope carried by `bedrock_models[].required_client_api` (GPT-6 Sol / Luna on Responses, Chat Completions, InvokeModel and Converse; GPT-6.1 Sol not on Responses; GPT-5.6 / 5.5 / 5.4 / Astra Converse only; implicit caching Responses-only on runtime for GPT-5.5 / 5.4, broader for 5.6 / 6). Public basis, read 2026-11-09: model-card-openai-gpt-6-sol.html "Guardrails (Responses, Chat Completions, InvokeModel, and Converse APIs)"; -6-luna same; -6-1-sol "Chat Completions, InvokeModel, and Converse APIs; not supported with Responses"; -6-astra, -56-sol/-terra/-luna, -55, -54 "Converse API only". The canonical and GCP shadow schema docs contain no blanket wording (grep verified); the GCP design block is replaced by aws#588's ea0798b when it rebases. Tests (tests/test_runtime_default_review_round3.py): blanket strings absent in Azure design, Azure/canonical/GCP schema docs and the rewriter; per-model lines + nine cards present in the Azure design and schema; rewriter sentence pinned. Validation: plugin `pytest tests -q` 1532 passed, 1 failed (known model-id-lint .venv noise); markdownlint-cli2 0 issues.
…st its required API; Responses-only selectors on a Chat/Messages source are an explicit decision (PR 588 F4, cloud half) GCP and Azure design-ai.md Step 0.8 gain rule 1b between rules 1 and 2, and rule 2 now reads "Any match that passes 1b": `web_search` and `background_inference` require the Responses API (web-search.html; inference-responses-api.html, read 2026-11-09); `workspaces` requires the Anthropic Messages API (`anthropic-workspace-id`); `projects` requires an OpenAI-compatible API (`OpenAI-Project`). A provider mismatch (`workspaces` on an OpenAI SDK workload, `projects` on an Anthropic SDK workload) keeps `bedrock-runtime`, records `endpoint_assumption`, discloses it in `rationale` / `capability_gaps[]` and re-runs Clarify. A Chat Completions or Anthropic Messages source matched by a Responses-only selector is an explicit API-continuity decision recorded in the plan and the Present Summary — keep the API on `bedrock-runtime` and drop the requirement (capability gap), or move to `mantle_openai_responses` with the Chat → Responses reshape — never a silent `mantle_openai_chat` / `mantle_messages` row. Same rule in prose in the canonical schema-design-aws-ai.md (+ 2 vendored copies via sync-vendored) and the GCP / Azure shadow schema docs. New fixture fixtures/aws-design-ai/chat-source-web-search-continuity-kept: GPT-5.5 Chat Completions source, `runtime_openai_cris` / `chat_completions`, verbatim `endpoint_assumption`, `capability_gaps[]` naming the dropped `web_search` and the rule-1b decision. fixtures/README.md counts 4 aws-design-ai fixtures and names the new case. Tests (tests/test_runtime_default_review_round3.py): Step 0.8 phrases in both design files (and that rule 2 depends on 1b); the decision sentence in all five schema docs; vendored schema docs byte-identical; the new fixture validates against aws-design-ai.schema.json with the expected row shape; corpus invariant — no `mantle_openai_chat` row with `web_search` / `background_inference`, no `mantle_messages` row with `background_inference` / `projects`. Validation: plugin `pytest tests -q` 1532 passed, 1 failed (known test_model_id_lint::test_real_plugin_passes local .venv noise, pre-existing); skills/llm-to-bedrock/scripts 672 passed; skills/agent-advisor/scripts 570 passed (unchanged); scripts/validate-artifacts.py --fixtures PASS (103 artifacts); tools/sync-vendored.py --check PASS (64); tools/check-skill-refs.py --check PASS; git diff --check clean; markdownlint-cli2 0 issues.
…GPT-5.6 Terra, whose Geo CRIS rate is published (review A, B1) The chat-source-web-search-continuity-kept design fixture quoted a parity "2.50/15.00" Bedrock price for us.openai.gpt-5.5, but the GPT-5.5 card publishes a commercial In-Region price only and no Geo/Global CRIS row (model-card-openai-gpt-55.html, read 2026-11-09); bedrock-endpoint-policy.md and openai-on-bedrock.md say such rows must never report parity. Switch the fixture to a same-model GPT-5.6 Terra move: source gpt-5.6-terra at OpenAI's $2.00/$12.00 standard list, target us.openai.gpt-5.6-terra at the published US Geo CRIS $2.20/$13.20 (model-card-openai-gpt-56-terra.html, read 2026-11-09; Global CRIS $2.00/$12.00), price_comparison "unfavorable", with the 1.10x Geo premium named in the rationale. The Step 0.8 rule 1b continuity decision, dropped web_search capability gap, chat_completions API and verbatim endpoint_assumption are unchanged. Round-3 test now pins the Terra ids and prices and asserts price_comparison is not parity; fixtures/README.md names the Terra source. Validation (plugins/aws-startup-advisor): - uv run --no-project --with pytest --with jsonschema python -m pytest tests -q: 1532 passed, 1 failed (known tests/test_model_id_lint.py::test_real_plugin_passes, local .venv noise) - python3 scripts/validate-artifacts.py --fixtures: PASS (103 artifacts, 0 errors) - python3 tools/sync-vendored.py --check: PASS (64 files) - git diff --check: clean
herosjourney
force-pushed
the
fix/llm-to-bedrock-runtime-default
branch
from
November 9, 2026 18:37
f3bd385 to
4f86112
Compare
herosjourney
pushed a commit
to herosjourney/agent-toolkit-for-aws
that referenced
this pull request
Nov 9, 2026
…er rebasing onto PR 585 Rebased fix/agent-advisor-runtime-default from e6795f3 onto the new PR 585 tip 4f86112 (backup ref backup/pr588-pre-rebase-91827f91); the ten commits replayed without conflicts. tools/sync-vendored.py re-synced skills/agent-advisor/references/vendored/ai/bedrock-endpoint-policy.md (F13 on aws#585). sync-vendored --check: 65 files OK.
The scan on main now writes schema 1.1 workloads, integration, and agents. Step 4.8 records mantle_signals on that profile instead of replacing the scan. Co-authored-by: Cursor <cursoragent@cursor.com>
…1 profiles The 1.1 scan contract requires workloads and integration. These two fixtures still said 1.0, so the fixture sweep failed after the main merge. Co-authored-by: Cursor <cursoragent@cursor.com>
This was referenced Nov 10, 2026
5 of 9 tasks
Keep Haiku 5.5 as the fast Converse tier from main and the GPT-5.4/5.5 runtime catalog re-read from this branch. Co-authored-by: Cursor <cursoragent@cursor.com>
Conflict resolution (plugins/aws-startup-advisor/skills/shared/ai/models/openai-bedrock.json, top-level metadata only; every model entry auto-merged): - source: union. Keeps main's Haiku 5.5 model-card evidence (read 2026-11-09) and the Opus 5.5 line, plus this branch's GPT-5.5/GPT-5.4 model-card re-read (2026-11-06, runtime CRIS ids, Chat Completions on mantle). Trailing sentence merged to "Other OpenAI and unchanged entries retain their evidence ...". - disclaimer: union. Main's Haiku 5.5 fast/low-cost Converse tier sentence first, then this branch's GPT-5.5/GPT-5.4 re-read note (runtime CRIS paths, Chat Completions on mantle, 1.05M context, long-context pricing), then the shared Opus 5.5 / Fable / Mythos / eu-west-2 text that both sides carried unchanged. - verified_at: main's 2026-11-09 (newer than the branch's 2026-09-24). - openai_gpt_5_5 / openai_gpt_5_4 entries: branch values (refreshed runtime metadata, pricing, inference_profiles, region_availability, capabilities); main did not touch these entries. - anthropic_claude_haiku_5_5, Haiku 4.5 display_name/capability_source, and the pretty-printed GPT-6 Sol/Luna/6.1 Sol arrays: main values; the branch did not touch these. - Vendored copy regenerated with tools/sync-vendored.py (--check passes), not hand-edited. Verification on this merge: - cd plugins/aws-startup-advisor && mise run ci: all tasks finished (lint:md 0 errors, lint:model-id, lint:model-staleness, lint:model-catalog-drift, lint:startup-model-pricing, lint:artifacts PASS 104 artifacts, lint:vendored-parity PASS 62 files, lint:skill-refs PASS 1498 refs, pytest 1489 passed + 225 subtests, telemetry 311 passed). - skills/agent-advisor/scripts pytest: 597 passed. - skills/llm-to-bedrock/scripts pytest: 453 passed (run via uv with httpx/openai/ aws-bedrock-token-generator/jsonschema/boto3 available).
vagmur-aws
reviewed
Nov 11, 2026
vagmur-aws
left a comment
vagmur-aws
left a comment
Contributor
There was a problem hiding this comment.
Reviewed: 14445a3 (includes #583), 118 files (+8,489/−897) against main: about 3,000 new lines, 3,700 test lines, 2,300 byte-identical copies, 355 fixture lines.
Ran:
mise -C plugins/aws-startup-advisor run cion this head merged with currentmain: 1,615 passed and 225 subtests; 311 telemetry tests.- The bedrock-migration-assessment contract probes (schema boundaries, google-genai and framework matrices; 2,482 combinations): 0 problems.
- A headless Claude Code run with this branch's plugin, 3 repeats, on
fixtures/bma-scan-openai-mantle-signals(eligible for the Clarify fast path), with no real API calls. openai3.49.0 andanthropic2.8.0 wheel sources read for environment-variable selectors.
Did not review line by line: the files outside bedrock-migration-assessment (llm-to-bedrock's Runtime Messages lane, gcp-to-aws and azure-to-aws Design and Generate, the endpoint policy). Those were covered only by CI and the scripted checks.
Blocking
None.
Should-fix
-
[Contradictions] The single-Region question is skipped whenever signals were detected. —
bedrock-migration-assessment/references/phases/clarify/clarify-ai-only.md:367When the summary reports no signals, ask the four sub-questions outright.
discover-app-code.md:348in_region_processingis never code-detectable — Clarify always asks it.When the scan finds any signal, Q10.5 only confirms those signals. Sub-question (d), single-Region processing, and any other sub-question the scan could not detect, is never asked. In the headless run, 2 of 3 repeats never asked (d) and recorded no
in_region_processinganswer.
Fix: always ask every sub-question that no detected signal answered, and always ask (d). The same pattern is in gcp-to-awsclarify-ai-only.md:327("When the profile has noworkloads[], no signals…") andclarify-ai.md:489. -
[Identity] The scan misses project and workspace selectors that the SDKs read from environment variables. —
discover-app-code.md:345projects: …project=kwarg …, or anOPENAI_PROJECT_ID-style environment variable name read in code | … no header means the default projectopenai3.49.0 readsprojectfromOPENAI_PROJECT_IDand sends theOpenAI-Projectheader with no code involved (_client.py: "projectfromOPENAI_PROJECT_ID").anthropic2.8.0 readsANTHROPIC_WORKSPACE_IDin its credential chain (lib/credentials/_constants.py). So "no header means the default project" is false whenever that variable is set, and the variable usually lives in.env, which the skill may not open. With finding 1, a run that detected another signal then never asks (c), the Projects/Workspaces question.
Fix: count those variable names in non-secret deploy config (DockerfileENV, compose, Kubernetes manifests), and rely on (c) always being asked otherwise. -
[Completeness] The fast path has no place for Q10.5. —
bedrock-migration-assessment/.../clarify-ai-only.mdStep 1.5Always: Q1.5, Q2, Q3, Q4, Q7, and Q11.
Q10.5 says "ask it in Batch 2", and the fast path has no Batch 2. This PR's own fixture is fast-path eligible (
eligible_for_clarify_fast_path: true). In 3 of 3 headless repeats the agent noticed and asked Q10.5 as an extra turn after the batch ("I need one more answer before I can save your preferences"), so the user is not harmed today, but the instructions contradict each other and the agent resolves it on its own each time.
Fix: add Q10.5 to the fast path's "Always" list, prefilled frommantle_signals_detected. -
[Identity] A per-model signal becomes a requirement for every workload. —
bedrock-migration-assessment/.../clarify-ai-only.md:388the summary's signals are per model, so use
*unless the user names a workloadThe bedrock-migration-assessment profile has
workloads[]withmodel_id. Aweb_searchsignal ongpt-4obecomesworkload_id: "*", which applies it to every workload, including another provider's model, which then gets a Mantle requirement or a capability-gap note it does not need. Across the 3 headless repeats the agent wrote*twice and the real workload ID once for the same input.
Fix: map each signal to the workloads whosemodel_idmatches, and use*only when the profile has noworkloads[].
Nits
bedrock-migration-assessment/references/phases/discover/discover.md:9lists the profile fields accepted per schema version and was not updated formantle_signals, which the schema now allows at every version.
Notes
- bedrock-migration-assessment's own OpenAI and Anthropic usage readers (#592, #605) do not record the usage-derived Mantle signals this PR adds to the gcp-to-aws and azure-to-aws readers. I'll add that side in a follow-up after this merges.
- This head merges with current
mainand with #605 without conflicts.
5 of 9 tasks
herosjourney
pushed a commit
to herosjourney/agent-toolkit-for-aws
that referenced
this pull request
Nov 11, 2026
…spend and volume in Q3 and Q7 in bedrock-migration-assessment (aws#605) * feat(aws-startup-advisor): measure Anthropic usage and show measured spend and volume in Q3 and Q7 in bedrock-migration-assessment Stage 3a.1b-5a of aws#389. Anthropic usage reader (discover-anthropic-api.md, scripts/anthropic-usage.py), a stdlib script like openai-usage.py so the agent never opens the Admin key file. Checked against Anthropic's current Usage and Cost API reference: - GET only, to usage_report/messages and cost_report on api.anthropic.com, with x-api-key and anthropic-version. Never grouped by account, API key, or service account. The consent says the Console Admin key has no narrower usage-only scope. - The probe prints each workspace's spend and writes nothing. The default workspace has a null workspace_id, which workspace_ids[] cannot select, so the script filters rows by workspace_id; cost_report has no workspace filter at all. Rows of unselected workspaces are dropped before writing, and the consent says so. - Capture deletes earlier files first, records result, and deletes the key file on every exit; a 404 probe (Claude Enterprise or Claude Platform on AWS) exits ORG_NOT_SUPPORTED and deletes it too. summarize reads only rows the manifest marks ok. - Cost amounts are cents. cost_status follows the existing convention; cost_report reports only the standard and batch tiers (Priority Tier is billed separately), so other tiers make it partial_window and are named in uncosted_service_tiers. The API reports no request counts, so the profile omits them rather than writing 0. Peaks count uncached input and cache writes, not cache reads. - merge adds anthropic/anthropic_api models with provenance usage_api, keeps raw_model verbatim, and refuses when Anthropic is a gateway upstream. Discover Step 2c runs each reader whose source the profile has, OpenAI then Anthropic, each with its own consent. The preview takes one --usage per profile and reports each source plus totals (preview_version 2); the total spend is null when any source's spend is unknown. Clarify: Q3 and Q7 show the measured spend and volume from migration-preview.json and mark the matching option, but the user still answers (chosen_by user), so Q7 stays never-skipped. Q3 now asks about "your current AI providers" instead of "OpenAI or Gemini". The usage-derived Mantle signals in aws#585 are not added here; they belong after aws#585 merges. * fix(aws-startup-advisor): delete the Admin key on any failure, keep cache reads out of the busiest day, and keep measured usage honest across sources and old previews Review findings on aws#605, each reproduced first. herosjourney: - A URLError during capture or probe skipped the key-file deletion, so a timeout or DNS failure left the Admin key on disk. capture now deletes it in a finally block, and probe deletes it on any failure before re-raising; a successful probe still keeps it for capture, and a capture call missing --workspace keeps it so the corrected call works. openai-usage.py had the same gap and gets the same fix. Both readers document the network-error exit. - The busiest day and day_input_tokens counted cache reads while the per-minute peak did not. Day totals now count uncached input and cache writes, the minute-sample day uses the same basis, and day_to_average_ratio divides by that same total. leon1418: - measured_usage reported the shortest source's active_days for the combined spend, so $3,000 over 30 days plus $20 over 2 days read as "$3,020 over 2 days". The aggregate no longer has active_days or partial_window; Q3 and Q7 give each source its own period. - A full-window capture with Priority Tier usage had cost_status partial_window but no capture_warnings entry, and llm-to-bedrock's baseline and report read capture_warnings, not uncosted_service_tiers. The profile now adds "cost_report does not include priority tier spend; this total excludes it", as the gcp-to-aws Anthropic reader does, so the downstream comparison is qualified. - A migration-preview.json written by the version on main (preview_version 1) has no sources or total_tokens, and resuming at Clarify does not rewrite it. Step 0 now re-runs Discover Step 2d first when the preview is not version 2; that reads only the files already in the run directory, scans nothing, and calls no API. fixtures/bma-preview-v1 is a preview generated by main.
herosjourney
pushed a commit
to herosjourney/agent-toolkit-for-aws
that referenced
this pull request
Nov 12, 2026
… bedrock-migration-assessment (aws#619) * feat(aws-startup-advisor): measure Anthropic usage and show measured spend and volume in Q3 and Q7 in bedrock-migration-assessment Stage 3a.1b-5a of aws#389. Anthropic usage reader (discover-anthropic-api.md, scripts/anthropic-usage.py), a stdlib script like openai-usage.py so the agent never opens the Admin key file. Checked against Anthropic's current Usage and Cost API reference: - GET only, to usage_report/messages and cost_report on api.anthropic.com, with x-api-key and anthropic-version. Never grouped by account, API key, or service account. The consent says the Console Admin key has no narrower usage-only scope. - The probe prints each workspace's spend and writes nothing. The default workspace has a null workspace_id, which workspace_ids[] cannot select, so the script filters rows by workspace_id; cost_report has no workspace filter at all. Rows of unselected workspaces are dropped before writing, and the consent says so. - Capture deletes earlier files first, records result, and deletes the key file on every exit; a 404 probe (Claude Enterprise or Claude Platform on AWS) exits ORG_NOT_SUPPORTED and deletes it too. summarize reads only rows the manifest marks ok. - Cost amounts are cents. cost_status follows the existing convention; cost_report reports only the standard and batch tiers (Priority Tier is billed separately), so other tiers make it partial_window and are named in uncosted_service_tiers. The API reports no request counts, so the profile omits them rather than writing 0. Peaks count uncached input and cache writes, not cache reads. - merge adds anthropic/anthropic_api models with provenance usage_api, keeps raw_model verbatim, and refuses when Anthropic is a gateway upstream. Discover Step 2c runs each reader whose source the profile has, OpenAI then Anthropic, each with its own consent. The preview takes one --usage per profile and reports each source plus totals (preview_version 2); the total spend is null when any source's spend is unknown. Clarify: Q3 and Q7 show the measured spend and volume from migration-preview.json and mark the matching option, but the user still answers (chosen_by user), so Q7 stays never-skipped. Q3 now asks about "your current AI providers" instead of "OpenAI or Gemini". The usage-derived Mantle signals in aws#585 are not added here; they belong after aws#585 merges. * fix(aws-startup-advisor): delete the Admin key on any failure, keep cache reads out of the busiest day, and keep measured usage honest across sources and old previews Review findings on aws#605, each reproduced first. herosjourney: - A URLError during capture or probe skipped the key-file deletion, so a timeout or DNS failure left the Admin key on disk. capture now deletes it in a finally block, and probe deletes it on any failure before re-raising; a successful probe still keeps it for capture, and a capture call missing --workspace keeps it so the corrected call works. openai-usage.py had the same gap and gets the same fix. Both readers document the network-error exit. - The busiest day and day_input_tokens counted cache reads while the per-minute peak did not. Day totals now count uncached input and cache writes, the minute-sample day uses the same basis, and day_to_average_ratio divides by that same total. leon1418: - measured_usage reported the shortest source's active_days for the combined spend, so $3,000 over 30 days plus $20 over 2 days read as "$3,020 over 2 days". The aggregate no longer has active_days or partial_window; Q3 and Q7 give each source its own period. - A full-window capture with Priority Tier usage had cost_status partial_window but no capture_warnings entry, and llm-to-bedrock's baseline and report read capture_warnings, not uncosted_service_tiers. The profile now adds "cost_report does not include priority tier spend; this total excludes it", as the gcp-to-aws Anthropic reader does, so the downstream comparison is qualified. - A migration-preview.json written by the version on main (preview_version 1) has no sources or total_tokens, and resuming at Clarify does not rewrite it. Step 0 now re-runs Discover Step 2d first when the preview is not version 2; that reads only the files already in the run directory, scans nothing, and calls no API. fixtures/bma-preview-v1 is a preview generated by main. * feat(aws-startup-advisor): measure OpenRouter usage and BYOK spend in bedrock-migration-assessment Stage 3a.1b-5b of aws#389. Stacked on aws#605. OpenRouter usage reader (discover-openrouter-api.md, scripts/openrouter-usage.py), a stdlib script like the OpenAI and Anthropic readers so the agent never opens the management key file. It keeps every field the gcp-to-aws OpenRouter reader produces, checked against OpenRouter's current API reference: - GET /credits (the probe), /workspaces, /activity, /key, and /analytics/meta, plus one read-only POST /analytics/query (three queries: BYOK classification, volume and fees, hourly peak). The script refuses any other endpoint, per-key or per-user parameter, grouping, or query body other than metrics by model, hourly at most, and a workspace filter. The analytics queries ask only for metrics /analytics/meta lists, so a renamed metric makes that field unknown instead of failing. The consent names the POST and says a management key has no read-only scope. - One workspace or the whole account, filtered (never grouped) on both /activity and the analytics queries. - Capture deletes earlier files, records result, and deletes the key on every exit, including a network error; a revoked key on /credits or /activity is key_invalid_mid_run and a failed /activity is all_usage_failed. A failed analytics row does not stop the capture. - summarize joins analytics on model_permaslug and rolls up to the model id: byok_passthrough, mixed, no_byok, uncertain, or unknown, with the gcp-to-aws thresholds and profile-level rule. It adds byok_fees_usd, request counts, p95 latency and throughput, web-search cost, guardrail counts, and an hourly peak (day peak when the hourly file is missing or truncated), and warns when /activity is empty but lifetime usage is not. - New: /activity now reports byok_usage_inference, so the profile records byok_inference_cost_usd (billed by the user's own provider keys, not in monthly_cost_usd) and warns when it is above zero. - merge applies the scan's OpenRouter prefix rules (openai/, anthropic/, google/ by serving provider) and adds models to the source and the gateway upstream only when the gateway's upstreams are observed; otherwise the profile is unchanged and the models are named. Discover Step 2c runs the reader when the profile has an OpenRouter gateway. The preview adds OpenRouter tokens, adds BYOK inference to the total, and leaves out OpenRouter rows a direct profile also counts unless they are no_byok (the llm-to-bedrock name rule), listing them in overlap_excluded; Q3 and Q7 say so. * fix(aws-startup-advisor): join OpenRouter analytics by permaslug, keep unmeasured values unknown, keep router fees, and merge across every gateway Review findings on aws#619 (leon1418), each reproduced first. - Analytics replies label the model dimension for display ("OpenAI: GPT-4.1"), and only filters take the permaslug, so joining a grouped reply to /activity's model_permaslug matched nothing. Each analytics row now sends one query per permaslug in /activity, filtered by it and never grouped, and stores the replies keyed by that permaslug (at most 50 models; the manifest names the rest). A truncated reply is left out and named. - An unoffered or unreported metric became 0, so a model with BYOK spend could be certified no_byok and its provider bill counted twice. A permaslug is classified only when both byok_usage and openrouter_usage were offered and reported; volume fields that were not measured are null with a warning, never 0. - Leaving out an overlapping OpenRouter row also dropped OpenRouter's own charge with no warning. The preview keeps a measured router fee on a byok_passthrough row, reports every dropped charge in overlap_excluded and spend_excludes_usd, says when a mixed row drops independent usage, and reports only the BYOK inference it actually includes (byok_inference_included_usd). Q3 reads those fields. - The merge refreshed only the OpenRouter upstream after changing a shared source, so a valid OpenRouter plus LiteLLM profile failed validation. Every gateway upstream that references a changed source is rebuilt before validation. - The merge only lowercased Gemini names. It now applies the scanner's Step 4 Gemini rule (drop version suffixes, keep -lite, -thinking, -image, -live, -tts), records the OpenRouter id in the evidence, and keeps it as raw_model for a new model. * fix(aws-startup-advisor): merge the OpenRouter models that can be placed, read the date__hour field, and describe left-out rows accurately Review findings on aws#619 (herosjourney), each reproduced first. - One model with an unplaceable prefix (meta-llama/, mistralai/, ...) blocked the whole merge. The merge now adds every model it can place and lists the rest under not_merged; it exits 5 only when none can be placed. - The analytics reference's example time field is date__day, so the hourly field is date__hour, which the hour lookup missed; two permaslugs in the same hour were then not added up. The lookup now reads hour or any date__ field. - The Q3 and preview wording said a direct provider also counts every left-out row, which is false for a mixed row's OpenRouter-funded usage. Both now say the row is left out so its BYOK traffic is not counted twice, and a mixed row's note says its OpenRouter-funded usage is not in the total. - An error body whose error is a string raised AttributeError and aborted the capture. All three readers now report it as the row's failure. - A rejected key on the optional /key row is recorded skipped, as the reference says. - SKILL.md's Discover output now names all three usage profiles. The preview's router-fee and mixed-row handling (Should-fix 1) was already fixed in 6802fd4; a test now runs the reviewer's probe (direct $100, a byok_passthrough row with a $4 fee, a mixed row) and asserts $104 and the mixed-row warning.
added 2 commits
November 11, 2026 18:04
Port GPT-5.4/5.5 catalog edits into bedrock-model-registry.json and re-render openai-bedrock.json via tools/build-model-catalogs.py, re-sync vendored copies. Add optional pricing object to registry/catalog schemas and renderer. Verified: mise run ci (plugins/aws-startup-advisor) passed, 1731 tests.
…un ci green (1857 passed); build-model-catalogs/sync-vendored --check pass
herosjourney
pushed a commit
to herosjourney/agent-toolkit-for-aws
that referenced
this pull request
Nov 12, 2026
Registry/schemas hand-merged, catalogs re-rendered, vendored copies re-synced. Verified: build-model-catalogs --check and sync-vendored --check pass; mise run ci green (1857 passed).
herosjourney
dismissed stale reviews from leon1418 and leon1418
November 12, 2026 04:55
Stale: made against an older head. Findings were addressed and all threads resolved; the branch was merged with current main and re-pushed. Re-review requested.
Contributor
Author
|
All review threads are replied to and resolved. The branch is now merged up to current main and the updated #583 (head |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
aws-startup-advisorplugin steers same-model AI migrations toward the OpenAI-compatiblebedrock-mantleendpoint by default. The shared OpenAI and OpenRouter guides call Mantle "the primary path" and "the smallest change"; thegcp-to-awsandazure-to-awsDesign and Generate phases list Mantle first and recordmantle_openai_responses/mantle_openai_chatwithout asking why Mantle is needed; thellm-to-bedrockcode-rewriter says the Mantle express lane "remains the default and smallest change". The Anthropic guide tells an Anthropic SDK app to rewrite to boto3 Converse, which changes the client library, the response shape, and the conversation-history format even though the public Messages API page shows the Anthropic SDK working againstbedrock-runtimewith only a base URL and credential change.The public Amazon Bedrock User Guide says to use
bedrock-runtimefor new applications and reservesbedrock-mantlefor capabilities that only exist there (background inference, server-side tools, web search, projects, workspaces, Mantle-only models or APIs, in-region processing). A startup following the plugin today is routed to the wrong endpoint for the common case, and the plugin has nobedrock-runtimelane for an Anthropic SDK app at all.Two smaller defects sit underneath that:
normalize_api_pathinpreflight_bedrock.pysilently resolved aruntime_openai_crisplan that recorded no source API to Converse, and the shared guides paraphrased the Mantle selector reasons in several places so the lists had already started to drift.Fix
State the endpoint rule once and point everything else at it, make Runtime the default for every same-model path, and give Anthropic SDK apps a Runtime Messages lane that runs end to end. Concretely:
skills/shared/ai/bedrock-endpoint-policy.md(vendored intogcp-to-awsandazure-to-aws) states the rule, defines themantle_selectorenum (background_inference | server_side_tools | web_search | projects | workspaces | mantle_only_model_or_api | in_region_processing), and carries a public capability matrix, lifecycle, and quota wording with a User Guide URL and read date on every row. Anymantle_*design row must now carry a recordedmantle_selector.runtime_openai_criswith the source app's own API (required_client_api=responsesorchat_completions) is the Tier 0 default; Runtime steps are shown first; "primary path" / "smallest change" phrasing is removed.design-ai.md,design.md,generate-artifacts-ai.md,generate-ai.md,generate.md, the estimate route lists, bothopenai-on-bedrock.mdclient-setup sections, and the threeschema-design-aws-ai.mdcopies gain theruntime_messages,messages,same_model_runtime, andmantle_selectorvocabulary. Estimate menus route runtime paths to a "Bedrock runtime (CRIS)" label.ai-anthropic-to-bedrock.mdkeepsanthropic.Anthropicand repoints it athttps://bedrock-runtime.{region}.amazonaws.com/anthropicwith a string Bedrock token minted per client (provide_token), recordsmigration_path: "runtime_messages"+required_client_api: "messages"by default, moves the boto3 Converse rewrite under an explicit-alternative heading, and gates Mantle Messages behind amantle_selector.normalize_api_pathraises for aruntime_openai_crisplan with no source API.llm-to-bedrock/SKILL.mdresolves that case with the user (resolved_runtime_api); only an explicitrequired_client_api: "converse"reaches the Converse rewrite.llm-to-bedrock.create_messages_client/probe_runtime_messages_model(Anthropic SDK onbedrock-runtime/anthropic,max_retries=0, client opened in awithblock),--runtime-api messageswith an embedding-id guard that still routes Titan/Cohere ids toInvokeModel,image_input.messages_message, aBedrockGetInferenceProfileIAM statement for CRIS targets, the rewriter's "Runtime Anthropic Messages lane" (Mantle express lane is selector-gated; Converse rewrite is for explicitconverseplans), evaluator §6b / §9.5b and the §10messagesbranch, report-generator §6.4, and an "Endpoint policy (bedrock-runtime default)" section inSKILL.md.anthropic>=0.75,<1is pinned in the scripts project. The rewriter's Mantle Anthropic snippet is corrected tobase_url .../anthropicwithapi_key=so the SDK resolves/anthropic/v1/messages, matching the public page.tests/test_endpoint_policy_contract.pypins the policy sentences, the vocabulary in all three schema docs, the Anthropic guide's string-token Runtime default, and thellm-to-bedrock/SKILL.mdpointer.llm-to-bedrockscript tests add fail-closed normalization, a missing-SDK fail-closed path, a real-SDK wire test (POST .../anthropic/v1/messages,x-api-key,anthropic-version), per-client token minting,main()routing for Claude-only and Claude + embedding lists,messages_messageshapes, the IAM statement count, and execution of theSKILL.md, §6b, and §9.5b heredocs. Existing text-contract tests only gain assertions.How the plugin learns a Mantle requirement
The first version of this PR defined the
mantle_selectorenum but nothing populated it: Design had no input to read, so every same-model workload would have landed onbedrock-runtimewhether or not the app relied on a Mantle-only capability. Source code does not always reveal that (anin_region_processingneed is a business constraint; Terraform shows no API usage at all), and an AI-only run may have no code to scan. Commitsa6f9ef1a,73835b14,922687ea,1117ea6b,e6795f32close that gap in three places:discover-app-code.md(gcp, azure, andbedrock-migration-assessment) and the OpenAI / Anthropic / OpenRouter usage-API discover files record the five code-visible selectors asworkloads[].mantle_signals[](bedrock-migration-assessmentwrites a root-levelmantle_signals[]) with{selector, file, line, construct}evidence:background=trueon a Responses call (background_inference);web_search/web_search_preview*tool types, Anthropicweb_search_20250305, OpenRouterplugins: [{"id": "web"}]or a:onlinemodel suffix (web_search);mcp,code_interpreter,file_search,computer_use_preview, Anthropiccode_execution_*/text_editor_*/computer_*/bash_*tool types (server_side_tools); anOpenAI-Projectheader orproject=kwarg (projects); ananthropic-workspace-idheader (workspaces). Negatives are explicit: a client-sidefunctiontool merely namedweb_search,store=true,OpenAI-Organizationalone, and OpenRouter's ownworkspace_idare not signals.mantle_only_model_or_apistays catalog-derived in Design;in_region_processingis never detected and is always asked. The positive and negative constructs are listed in the new "How a selector is learned" section ofbedrock-endpoint-policy.md, sourced from the public Responses API page (inference-responses-api.html, read 2026-11-07).validate-ai-profile.pychecks every signal against a known workload (unmatched_mantle_signal) and reportsmantle_signals_detectedin its summary.clarify-ai.md/clarify-assemble.mdand azureclarify-ai.md, and as Q10.5 in the gcp, azure, andbedrock-migration-assessmentclarify-ai-only.md. When Discover scanned code, the answers are prefilled frommantle_signalswith their file:line evidence shown for confirmation; when no code was scanned the questions are asked outright, and the AI-only fast paths never drop them. Answers persist topreferences.json→ai_constraints.mantle_requirements[]as{workload_id, selector, source: detected|user|none, evidence?, region?}(schema-preferences.mdin gcp and BMA). The copy tells the user up front that a "yes" for a workload whose source API has no Mantle path (Gemini / Vertex AI) records a capability gap rather than an endpoint change.design-ai.md(mirrored in allschema-design-aws-ai.mdcopies) derivesbedrock_models[].mantle_selectoronly from adetectedoruserentry inai_constraints.mantle_requirementsplus the catalog formantle_only_model_or_api; it never infers an endpoint from the source API or the model. Everymantle_*row records the cited entry asmantle_requirement; every runtime row carries the verbatimendpoint_assumptionsentence: "No Mantle-only requirement was recorded for <workload_id>; bedrock-runtime selected. If this workload needs background inference, server-side tools, Projects/Workspaces, or single-Region processing, re-run Clarify." A matched requirement on a workload with no Mantle path keepsbedrock-runtimeand surfaces the unhonored selector, its source, and evidence inrationaleandcapability_gaps[](aregional_warnings[]entry forin_region_processing). Generate echoes whichever field is present in the script/adapter header;llm-to-bedrock/SKILL.mdemits an "Endpoint basis:" context line whenever any row carries a Mantle marker, andllm2bedrock-report-generator.mdrenders it as the Executive Summary Endpoint line (a Mantle row without a recorded requirement renders as "source unknown — plan predates mantle_requirements").Tests:
tests/test_mantle_selector_discovery.py(15; positives and negatives against the newfixtures/bma-scan-openai-mantle-signals),tests/test_mantle_requirements_clarify_contract.py(13),tests/test_mantle_requirements_design_contract.py(17), plus theaws-design-aifixturesmulti-provider-direct(detectedweb_searchMantle row and a runtimeendpoint_assumption),openrouter-gateway-multi-upstream(in_region_processingrequirement), and the newruntime-default-no-requirement.Stacked on #583
This is PR 2 of 3. It is based on PR 1 (#583,
fix/gpt54-55-runtime-metadata) ate39c59a3(rebased from8c37ef0f; this branch's tip is now4f861123, previouslyf3bd3855; pre-rebase tip kept atbackup/pr585-pre-rebase-f3bd3855), not onmain. Until #583 merges, the diff shown here includes PR 1's commits (the GPT-5.4/5.5 metadata refresh and the GPT-5.6in.prefix scoping); the commits that belong to this PR are the nineteen on top ofe39c59a3(the formerf3bd3855was dropped during the rebase because its regex change already lives ine39c59a3). Every GPT-5.4/5.5 window this PR edits keeps PR 1's wording (us./global.only for GPT-5.4, GPT-5.5, GPT-5.6 Sol, and GPT-6;in.only for GPT-5.6 Terra and Luna), and PR 1'stest_gpt54_55_cris_prefix_scope.pystays green.What this PR does NOT do
agent-advisor.git diff 8c37ef0f..HEAD -- skills/agent-advisor skills/shared/ai/modelsis empty (theagent-advisorMantle-requirement intake is owned by fix(aws-startup-advisor): agent-advisor defaults to bedrock-runtime, selects Mantle only by a verified selector, adds Runtime Responses/Chat/Messages paths #588). Theagent-advisorcandidate walk,_SOURCE_MODEL_KEYS,model-recommendation.json,verify_model_path.py, andhandoff-migration.mdare untouched, soagent-advisorstill recommends Mantle paths by catalog order. Teaching the advisor themantle_selectorrule and theruntime_messages/same_model_runtimeidentities is PR 3 of this stack; aTODO(PR 3 ...)comment ingcp-to-awsdesign-ai.mdmarks the handoff point.mantle_messagesplan keeps today's dispatch (Mantle express lane in the rewriter; evaluator and preflight through the existing Converse path).README.md,mise.toml, or any non-AI skill.Scope
Skills:
azure-to-aws,gcp-to-aws,llm-to-bedrock,bedrock-migration-assessment(Mantle-requirement discovery and Clarify only),shared. No new skills.Files (this PR only,
8c37ef0f..HEAD): 93 changed (15 added, 78 modified, 0 deleted), +6,178 / −588, all underplugins/aws-startup-advisor/. Of these, the Mantle-requirement commits (00cf7b6d..HEAD) account for 48 files (7 added, 41 modified), +1,755 / −56.skills/shared/ai/:bedrock-endpoint-policy.md(new),ai-anthropic-to-bedrock.md,ai-openai-to-bedrock.md,ai-openrouter-to-bedrock.md,schema-design-aws-ai.md,README.md.azure-to-aws/references/vendored/andgcp-to-aws/references/vendored/(each:README.md,ai/bedrock-endpoint-policy.mdnew, and the four guides above), byte-identical to the canonical files.azure-to-aws:design-ai.md,design.md,estimate-assemble.md,generate-artifacts-ai.md,generate.md,shared/openai-on-bedrock.md,shared/schema-design-aws-ai.md.gcp-to-aws:SKILL.md,design-ai.md,estimate.md,generate-ai.md,generate-artifacts-ai.md,shared/openai-on-bedrock.md,shared/schema-design-aws-ai.md.llm-to-bedrock:SKILL.md,resolve-bedrock-model-id.md,behavior-delta-detection/references/anthropic-to-bedrock.md,scripts/preflight_bedrock.py,scripts/image_input.py,scripts/iam_policy.py,scripts/pyproject.toml,scripts/uv.lock,scripts/schemas/eval.json, and the four test filestest_astra_execution_contract.py,test_preflight_bedrock.py,test_iam_policy.py,test_image_input.py.agents/:llm2bedrock-code-rewriter.md,llm2bedrock-prompt-evaluator.md,llm2bedrock-report-generator.md.fixtures/aws-design-ai/:mantle_selectoradded to the two existingmantle_openai_responsesrows.tests/:test_endpoint_policy_contract.py(new),test_ai_only_option_c_framework_route.py,test_generate_mantle_dispatch.py,test_gpt6_review_round2.py,test_gpt6_round4_findings.py.Mantle-requirement commits (
00cf7b6d..HEAD) additionally touch:skills/shared/ai/:bedrock-endpoint-policy.md("How a selector is learned"),schema-design-aws-ai.md(mantle_requirement/endpoint_assumption), plus the vendored copies of both undergcp-to-awsandazure-to-awsreferences/vendored/ai/.gcp-to-awsandazure-to-aws:discover-app-code.md,discover-openai-api.md,discover-anthropic-api.md,discover-openrouter-api.md,clarify-ai.md,clarify-ai-only.md,design-ai.md,generate-artifacts-ai.md,shared/schema-discover-ai.md,shared/schema-design-aws-ai.md; gcp alsoclarify-assemble.md,generate-ai.md,shared/schema-preferences.md.bedrock-migration-assessment:discover-app-code.md(Step 4.5),clarify-ai-only.md(Q10.5),shared/schema-preferences.md,contracts/ai-workload-profile.schema.json,scripts/validate-ai-profile.py.scripts/contracts/ai-workload-profile.schema.json(rootmantle_signals[], byte-identical to the BMA copy).llm-to-bedrock/SKILL.md("Endpoint basis:" context line) andagents/llm2bedrock-report-generator.md(Endpoint line, Model Mapping notes, Limitations).bma-scan-openai-mantle-signals/(new),aws-design-ai/runtime-default-no-requirement/(new),aws-design-ai/multi-provider-direct,aws-design-ai/openrouter-gateway-multi-upstream,bma-ai-only-clarify/preferences.json.tests/:test_mantle_selector_discovery.py,test_mantle_requirements_clarify_contract.py,test_mantle_requirements_design_contract.py(all new),test_bedrock_migration_assessment_core.py.Base: #583 (
fix/gpt54-55-runtime-metadata) ate39c59a3, itself onmain. Nineteen commits (the thirteen below, the round-3 commits listed under Validation, and the review-round-2 Mantle-requirement follow-ups): the endpoint policy and Runtime-first OpenAI guidance; the Anthropic Runtime Messages default in the shared guide and design/generate phases; the fail-closed CRIS resolution and thellm-to-bedrockMessages lane; a review follow-up (embedding-id guard, Mantle snippet alignment, executable §6b/§9.5b tests, selector paraphrases replaced by pointers); four review round-1/round-2 commits; and five Mantle-requirement commits (Discover detection, Clarify questions, Design derivation and disclosure, vocabulary alignment, and the no-Mantle-path Step 0.8 case).Not touched:
skills/agent-advisor/**,skills/shared/ai/models/**,heroku-to-aws,render-to-aws,plugin.json, plugin manifests, marketplaces, pluginREADME.md,mise.toml,.mcp.json.bedrock-migration-assessmentis touched only for the Mantle-signal scanner step, the Q10.5 Clarify block, and the shared profile schema / validator.Validation
Sources: the public Amazon Bedrock User Guide, all read 2026-11-07, all under
https://docs.aws.amazon.com/bedrock/latest/userguide/:endpoints.html,models-endpoint-availability.html(endpoint rule and model/endpoint matrix)inference-messages-api.html(Anthropic SDK onbedrock-runtime/anthropicwith a string token; Mantle form atbedrock-mantle.{region}.api.aws/anthropic)inference-profiles-prereq.html(bedrock:InvokeModel*on profile and destination model ARNs;bedrock:GetInferenceProfileon the profile ARN)inference-responses-api.html,inference-chat-completions.htmltool-use-server-side.html,web-search.html,projects.html,workspaces.html(Mantle-only capabilities behind the selector enum)cost-mgmt-application-inference-profiles.html,model-invocation-logging.html,monitoring-mantle-metrics.htmlquotas-runtime.html,quotas-mantle.html,scaling-throughput-best-practices.htmlLocal gates, run from this branch on 2026-11-07 against the #583 tip
6b873f8c, all exit 0 unless noted:mise -C plugins/aws-startup-advisor run ci: PASS.test1,408 passed / 222 subtests (1,398 at base, 10 new);test:telemetry311 passed;test:heroku-source-review44 pass / 0 fail;lint:md0 errors;lint:skill-refs1,520 checked / 0 missing;lint:vendored-parity64 files byte-identical;lint:artifacts99 checked / 0 errors;lint:model-id,lint:model-staleness,lint:model-catalog-drift,lint:startup-model-pricing,lint:consent-disclosureOK.llm-to-bedrockpytest (skills/llm-to-bedrock/scripts): 435 passed / 0 failed (418 at base, 17 new).agent-advisorpytest (skills/agent-advisor/scripts): 570 passed / 0 failed (unchanged from base; the suite executes the shared Anthropic guide's Converse snippet, so it was re-run after every edit toskills/shared/ai).python3 tools/sync-vendored.py --check: PASS, 64 files byte-identical.python3 tools/validate.py"All validations passed";mise run validateexit 0;mise run buildlint and validate steps pass (security step: see Limitations).gitleaks protect --stagedover this change set: no leaks found.plugins/aws-startup-advisor*.md/*.jsonfor Mantle-default phrasing ("Mantle is the PRIMARY path", "primary path", "smallest change", "remains the default", "Converse rewrite (default)", "retains the Converse default", "Converse client swap"): no remaining Bedrock-endpoint hits. The only matches are negative test assertions and pre-existing prose about unrelated services (Elastic Beanstalk, OpenSearch, MWAA, Heroku discovery).Review round 1
Head
f3c01556(commitscea38000,34c67d0e,f3c01556on top of6ca403b6) addresses the four review threads: theproject/defaultgrant is decided by id form and per-row API iniam_policy.py(--target ID=API, any-row-wins, fail closed) and stated in the GCP/Azure setup recipes;preflight_bedrock.pycarries eachbedrock_models[]row's API (--target ID=API, fail closed on an unknown API); a--check-lanesgate stops amantle_messagesplan before B2/B4/Phase C andiam_policy.pyrefuses runtime permissions for it, with GCP Step 0 branches made disjoint;resolved_runtime_apiis in both resume contexts with an invalidation row. Gates re-run in the worktree on 2026-11-07:mise -C plugins/aws-startup-advisor run ci: exit 0.test1,431 passed / 222 subtests;test:telemetry311 passed;lint:md0 errors;lint:vendored-parity64 files byte-identical;lint:artifacts99 checked / 0 errors;lint:skill-refs1,521 checked / 0 missing.llm-to-bedrockpytest (uv project env): 622 passed / 0 failed (435 at the previous body; +187 across the three round-1 commits).agent-advisorpytest: 570 passed / 0 failed (unchanged).tests/test_gpt54_55_cris_prefix_scope.py: 3 passed.python3 tools/validate.py: All validations passed.python3 tools/sync-vendored.py --check: PASS, 64 byte-identical.git diff --check: clean.mise run build: exit 1 only on the pre-existing gitleaks finding at upstreamf8d09fd1(see Limitations); not introduced by this branch.Review round 2
Head
00cf7b6d(one commit on top off3c01556) addresses leon1418's three P2 threads: the report §6.4 IAM recipe sources ids from the validatedanalysis.target_models(C4 overrides applied) viaiam_policy.validated_target_pairs()and never falls back to the design row'saws_model_id; retainedresponses.retrieve/cancel/deletecalls getbedrock:GetInvoke/CancelInvoke/DeleteInvokeonproject/defaultthrough a newstored_response_opsanalysis field andiam_policy.py --stored-response-ops(only the retained actions, only for a runtime GPT CRISresponsesrow; Converse, Messages, Chat Completions, embeddings, bare mantle ids refused); andSKILL.mdB4 reuses a savedresolved_runtime_apionly afterpreflight_bedrock.py --resume-runtime-apiconfirms the savedassess_design_sha256andtarget_modelsidentity match the current design. Source for the stored-response permissions:https://docs.aws.amazon.com/bedrock/latest/userguide/inference-responses-api.html("Permissions"), read 2026-11-08. Gates re-run in the worktree on 2026-11-08:mise -C plugins/aws-startup-advisor run ci: exit 0.test1,441 passed / 222 subtests;test:telemetry311 passed;lint:md0 errors;lint:vendored-parity64 files byte-identical;lint:skill-refsPASS;lint:artifactsPASS;lint:model-id,lint:consent-disclosure,lint:model-catalog-driftOK (3 pre-existing warnings).llm-to-bedrockpytest (uv project env): 655 passed / 0 failed (622 at round 1; +33 intest_iam_policy_round2.pyandtest_preflight_resume_runtime_api.py).agent-advisorpytest: 570 passed / 0 failed (unchanged; noagent-advisoredits).tests/test_gpt54_55_cris_prefix_scope.py,tests/test_runtime_default_review_round1.py,tests/test_runtime_default_review_round2.py: green.python3 tools/validate.py: All validations passed.python3 tools/sync-vendored.py --check: PASS.git diff --check: clean.mise run build: exit 1 only on the pre-existing gitleaks finding at upstreamf8d09fd1(see Limitations);lint:manifestspassed; not introduced or touched by this commit.Mantle-requirement discovery and Clarify
Head
e6795f32(five commits on top of00cf7b6d:a6f9ef1a,73835b14,922687ea,1117ea6b,e6795f32) adds the Discover detection, Clarify questions, and Design derivation described under "How the plugin learns a Mantle requirement". Source for the detection constructs and the Runtimebackground=true400:https://docs.aws.amazon.com/bedrock/latest/userguide/inference-responses-api.html, read 2026-11-07. Gates run in the worktree on the final commit:mise -C plugins/aws-startup-advisor run ci: exit 0.test1,496 passed / 222 subtests (1,441 at round 2; +55 across the three new test files and the BMA core update);test:telemetry311 passed;test:heroku-source-reviewPASS;lint:md0 errors;lint:artifactsPASS 101 artifacts / 0 errors (99 at round 2; two new fixtures);lint:vendored-parity64 files byte-identical;lint:skill-refs1,532 checked / 0 missing;lint:model-id,lint:model-staleness,lint:model-catalog-drift,lint:startup-model-pricing,lint:consent-disclosurePASS.llm-to-bedrockpytest (skills/llm-to-bedrock/scripts): 655 passed / 0 failed (unchanged from round 2; theSKILL.mdand report-generator edits are text contracts covered by the plugin suite).agent-advisorpytest (skills/agent-advisor/scripts): 570 passed / 0 failed (unchanged; no file underagent-advisorchanged).python3 scripts/validate-artifacts.py --fixtures: PASS, 101 artifacts, 0 errors, baseline unchanged.python3 tools/sync-vendored.py --check: PASS, 64 files byte-identical.python3 tools/validate.py: All validations passed.git diff --check: clean.mise run build(gitleakssecuritystep) not re-run for these commits: it fails on the pre-existing upstream finding atf8d09fd1(see Limitations), which this branch does not touch.Review round 3 (items raised on #588)
Head
4f861123(rebased onto the #583 tipe39c59a3; commitsc23d9072,c7641574,afcd9834,a8a0ec30,4f861123) carries the cloud-side half of five findings raised in the #588 review; theagent-advisorengine half stays on #588.c23d9072).iam_policy.generate_policydispatches retained stored-response grants by endpoint: runtime GPT CRISresponsesrows keepBedrockStoredResponseOps(bedrock:GetInvoke/CancelInvoke/DeleteInvokeonproject/default); a bare Mantle id onresponsesis covered byBedrockMantleInference(GET /v1/responses/{id}underbedrock-mantle:CreateInference,inference-responses-api.htmlread 2026-11-09) and no longer raisesstored_response_ops_no_responses_row; Converse/Messages/Chat/embedding/id-only plans still fail closed. Report §6.4, GCP/Azuregenerate-artifacts-ai.md, and the canonical OpenAI guide updated. Tests:test_iam_policy_round2.py::TestStoredResponseOps(five new cases) andtests/test_runtime_default_review_round3.py::test_report_recipe_dispatches_stored_response_grants_by_endpoint.c23d9072).iam_policy.pygains--mantle-selector(fail-closedparse_mantle_selectors, reasonunknown_mantle_selector) and--external-web-access; aweb_searchrow emitsBedrockWebSearch=bedrock-websearch:InvokeSearch+bedrock-websearch:InvokeFetchon*with anaws:RequestedRegioncondition, plusbedrock-websearch:ExternalWebAccessonly when the user confirmed external access (web-search.html+security-web-search.htmlread 2026-11-09:ExternalWebAccessis an IAM action, not a condition key).web_searchwithout a Mantle row →web_search_without_mantle_row; the flag without the selector →external_web_access_without_web_search; non-Web-Search plans get nobedrock-websearch:*. Report §6.4 "Web Search needs its own grants" paragraph and a How-to-Apply "Tool-level permission check" bullet; GCP/Azure setup recipes and the canonical guide carry the actions, the"external_web_access": falsedefault, and a Step-5 tool-level check. Tests:test_iam_policy_round2.py::TestWebSearchGrants(9, incl. CLI) and fivetest_runtime_default_review_round3.pycontract tests.c7641574).bedrock-endpoint-policy.md(canonical + gcp/azure vendored) and the gcp/azure/bedrock-migration-assessmentdiscover-app-code.mdscanners move Anthropiccomputer_*(incl.computer_toolset_*),browser_toolset_*,text_editor_*,bash_*,memory_*from theserver_side_toolspositive list to "Not a signal" as client-executed tools on both endpoints (Sonnet 5 / Opus 5.5 cards, read 2026-11-09, added to § Sources);code_execution_*,mcp,code_interpreter,file_search,computer_use_preview,image_generationstay positive. New fixturefixtures/bma-scan-anthropic-client-tools. Tests: two new cases intests/test_mantle_selector_discovery.py(validator ok,mantle_signals_detected == [],runtime_messagesemitsBedrockInvokeModelScoped). Theagent-advisorvendored policy copy is re-synced on fix(aws-startup-advisor): agent-advisor defaults to bedrock-runtime, selects Mantle only by a verified selector, adds Runtime Responses/Chat/Messages paths #588.afcd9834). Azuredesign-ai.md(Guardrails block) andshared/schema-design-aws-ai.md(§required_client_api) replace the blanket "neverresponsesfor 6.1 Sol and the other GPT-6/5.6 CRIS models" with the per-model rule (Converse-only: GPT-6 Astra, GPT-5.6 Sol/Terra/Luna, GPT-5.5, GPT-5.4, GPT-6.1 Terra; all four APIs: GPT-6 Sol, GPT-6 Luna; not Responses: GPT-6.1 Sol; nine card URLs, read 2026-11-09);llm2bedrock-code-rewriter.mdcarries the per-card Guardrails / prompt-caching scope sentence. Canonical and GCP schema docs had no blanket wording. Tests: four cases intests/test_runtime_default_review_round3.py.a8a0ec30,4f861123). GCP + Azuredesign-ai.mdStep 0.8 gain rule 1b ("Validate the matched selector against the API it requires":web_search/background_inference→ Responses;workspaces→ Anthropic Messages;projects→ OpenAI-compatible; a provider mismatch keeps runtime and re-runs Clarify; a Chat/Messages source with a Responses-only selector is an explicit API-continuity decision, keep the API and drop the selector or move tomantle_openai_responses, never a silent same-API Mantle row), and rule 2 reads "Any match that passes 1b"; the same sentence lands in canonicalschema-design-aws-ai.md(+2 vendored) and the GCP/Azure shadow schemas. New fixturefixtures/aws-design-ai/chat-source-web-search-continuity-kept(same-model GPT-5.6 Terra move,gpt-5.6-terra2.00/12.00 →us.openai.gpt-5.6-terraUS Geo CRIS 2.20/13.20,price_comparison: "unfavorable",model-card-openai-gpt-56-terra.htmlread 2026-11-09; the first cut quoted a parity price forus.openai.gpt-5.5, whose card publishes In-Region only), droppedweb_searchdisclosed incapability_gaps[];fixtures/README.mdcounts 4 cases. Tests: five cases intests/test_runtime_default_review_round3.py, including a corpus-wide check that no fixture records a Responses-only selector on a same-API Mantle row. Engine half (agent-advisormodules) stays on fix(aws-startup-advisor): agent-advisor defaults to bedrock-runtime, selects Mantle only by a verified selector, adds Runtime Responses/Chat/Messages paths #588.Gates run in the worktree on
4f861123(2026-11-09):llm-to-bedrockpytest 672 passed / 0 failed (655 at the previous round);agent-advisorpytest 570 passed (unchanged, noagent-advisoredits); plugin tests 1,532 passed / 1 failed (test_model_id_lint::test_real_plugin_passes, caused by an untracked local.venv, not by this branch);scripts/validate-artifacts.py --fixturesPASS 103 artifacts / 0 errors;tools/sync-vendored.py --checkPASS 64 byte-identical;lint:skill-refsPASS;git diff --checkclean; markdownlint 0 issues in tracked files (mise run cifails only atlint:mdon the same untracked.venv).mise run build(gitleaks) not re-run: pre-existing upstreamf8d09fd1finding (see Limitations).Limitations
agent-advisorstill recommends by catalog order and does not know themantle_selectorrule or theruntime_messages/same_model_runtimeidentities, so its handoff can still name a Mantle path for a source that this PR's design guidance would route to Runtime. Deferred to PR 3 so this PR stays withinllm-to-bedrockand the gcp/azure design/generate guidance.mantle_messagesplan is accepted by the design vocabulary but executes through the existing Converse dispatch in the evaluator and preflight; only the rewriter targets Mantle. This is deliberate for this PR and is called out inllm-to-bedrock/SKILL.mdonly by the pass-through rule, not by an explicit sentence.httpx.Authtoken refresher on a long-lived client would be a later improvement.--runtime-apiargparse default is stillconverse. The fail-closed guarantee is enforced at plan resolution (normalize_api_pathand theSKILL.mdcontext block); for the probe it is procedural (SKILL.mdB4 says never to omit the flag).ai-openai-to-bedrock.mdand gcpdesign-ai.mdsay the selector enum is defined in the policy file "and only there"; the threeschema-design-aws-ai.mdcopies also enumerate the values as vocabulary. The wording should say "canonical definition". Minor; happy to fix in this PR on request.mise run build'ssecuritystep fails on a pre-existing gitleaks finding in upstream commitf8d09fd1(plugins/aws-startup-advisor/tests/test_bedrock_migration_assessment_discover.py:118, a generic-api-key match on the literalclaude-3.5-sonnet). That file is not touched by this branch;gitleaks protect --stagedon this change set is clean.Update after merging main
Head
cabaff87. Main and the updated #583 (heade17c00ac) were merged into this branch, so the base, tip and commit-count figures in "Stacked on #583" and "Scope" describe the earlier revision. The only conflict was the generated OpenAI model catalog, which main now renders fromskills/shared/ai/models/bedrock-model-registry.json; it was resolved through the registry and #583's port of its corrections. Because main changed the agent-advisor selectors and shared catalogs after this PR's last revision, the earlier statement thatgit diff 8c37ef0f..HEAD -- skills/agent-advisor skills/shared/ai/modelsis empty no longer holds for the merged branch; this PR's own commits still do not edit agent-advisor (that is #588). The full plugin CI (mise run ci) passed at this head (1,857 tests). Landing order for the stack is #583, then #585, then #588.Type of Change
Checklist
python3 tools/validate.pyStartup Advisor review checklist
mise -C plugins/aws-startup-advisor run ciand all checks pass.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.