Repository navigation
feat(aws-startup-advisor): measure Anthropic usage and show measured spend and volume in Q3 and Q7 in bedrock-migration-assessment - #605
Conversation
…spend and volume in Q3 and Q7 in bedrock-migration-assessment Stage 3a.1b-5a of #389. Anthropic usage reader (discover-anthropic-api.md, scripts/anthropic-usage.py), a stdlib script like openai-usage.py so the agent never opens the Admin key file. Checked against Anthropic's current Usage and Cost API reference: - GET only, to usage_report/messages and cost_report on api.anthropic.com, with x-api-key and anthropic-version. Never grouped by account, API key, or service account. The consent says the Console Admin key has no narrower usage-only scope. - The probe prints each workspace's spend and writes nothing. The default workspace has a null workspace_id, which workspace_ids[] cannot select, so the script filters rows by workspace_id; cost_report has no workspace filter at all. Rows of unselected workspaces are dropped before writing, and the consent says so. - Capture deletes earlier files first, records result, and deletes the key file on every exit; a 404 probe (Claude Enterprise or Claude Platform on AWS) exits ORG_NOT_SUPPORTED and deletes it too. summarize reads only rows the manifest marks ok. - Cost amounts are cents. cost_status follows the existing convention; cost_report reports only the standard and batch tiers (Priority Tier is billed separately), so other tiers make it partial_window and are named in uncosted_service_tiers. The API reports no request counts, so the profile omits them rather than writing 0. Peaks count uncached input and cache writes, not cache reads. - merge adds anthropic/anthropic_api models with provenance usage_api, keeps raw_model verbatim, and refuses when Anthropic is a gateway upstream. Discover Step 2c runs each reader whose source the profile has, OpenAI then Anthropic, each with its own consent. The preview takes one --usage per profile and reports each source plus totals (preview_version 2); the total spend is null when any source's spend is unknown. Clarify: Q3 and Q7 show the measured spend and volume from migration-preview.json and mark the matching option, but the user still answers (chosen_by user), so Q7 stays never-skipped. Q3 now asks about "your current AI providers" instead of "OpenAI or Gemini". The usage-derived Mantle signals in #585 are not added here; they belong after #585 merges.
herosjourney
left a comment
There was a problem hiding this comment.
Reviewed: head b9209bbe (base 938d20db), 17 files (+1361/−30) — bedrock-migration-assessment, single plugin tree. Draft.
Ran: git merge-tree --write-tree upstream/main (clean; upstream is 28000d61, so the branch is behind and not conflicting); JSON parse of the five new capture fixtures (all valid); 42 passed in test_bedrock_migration_assessment_anthropic_usage.py and test_bedrock_migration_assessment_usage_preview.py; two mutation probes below. Primary sources read 2026-11-10: Usage and Cost API, Get Messages Usage Report, Get Cost Report. GitHub build, startup-advisor, CodeQL, and Python analysis passed on this head.
Did not run: root mise run build; the full plugin ci (1,478 tests); no live Anthropic call.
Cross-plugin parity: Not applicable. One plugin tree. No vendored skills/shared file changed.
Outside skills/ mirrored by hand: tests, fixtures, and README.md exist only in this tree.
Change narrative: After Discover, measure the selected Anthropic workspaces and show that spend and volume in Q3 and Q7, with the user still answering. Mechanism: a stdlib GET-only reader beside the OpenAI reader, plus preview totals. Surfaces: discover-anthropic-api.md, anthropic-usage.py, Discover step 2c, preview, Clarify Q3/Q7. Body matches the diff. The OpenRouter reader, the newer Analytics API, and usage-derived Mantle signals are named as out of scope. The legacy endpoints are still documented as supported.
| Axis | Stance | One line |
|---|---|---|
| Parsimony | clear | Same shape as openai-usage.py: one script is the only reader of the key file. |
| Completeness | finding | day_input_tokens counts cache reads; the per-minute peak does not. |
| Security | finding | A network error during capture leaves the Admin key file on disk. |
| Alternatives | clear | The newer Analytics API is already named, and the docs still support these endpoints. |
| Contradictions | finding | The discovery doc and day_input_tokens disagree. |
| Overlap | clear | Cost-status values match the OpenAI reader. No second Anthropic reader. |
| Cross-plugin parity | clear | Single tree. |
| Propagation & data | clear | Cents, tier set, limits, and null default workspace match the API reference. cost_status stays in complete / partial_window / cost_unavailable. |
Verified against the cost reference: amount is cents ("123.45" means $1.23); group_by is only description and workspace_id; there is no workspace_ids parameter; service_tier on a cost row is batch or standard. The usage reference: default workspace is workspace_id: null; limits are 31 / 168 / 1440 for 1d / 1h / 1m; grouping by account_id, api_key_id, and service_account_id exists and the script rejects those groups (ValueError on api_key_id, probed). A constructed bucket with 10 uncached, 1000 cache-read, and 5 cache-write tokens produced a per-bucket peak of [15, 3] and a day total of [1015, 3].
Blocking
None.
Should-fix
-
[Contradictions] The busiest-day input counts cache reads. —
scripts/anthropic-usage.py:254anddiscover-anthropic-api.md:136d[0] += uncached + read + c5 + c1
The peak's input counts uncached input and cache writes, not cache reads.
The per-bucket peak used for the per-minute rate excludesread. The day total used forday_input_tokensand for choosing the busiest day includes it. Probe: 1000 cache-read tokens became part of the day total and not the per-minute peak.
Consequence: a cache-heavy day is reported as the peak input, and Q7's volume story can cite that day.
Fix:d[0] += uncached + c5 + c1. Regression: extendtest_peak_input_counts_cache_writes_but_not_cache_readsto assertday_input_tokensequals uncached plus both cache writes. -
[Security] Capture does not delete the Admin key when the request raises
URLError. —anthropic-usage.py:222anddiscover-anthropic-api.md:13delete_key(args.key_file)sits after the request loop, and theexceptclauses coverKeyRejected,RuntimeError,Forbidden, andOrgNotSupported.
The script deletes it after every capture, successful or not
Probe:openerraisingURLErrorleft the key file in place. The key was not printed. The phase also tells the agent torm -fthe file when a step ends, so a run that continues still removes it. A crashed run does not.
Consequence: the Admin key file remains in$MIGRATION_DIRafter a timeout or DNS failure.
Fix: wrapcmd_captureintry/finally: delete_key(args.key_file). Oncmd_probe, delete the key in anexceptforURLErrorand then re-raise; a successful probe must still leave the file for capture. Regression: a fakeURLErrorasserts the key file is gone and nothing in stdout containssk-ant-admin01-.
Heads-up
The reference examples send anthropic-version: 2026-10-21. This script and test_capture_is_get_only_scoped_and_never_leaks_the_key pin 2023-06-01. The header is documented as optional. I could not show that 2023-06-01 drops inference_geo, speed, or the cache split.
The minute-window day picker (_busiest_on_demand_day) still sums cache reads. The discovery doc does not define that picker as peak input.
Alternatives
None worth raising. The body already keeps the legacy endpoints because they remain supported and the OpenAI reader uses the same family.
Verdict
Comment — consent, the GET allowlist, workspace filtering, and the cents conversion match the API reference. The busiest-day total counts cache reads, and a network error leaves the Admin key file. Code owners on plugins/aws-startup-advisor decide.
|
Problem, Fix, and Scope are filled in. A change type is selected and all four Startup Advisor author confirmations are complete. The author checklist check now passes. CI and reviewers still need to verify the change. |
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Reviewed a3937bb5be674f36b5d976079d1bf172d0d49fe4 with OCR delegation, a repository review, and one independent general review.
Please address the three additional issues inline:
- Preserve each provider's activity period when displaying combined spend. A 30-day source plus a 2-day source must not be described as a combined 2-day measurement.
- Carry excluded-tier cost warnings through the existing usage-baseline/report contract. A full-window Priority Tier capture must still disclose the missing dollars in the downstream comparison.
- Handle saved version-1 previews on resume. They have neither
sourcesnortotal_tokens; Q3/Q7 need a compatible fallback or a refreshed preview before using those fields.
I also reproduced the existing key-cleanup concern and daily-peak input concern at this head. Those remain in their original threads: verify cleanup after transport failures and assert both bucket and daily peak token pools when fixing them.
Validation: 82 focused tests passed on the original source, git diff --check passed, and offline probes exercised capture errors, aggregate windows, excluded-tier metadata, and an actual base-generated version-1 preview. No live API or deployed workflow was run. The current author-checklist check is also failing; its existing bot comment lists the required description updates.
…ache reads out of the busiest day, and keep measured usage honest across sources and old previews Review findings on #605, each reproduced first. herosjourney: - A URLError during capture or probe skipped the key-file deletion, so a timeout or DNS failure left the Admin key on disk. capture now deletes it in a finally block, and probe deletes it on any failure before re-raising; a successful probe still keeps it for capture, and a capture call missing --workspace keeps it so the corrected call works. openai-usage.py had the same gap and gets the same fix. Both readers document the network-error exit. - The busiest day and day_input_tokens counted cache reads while the per-minute peak did not. Day totals now count uncached input and cache writes, the minute-sample day uses the same basis, and day_to_average_ratio divides by that same total. leon1418: - measured_usage reported the shortest source's active_days for the combined spend, so $3,000 over 30 days plus $20 over 2 days read as "$3,020 over 2 days". The aggregate no longer has active_days or partial_window; Q3 and Q7 give each source its own period. - A full-window capture with Priority Tier usage had cost_status partial_window but no capture_warnings entry, and llm-to-bedrock's baseline and report read capture_warnings, not uncosted_service_tiers. The profile now adds "cost_report does not include priority tier spend; this total excludes it", as the gcp-to-aws Anthropic reader does, so the downstream comparison is qualified. - A migration-preview.json written by the version on main (preview_version 1) has no sources or total_tokens, and resuming at Clarify does not rewrite it. Step 0 now re-runs Discover Step 2d first when the preview is not version 2; that reads only the files already in the run directory, scans nothing, and calls no API. fixtures/bma-preview-v1 is a preview generated by main.
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Re-reviewed 1255fd3d0a843e00297f2605adcfc9d56a1c8d9d with OCR delegation, a repository review, and one fresh independent general reviewer. All five previous concerns are verified fixed, and this review found no additional substantiated issues.
The fixes cover both readers' transport-error key cleanup, consistent daily/minute peak token pools, per-provider activity periods, excluded-tier cost warnings, and refreshing version-1 previews before Clarify. Validation: 91 focused tests passed on the original source, git diff --check passed, and offline probes checked failure/retry behavior and actual preview regeneration with the run ID and saved answers preserved. Consumer/report behavior was checked in source; no live API or deployed LLM workflow was run.
I replied with evidence and resolved only the three conversations I originally opened. The other reviewer's two conversations remain open for that reviewer. Approval is recommended for the user to submit; this is a COMMENT review, and the earlier CHANGES_REQUESTED review has not been dismissed.
herosjourney
left a comment
There was a problem hiding this comment.
Verified at 1255fd3: busiest-day input now counts uncached plus both cache writes (same basis as the per-minute peak), and the Admin key file is removed on any failure in probe and capture for both the Anthropic and OpenAI usage readers, with URLError regression tests. 69 usage tests pass on the head. Approving.
… bedrock-migration-assessment (aws#619) * feat(aws-startup-advisor): measure Anthropic usage and show measured spend and volume in Q3 and Q7 in bedrock-migration-assessment Stage 3a.1b-5a of aws#389. Anthropic usage reader (discover-anthropic-api.md, scripts/anthropic-usage.py), a stdlib script like openai-usage.py so the agent never opens the Admin key file. Checked against Anthropic's current Usage and Cost API reference: - GET only, to usage_report/messages and cost_report on api.anthropic.com, with x-api-key and anthropic-version. Never grouped by account, API key, or service account. The consent says the Console Admin key has no narrower usage-only scope. - The probe prints each workspace's spend and writes nothing. The default workspace has a null workspace_id, which workspace_ids[] cannot select, so the script filters rows by workspace_id; cost_report has no workspace filter at all. Rows of unselected workspaces are dropped before writing, and the consent says so. - Capture deletes earlier files first, records result, and deletes the key file on every exit; a 404 probe (Claude Enterprise or Claude Platform on AWS) exits ORG_NOT_SUPPORTED and deletes it too. summarize reads only rows the manifest marks ok. - Cost amounts are cents. cost_status follows the existing convention; cost_report reports only the standard and batch tiers (Priority Tier is billed separately), so other tiers make it partial_window and are named in uncosted_service_tiers. The API reports no request counts, so the profile omits them rather than writing 0. Peaks count uncached input and cache writes, not cache reads. - merge adds anthropic/anthropic_api models with provenance usage_api, keeps raw_model verbatim, and refuses when Anthropic is a gateway upstream. Discover Step 2c runs each reader whose source the profile has, OpenAI then Anthropic, each with its own consent. The preview takes one --usage per profile and reports each source plus totals (preview_version 2); the total spend is null when any source's spend is unknown. Clarify: Q3 and Q7 show the measured spend and volume from migration-preview.json and mark the matching option, but the user still answers (chosen_by user), so Q7 stays never-skipped. Q3 now asks about "your current AI providers" instead of "OpenAI or Gemini". The usage-derived Mantle signals in aws#585 are not added here; they belong after aws#585 merges. * fix(aws-startup-advisor): delete the Admin key on any failure, keep cache reads out of the busiest day, and keep measured usage honest across sources and old previews Review findings on aws#605, each reproduced first. herosjourney: - A URLError during capture or probe skipped the key-file deletion, so a timeout or DNS failure left the Admin key on disk. capture now deletes it in a finally block, and probe deletes it on any failure before re-raising; a successful probe still keeps it for capture, and a capture call missing --workspace keeps it so the corrected call works. openai-usage.py had the same gap and gets the same fix. Both readers document the network-error exit. - The busiest day and day_input_tokens counted cache reads while the per-minute peak did not. Day totals now count uncached input and cache writes, the minute-sample day uses the same basis, and day_to_average_ratio divides by that same total. leon1418: - measured_usage reported the shortest source's active_days for the combined spend, so $3,000 over 30 days plus $20 over 2 days read as "$3,020 over 2 days". The aggregate no longer has active_days or partial_window; Q3 and Q7 give each source its own period. - A full-window capture with Priority Tier usage had cost_status partial_window but no capture_warnings entry, and llm-to-bedrock's baseline and report read capture_warnings, not uncosted_service_tiers. The profile now adds "cost_report does not include priority tier spend; this total excludes it", as the gcp-to-aws Anthropic reader does, so the downstream comparison is qualified. - A migration-preview.json written by the version on main (preview_version 1) has no sources or total_tokens, and resuming at Clarify does not rewrite it. Step 0 now re-runs Discover Step 2d first when the preview is not version 2; that reads only the files already in the run directory, scans nothing, and calls no API. fixtures/bma-preview-v1 is a preview generated by main. * feat(aws-startup-advisor): measure OpenRouter usage and BYOK spend in bedrock-migration-assessment Stage 3a.1b-5b of aws#389. Stacked on aws#605. OpenRouter usage reader (discover-openrouter-api.md, scripts/openrouter-usage.py), a stdlib script like the OpenAI and Anthropic readers so the agent never opens the management key file. It keeps every field the gcp-to-aws OpenRouter reader produces, checked against OpenRouter's current API reference: - GET /credits (the probe), /workspaces, /activity, /key, and /analytics/meta, plus one read-only POST /analytics/query (three queries: BYOK classification, volume and fees, hourly peak). The script refuses any other endpoint, per-key or per-user parameter, grouping, or query body other than metrics by model, hourly at most, and a workspace filter. The analytics queries ask only for metrics /analytics/meta lists, so a renamed metric makes that field unknown instead of failing. The consent names the POST and says a management key has no read-only scope. - One workspace or the whole account, filtered (never grouped) on both /activity and the analytics queries. - Capture deletes earlier files, records result, and deletes the key on every exit, including a network error; a revoked key on /credits or /activity is key_invalid_mid_run and a failed /activity is all_usage_failed. A failed analytics row does not stop the capture. - summarize joins analytics on model_permaslug and rolls up to the model id: byok_passthrough, mixed, no_byok, uncertain, or unknown, with the gcp-to-aws thresholds and profile-level rule. It adds byok_fees_usd, request counts, p95 latency and throughput, web-search cost, guardrail counts, and an hourly peak (day peak when the hourly file is missing or truncated), and warns when /activity is empty but lifetime usage is not. - New: /activity now reports byok_usage_inference, so the profile records byok_inference_cost_usd (billed by the user's own provider keys, not in monthly_cost_usd) and warns when it is above zero. - merge applies the scan's OpenRouter prefix rules (openai/, anthropic/, google/ by serving provider) and adds models to the source and the gateway upstream only when the gateway's upstreams are observed; otherwise the profile is unchanged and the models are named. Discover Step 2c runs the reader when the profile has an OpenRouter gateway. The preview adds OpenRouter tokens, adds BYOK inference to the total, and leaves out OpenRouter rows a direct profile also counts unless they are no_byok (the llm-to-bedrock name rule), listing them in overlap_excluded; Q3 and Q7 say so. * fix(aws-startup-advisor): join OpenRouter analytics by permaslug, keep unmeasured values unknown, keep router fees, and merge across every gateway Review findings on aws#619 (leon1418), each reproduced first. - Analytics replies label the model dimension for display ("OpenAI: GPT-4.1"), and only filters take the permaslug, so joining a grouped reply to /activity's model_permaslug matched nothing. Each analytics row now sends one query per permaslug in /activity, filtered by it and never grouped, and stores the replies keyed by that permaslug (at most 50 models; the manifest names the rest). A truncated reply is left out and named. - An unoffered or unreported metric became 0, so a model with BYOK spend could be certified no_byok and its provider bill counted twice. A permaslug is classified only when both byok_usage and openrouter_usage were offered and reported; volume fields that were not measured are null with a warning, never 0. - Leaving out an overlapping OpenRouter row also dropped OpenRouter's own charge with no warning. The preview keeps a measured router fee on a byok_passthrough row, reports every dropped charge in overlap_excluded and spend_excludes_usd, says when a mixed row drops independent usage, and reports only the BYOK inference it actually includes (byok_inference_included_usd). Q3 reads those fields. - The merge refreshed only the OpenRouter upstream after changing a shared source, so a valid OpenRouter plus LiteLLM profile failed validation. Every gateway upstream that references a changed source is rebuilt before validation. - The merge only lowercased Gemini names. It now applies the scanner's Step 4 Gemini rule (drop version suffixes, keep -lite, -thinking, -image, -live, -tts), records the OpenRouter id in the evidence, and keeps it as raw_model for a new model. * fix(aws-startup-advisor): merge the OpenRouter models that can be placed, read the date__hour field, and describe left-out rows accurately Review findings on aws#619 (herosjourney), each reproduced first. - One model with an unplaceable prefix (meta-llama/, mistralai/, ...) blocked the whole merge. The merge now adds every model it can place and lists the rest under not_merged; it exits 5 only when none can be placed. - The analytics reference's example time field is date__day, so the hourly field is date__hour, which the hour lookup missed; two permaslugs in the same hour were then not added up. The lookup now reads hour or any date__ field. - The Q3 and preview wording said a direct provider also counts every left-out row, which is false for a mixed row's OpenRouter-funded usage. Both now say the row is left out so its BYOK traffic is not counted twice, and a mixed row's note says its OpenRouter-funded usage is not in the total. - An error body whose error is a string raised AttributeError and aborted the capture. All three readers now report it as the row's failure. - A rejected key on the optional /key row is recorded skipped, as the reference says. - SKILL.md's Discover output now names all three usage profiles. The preview's router-fee and mixed-row handling (Should-fix 1) was already fixed in 6802fd4; a test now runs the reviewer's probe (direct $100, a byok_passthrough row with a $4 fee, a mixed row) and asserts $104 and the mixed-row warning.
Problem
bedrock-migration-assessmentcould measure only OpenAI usage (#592). An app that calls Anthropic directly got no measured spend or volume, so Clarify's Q3 (spend) and Q7 (volume) relied on the user's guess. This is stage 3a.1b-5a of #389.Fix
Anthropic usage reader (
discover-anthropic-api.md,scripts/anthropic-usage.py). Likeopenai-usage.py, a shipped stdlib script is the only code that reads the Admin key file. Checked against Anthropic's current Usage and Cost API, usage report, cost report, and Admin key references.GETonly, tousage_report/messagesandcost_reportonhttps://api.anthropic.com, withx-api-keyandanthropic-version. Never grouped by account, API key, or service account. A Console Admin key has no narrower usage-only scope, and the consent says so.workspace_id: null, whichworkspace_ids[]cannot select, andcost_reporthas no workspace filter. So the script filters rows byworkspace_idand drops other workspaces' rows before writing anything. The consent says the replies can include those rows.ORG_NOT_SUPPORTED.capturedeletes earlier files first and recordsresult.summarizereads only rows the manifest marksok.cost_statususes the existingcomplete/partial_window/cost_unavailablevalues.cost_reportcovers only thestandardandbatchtiers (Priority Tier is billed separately), so usage in any other tier makes itpartial_window,metadata.uncosted_service_tiersnames those tiers, andcapture_warningssays the total excludes their spend, sollm-to-bedrock's comparison is qualified.llm-to-bedrockreads. The per-minute peak, the busiest day, andday_input_tokenscount uncached input and cache writes, not cache reads. The API reports no request counts, so the profile leaves them out rather than writing 0.anthropic/anthropic_apimodels withprovenance: "usage_api", keepsraw_modelverbatim, and refuses when Anthropic is a gateway upstream.Discover, preview, and Clarify.
preview_version2) takes one--usageper profile.measured_usagelists each source with its own active period and adds total spend and tokens; total spend isnullwhen any source's spend is unknown. The Discover display shows spend per provider and the total token volume.main(preview_version1) first re-runs Discover Step 2d from the files already in the run directory. It scans nothing and calls no API.migration-preview.json, give each source its own period, and mark the matching option "(measured)". A source with a short window is labeled as not scaled to a month. The user still answers, recorded withchosen_by: "user", so Q7 stays never-skipped. Q3 now asks about "your current AI providers" instead of "OpenAI or Gemini", since it can show Anthropic spend.Review fix that also applies to the OpenAI reader:
openai-usage.pyhad the same gap, where a network error left the key file on disk. It gets the same fix.Scope
bedrock-migration-assessmentonly. New:references/phases/discover/discover-anthropic-api.md,scripts/anthropic-usage.py,fixtures/bma-anthropic-usage/(a capture with known totals, peaks, and costs), andfixtures/bma-preview-v1/(a preview generated bymain). Changed:discover.md,discover-openai-api.md,clarify-ai-only.md,SKILL.md,shared/schema-preferences.md,scripts/migration-preview.py,scripts/openai-usage.py, the plugin README, and thebma-ai-only-clarifyfixture's Q3 prompt.test_bedrock_migration_assessment_anthropic_usage.py;test_bedrock_migration_assessment_usage_preview.pyandtest_consent_disclosure_audit.py(now 8 discovery sub-files) updated.gcp-to-aws,azure-to-aws, or anyskills/shared/file.Type of Change
Select at least one.
Startup Advisor review checklist
By submitting this PR, I confirm that:
mise -C plugins/aws-startup-advisor run ciand all checks pass.Known limitations and what is not in this PR: the OpenRouter reader (3a.1b-5b); the usage-derived Mantle signals #585 adds to the gcp-to-aws and azure-to-aws readers, which belong in a follow-up once #585 merges; and the newer Usage and Cost Analytics API (the legacy endpoints used here remain supported, and the other skills' readers use them). The script sends
anthropic-version: 2023-06-01, as the other Anthropic readers do; the header is optional, and the current reference examples use2026-10-21. No real Anthropic API call was made: the script tests use a fake API.Validation:
mise -C plugins/aws-startup-advisor run cipasses (1,504 plugin tests and 225 subtests; 311 telemetry tests). Rootmise run validate,python3 tools/validate.py, gitleaks, and the consent-disclosure audit pass. The new tests fail onmain, and the review-fix tests fail on a3937bb (the head before these fixes). Script tests run the capture against a fake API: GET only, both headers, no forbidden grouping, pagination, a 429 retry,workspace_ids[]only for named workspaces, unselected rows dropped (including with the default workspace selected), a revoked key, a network error during probe and capture, a 403 that fails one row, a failed usage row that ends the capture, probe 401/403/404, cost failed versus empty, an uncosted tier on a full window, a full-window plus partial-window pair in the preview, a cache-read-heavy day, and no key in any output, file, or URL. Pre-review probes (schema boundaries, google-genai and framework matrices) report 0 problems. Live headless runs with Claude Code and this branch (no real API calls): declining both consents on an OpenAI-plus-Anthropic repo 3/3; a resumed run where Q3 and Q7 are asked with the measured values and the user's answers are recorded 3/3; a resumed run with a version-1 preview, which is refreshed without a scan or API call before Q7 shows the measured volume 3/3; existing scenarios (OpenAI usage decline, fast path, Specialized fast path, resume after Discover, mixed confidence, foreign run directory, two routing scenarios) 3/3 each. No run opened.env.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Refs #389