Repository navigation
feat(aws-startup-advisor): heroku-to-aws shared-data-store cohorts, estate workbook export, contract reconciliation - #609
Conversation
…state workbook export, contract reconciliation Three things a reviewer of a large Heroku estate needs that the skill did not produce, so the SA built them by hand from the skill's JSON: 1. Shared data stores. Live discovery now records attached_apps per add-on (from addons.json attachments[]), and assemble rule 10 derives shared_data_stores.cohorts: apps linked by the same Postgres/Redis/Kafka add-on. It is a reporting label beside the flat resource list - no edges, no ordering, no cluster_id on resources - and the Philosophy bullet says so. The report gets a conditional shared-data-stores section. 2. Estate workbook. scripts/emit-estate-workbook.py copies the run's inventory, design and estimate into heroku-estate.xlsx (stdlib-only xlsx writer; one tab per resource family, Summary, Shared data stores, Cost estimate). Runs fail-open at the end of Discover (customer can review what was found before any AWS design exists) and is refreshed after Estimate and Generate. Excludes config var names/values, hostnames, UUIDs and VPC ids, matching discovery's posture; tests assert that. 5 tests, bandit clean. 3. Contract reconciliation. Clarify Q16 (asked only when no billing data) captures Enterprise order-form line totals into preferences.commercial.heroku_contract. The Estimate writes current_costs.contract with per-category contract vs discovered vs gap and a mechanism-naming gap_explanation, plus a like-for-like discovered figure (minus Deferred add-ons). The report renders the table under exec-costs with a fixed caveat and never uses contract-minus-AWS as the headline. Ported from awslabs/startups#365 (closed in favour of this PR).
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Reviewed c6b9b23. Requesting changes for five P2 issues:
- Run the Discover workbook command in the main-window handoff; the dispatched
rwassembler has no shell. - Preserve Fargate plus ALB mappings and copy the supported Fargate/EKS sizing fields into the workbook.
- Include Q16 in the full interview batch and presentation so an order form can reach the estimate.
- Keep pending downstream mappings and costs out of a rediscovery workbook while preserving the old artifacts.
- Derive and display the separate scope-matched comparison across the estimate, report and workbook; retain the full-spend comparison and its caveats.
The inline comments give the triggers and corrections. There is also one lower-priority P3 input-decoding issue in the writer's fail-open handling.
Validation: 39 focused tests passed on original source, git diff --check passed, and 17 synthetic report-validator probes on the audited source copy behaved as expected. Those probes confirm that the new optional report content is not enforced by the validator. No live Heroku migration or Excel UI run was performed.
…book, cohorts and reconciliation - Discover: the estate workbook export moves from the dispatched rw assembler (no shell) to discover.md's main-window Handoff, same fail-open behaviour. - Clarify: Q16 joins Batch 3 membership and the presentation prompt under its no-invoice condition, so an order form can actually reach the Estimate. - Estimate/report/workbook: cost_comparison.like_for_like_comparison is mandatory whenever any add-on is Deferred (AWS vs scope-matched baseline, difference, pct, deferred contribution); the report renders it under the full-baseline row with a data-cost-key anchor; the workbook copies it. - Workbook: one-to-many source mappings keep the compute/data-store entry as the target and list the rest (ALB) as associated services; Fargate and EKS sizing fields are shown; aws-design.json and estimation-infra.json are used only when .phase-status.json marks their phase completed (Discover re-entry leaves stale files behind); UnicodeDecodeError is caught alongside JSON and OS errors. Tests cover each. - Schema: attached_apps added to the addon config fenced examples so the artifact validator admits it (found by probing, same class as aws#610's issue). Review findings: leon1418 on PR aws#609.
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Follow-up review of 803af2f completed with OCR, an independent reviewer, and the convergence evidence gate. The Discover handoff, workbook target/sizing, pending-phase filtering and UTF-8 handling are verified fixed; those four conversations are resolved. Q16's batch wiring is also fixed.
Two P2 concerns remain in their existing threads:
- Q16 must treat an absent/null billing profile as no invoice. I missed this producer/condition mismatch in the earlier review.
- The matched comparison must use the same priced deferred contribution for its baseline subtraction and excluded-amount disclosure; two definitions still request list prices.
Validation: 41 focused tests passed on original source, eight original-writer cases passed, and git diff --check passed. The independent reviewer also ran the seven authored workbook tests and additional producer-shape probes on audited copies. No live Heroku migration or Excel UI run was performed. The existing changes-requested review remains outstanding.
…ferred contribution; Q16 treats an absent billing profile as no invoice - Estimate: the scope-matched baseline subtracts current_costs.deferred_resource_cost (Part 1b: what the selected baseline actually counted for Deferred add-ons), and like_for_like_comparison.deferred_contribution_monthly is that same number - never list prices. Deferred add-ons the baseline cannot attribute stay in the baseline and are listed in unattributed_deferred_services with a note. - Clarify Q16: fires when billing_profile is absent, null, or available != true (Discover omits the section when no billing file exists); skipped only when available == true or the user answers none. Review follow-up: leon1418 on PR aws#609.
…measured-data sizing basis Large Heroku estates carry Postgres databases on plans chosen years ago that hold a small fraction of their allocated storage. Sizing AWS from the plan name copies that waste into the AWS estimate, and the customer - who usually knows they are over-provisioned - distrusts the comparison. - Clarify Q6d (fires only when some database holds <= 25% of its plan's storage, shown as a table first): size AWS from measured data with 2x headroom (recommended) or from the current plan. Writes preferences.data.database_sizing_basis. - Design 2B step 2a: with "measured", the sizing row is looked up by required storage within the same tier family instead of by plan name - still a direct table lookup, keeps HA/compliance semantics, never applied to Advanced-tier add-ons. Records sizing_basis / sized_as_plan / source_plan in aws_config and warns per app. - Estimate Part 1: current_costs.rightsizing_candidates[] (plan storage, data, utilization, fitting plan in the same family, monthly saving) and rightsizing_monthly_saving_usd. The Heroku baseline is left as what they pay today; the saving is labelled as Heroku-side and available before migration. Followers and Advanced-tier add-ons are skipped; Redis is not assessed (plan memory, not used memory, is what discovery captures). - Report: "Right-size before you move" table under exec-costs. - Estate workbook: Right-sizing tab copied from the Estimate; test updated. Stacked on the estate-workbook branch (aws#609) because the workbook tab lives in scripts/emit-estate-workbook.py.
herosjourney
left a comment
There was a problem hiding this comment.
Reviewed: head c6b9b23 (base 00840a0), 11 files (+741/−1). Skill touched: heroku-to-aws. Tree: plugins/aws-startup-advisor only.
Ran:
pytest scripts/test_emit_estate_workbook.py: 5 passed.emit-estate-workbook.pyon 14 constructed run directories, each output opened with openpyxl 3.1.5 and every XML part parsed. Inputs covered: a realistic inventory shaped by the schema; inventory only and the full inventory + design + estimate-with-contract chain; an empty estate,{}and[]; formula-leading, unicode and comma names; control characters,NaN/Infinityand a lone surrogate; duplicate names, a 40k-character cell and wrong-typed fields.- Two mutation probes on a copy of the script.
- A contract trace of every field the script reads or the diff writes, against the schema, the phase files and
scripts/artifact-contracts.json. - An
_exectier trace againstINTERPRETER.md. - A producer sweep for new fields across main, this head and the other open heroku-to-aws PRs (#608, #610, #611, #612, #614, #615).
GitHub CI is green at head (startup-advisor runs mise run ci; CodeQL, check-frontmatter and the author checklist also pass). Merges cleanly into main per git merge-tree.
Did not run: mise run ci locally; any live Heroku CLI call; an end-to-end Discover → Generate run; opening the workbooks in Excel or LibreOffice; checking the heroku addons --all --json output shape against a primary source.
Cross-plugin parity: n/a. This repo has a single copy of skills/heroku-to-aws/, and no vendored files were touched. Outside skills/: nothing changed.
Change narrative: Three things get rebuilt by hand today in large Enterprise estates: which apps share a data store, a spreadsheet the customer will review, and the committed contract spend. The PR adds attached_apps and cohort grouping in Discover; a standard-library .xlsx writer run at Discover, Estimate and Generate; and a Clarify question (Q16) feeding a contract reconciliation in Estimate and new report sections. Body vs diff: it matches, except "Independent of #608". Only the pg_role dependency is disclosed; the environment_tier and apps_by_environment_tier dependencies are not (Should-fix 4).
| Axis | Stance | One line |
|---|---|---|
| Parsimony | advisory | Additive and in the repo's idiom (a fail-open writer copying emit-plan-json.py). The Discover-time run is the one wiring point that costs more than it returns (Alternatives 1). |
| Completeness | finding | Q16 can't be reached (Blocking 1). Discover's dispatched worker is told to run python3 with no shell (Should-fix 1). |
| Security | advisory | No config var names or values, hostnames or UUIDs reach the workbook (test plus probe). Formula-leading text is stored as inline strings with no formula element. Order forms are not named in Input Security (Nit 3). |
| Alternatives | advisory | Two below. |
| Contradictions | finding | The like-for-like rule conflicts with "Keep the full baseline" (Should-fix 3). The schema cites "rule 9" for what is rule 10 (Nit 1). |
| Overlap | advisory | The like-for-like step re-derives Part 1b's deferred_resource_cost (folded into Should-fix 3). |
| Cross-plugin parity | clear | Single tree; nothing to mirror. |
| Propagation & data | finding | shared_data_stores / attached_apps: schema updated first, every consumer swept. preferences.commercial: its home clarify-assemble.md was not updated (Blocking 1). environment_tier, environment_tier_source and apps_by_environment_tier have no producer on main or at this head; only #608 has one (Should-fix 4). |
The contract between the script and the phase files was checked and matches:
- Invocation: the command string is identical in
discover-assemble.md:65,estimate-assemble.md:386andgenerate-assemble.md:124, and matches argparse. - Inputs:
heroku-resource-inventory.json,aws-design.jsonandestimation-infra.jsonmatch the_producesof their phases. - Status lines:
ESTATE_OK | …/ESTATE_SKIP | reason=…match all three phase files. - Design join:
deferred[]is matched on(addon_name, addon_plan), consistent withdesign-mapping.md§ 2F.
Blocking
- [Completeness] Q16 is never asked and its answer has nowhere to land, so contract reconciliation can't run on any path.
clarify/clarify-interview.md:159
| Q16 — Heroku contract | \billing_profile.available == false` (no invoice data) | Billing data present, or user says "none" |`
Consequence:
- The batch table puts Batch 3 at "
Q11, Q12b, Q12–Q15" (:230), the batch rule says "Q12–Q15 always fire" (:234), and the Batch 3 template presents "[Present active questions Q11–Q15]" (:287). Q16 is active but in no batch, and questions are only presented by batch. - Even if asked,
clarify-assemble.mdStep 4 (the declaredpreferences.jsonshape perscripts/artifact-contracts.json) has nocommercialkey, sopreferences.commercial.heroku_contractis never written. - As a result,
estimate-cost-engine.md:151never fires,current_costs.contractis never produced, and the report table, caveat and workbook block are dead on every run. - The trigger also reads
available == false, while runs with no billing file carrybilling_profile: nullor no section (fixturesheroku-nonweb-scaling,heroku-workshop).
Fix:
- Batch 3 cell (
:230) →Q11, Q12b, Q12–Q16; rule at:234→ "Q16 fires when no invoice data";:287→[Present active questions Q11–Q16]. - Trigger → "
billing_profileabsent, null, oravailable == false". - In
clarify-assemble.mdStep 4, add"commercial": { "heroku_contract": { … } }(omitted when Q16 is answered none or defaulted) and"Q16"tosources.
Regression: a contract test asserting every question in the Applicability table appears in exactly one batch row (or the "Not in any batch" list), and that clarify-assemble.md names commercial.
Should-fix
-
[Completeness] Discover's workbook step tells a shell-less worker to run
python3.discover/discover-assemble.md:64–65.discover.mddeclares_exec: _agent: rw, andINTERPRETER.md:307/314gives that tier Read/Grep/Glob/Write/Edit only ("rwdeliberately excludes shell/Bash"). On dispatching hosts the Discover-time workbook, sold in the body as "review what was found before any AWS design exists", is never written, and the step invites a fabricatedESTATE_OK. Fix: remove step 11 from the assembler and add a main-window step todiscover.mdafterHANDOFF_OK, mirroringgenerate-assemble.md:123. Generate (rwx) and Estimate (inline) are fine as written. -
[Generated artifacts] The writer prints
ESTATE_OKfor a workbook that won't open.scripts/emit-estate-workbook.py:44–47,:75–76. Reproduced at head:
confidence_note: "bad\x01char\x0bhere"givesESTATE_OK, thensheet2.xmlis not well-formed and openpyxl raisesParseError.monthly_price_usd: NaN(accepted byjson.load) givesESTATE_OK, then openpyxl reports "Unable to read workbook … invalid XML".- A lone surrogate in an app name gives
ESTATE_SKIP, but leavesheroku-estate.xlsx.tmpin$MIGRATION_DIR(:123–131).
Free-text fields copied verbatim are the realistic carriers (confidence_note, gap_explanation, order-form notes pasted from Word, where \x0b is a soft line break). Mutation probe: replacing escape() with return text still passes all 5 tests, so escaping has no coverage. Fix: strip [\x00-\x08\x0b\x0c\x0e-\x1f\ufffe\uffff\ud800-\udfff] in escape(), emit an empty cell for non-finite floats, and tmp.unlink(missing_ok=True) on any exception during the zip write. Regression: a test with &<>", \x0b and NaN that asserts ESTATE_OK and that every worksheet part parses.
-
[Contradictions] The like-for-like rule pulls against "Keep the full baseline", and the figure has two names.
estimate/estimate-cost-engine.md:182–184says the like-for-like figure is "the only Heroku figure that is scope-matched … Part 3 uses it". Part 3 (:433) says "Keep the full baseline — do not subtract or hide the deferred resource's cost." Part 3 also stores the figure asheroku_monthly_like_for_like(:402), notdiscovered_like_for_like_monthly, and step 5 re-derives Part 1b'sdeferred_resource_costby a different method. Fix: "Writecost_comparison.heroku_monthly_like_for_like= Part 1 baseline −current_costs.deferred_resource_cost. Show it beside, never instead of,heroku_monthly_baseline; everymonthly_differencestays against the full baseline." The same goes for step 2 (:158–161): the discovered add-on bucket has no contract category to land on, so per-linediscovered_usdwon't sum todiscovered_monthly. Map add-ons toaddon_credits. -
[Propagation / PR honesty] Consumers of #608-only fields ship without #608, and the body says "Independent of #608".
generate/generate-report.md:64(metadata.apps_by_environment_tier) andemit-estate-workbook.py:202–203, 218(environment_tier,environment_tier_source) have no producer on main or at this head; only #608'sdiscover-assemble.mdrule 5a writes them. If this lands first, the Apps tab carries two always-blank columns and Summary shows every app as "(not inferred)". Fix: state the dependency and land #608 → #609, or move these consumers into #608.
Nits
references/shared/schema-discover-heroku.md:73: "written by assemble rule 9" should read "rule 10" (discover-assemble.md:46).scripts/emit-estate-workbook.py:331–340: the Cost estimate tab omitsheroku_monthly_like_for_likeand the per-linediscovered_usd/gap_usd, so the workbook and report show the contract differently.SKILL.md:145Input Security lists "billing exports" but not the Enterprise order form that Q16 now invites. Add it to the untrusted-file list.
Heads-up (out of diff or pre-existing, not blocking)
- The new test file is not run by CI. The
mise run citesttask runspytest tests -qfrom the plugin root, soskills/heroku-to-aws/scripts/test_*.pyis never collected (the same is true of the existingtest_validate_heroku_migration_report*.py). Green CI says nothing about this script. - Unknown whether cohorts can be detected. Whether
heroku addons --all --jsonemitsattachments[]was not verified. The replay fixturefixtures/heroku-live-capture/live-capture/addons.jsonhas noattachmentskey, so no fixture exercises cohorts. Please link a captured sample or the CLI source. - A 40,012-character string is written unchecked; Excel's cell limit is 32,767. openpyxl reads it; Excel's behaviour is not verified.
- Landing order: #611 is stacked on this PR. Conflicts with #610 in
discover-assemble.mdandgenerate-report.md, and with #612 ingenerate-report.md; shares files with #608 and #614 without conflict. Suggested order: #608 → #609 → #611, then rebase #610 and #612.
Alternatives (max 2)
- Write the workbook at Estimate and Generate only. This removes the shell-less wiring point (Should-fix 1); the cost is no workbook before Design. Keep the Discover run only if it moves to the main window.
- Compute cohorts in code, not prose. A small helper for the connected components over
attached_apps, shared by the workbook and report and covered by a test. Follow-up.
Verdict
Comment. Blocking 1 would set Request changes from a non-author identity.
Commenting rather than approving: author == reviewer identity.
herosjourney
left a comment
There was a problem hiding this comment.
Reviewed: head c6b9b23 (base 00840a0), 11 files (+741/−1). Skill touched: heroku-to-aws. Tree: plugins/aws-startup-advisor only.
Ran:
pytest scripts/test_emit_estate_workbook.py: 5 passed.emit-estate-workbook.pyon 14 constructed run directories, each output opened with openpyxl 3.1.5 and every XML part parsed. Inputs covered: a realistic inventory shaped by the schema; inventory only and the full inventory + design + estimate-with-contract chain; an empty estate,{}and[]; formula-leading, unicode and comma names; control characters,NaN/Infinityand a lone surrogate; duplicate names, a 40k-character cell and wrong-typed fields.- Two mutation probes on a copy of the script.
- A contract trace of every field the script reads or the diff writes, against the schema, the phase files and
scripts/artifact-contracts.json. - An
_exectier trace againstINTERPRETER.md. - A producer sweep for new fields across main, this head and the other open heroku-to-aws PRs (#608, #610, #611, #612, #614, #615).
GitHub CI is green at head (startup-advisor runs mise run ci; CodeQL, check-frontmatter and the author checklist also pass). Merges cleanly into main per git merge-tree.
Did not run: mise run ci locally; any live Heroku CLI call; an end-to-end Discover → Generate run; opening the workbooks in Excel or LibreOffice; checking the heroku addons --all --json output shape against a primary source.
Cross-plugin parity: n/a. This repo has a single copy of skills/heroku-to-aws/, and no vendored files were touched. Outside skills/: nothing changed.
Change narrative: Three things get rebuilt by hand today in large Enterprise estates: which apps share a data store, a spreadsheet the customer will review, and the committed contract spend. The PR adds attached_apps and cohort grouping in Discover; a standard-library .xlsx writer run at Discover, Estimate and Generate; and a Clarify question (Q16) feeding a contract reconciliation in Estimate and new report sections. Body vs diff: it matches, except "Independent of #608". Only the pg_role dependency is disclosed; the environment_tier and apps_by_environment_tier dependencies are not (Should-fix 4).
| Axis | Stance | One line |
|---|---|---|
| Parsimony | advisory | Additive and in the repo's idiom (a fail-open writer copying emit-plan-json.py). The Discover-time run is the one wiring point that costs more than it returns (Alternatives 1). |
| Completeness | finding | Q16 can't be reached (Blocking 1). Discover's dispatched worker is told to run python3 with no shell (Should-fix 1). |
| Security | advisory | No config var names or values, hostnames or UUIDs reach the workbook (test plus probe). Formula-leading text is stored as inline strings with no formula element. Order forms are not named in Input Security (Nit 3). |
| Alternatives | advisory | Two below. |
| Contradictions | finding | The like-for-like rule conflicts with "Keep the full baseline" (Should-fix 3). The schema cites "rule 9" for what is rule 10 (Nit 1). |
| Overlap | advisory | The like-for-like step re-derives Part 1b's deferred_resource_cost (folded into Should-fix 3). |
| Cross-plugin parity | clear | Single tree; nothing to mirror. |
| Propagation & data | finding | shared_data_stores / attached_apps: schema updated first, every consumer swept. preferences.commercial: its home clarify-assemble.md was not updated (Blocking 1). environment_tier, environment_tier_source and apps_by_environment_tier have no producer on main or at this head; only #608 has one (Should-fix 4). |
The contract between the script and the phase files was checked and matches:
- Invocation: the command string is identical in
discover-assemble.md:65,estimate-assemble.md:386andgenerate-assemble.md:124, and matches argparse. - Inputs:
heroku-resource-inventory.json,aws-design.jsonandestimation-infra.jsonmatch the_producesof their phases. - Status lines:
ESTATE_OK | …/ESTATE_SKIP | reason=…match all three phase files. - Design join:
deferred[]is matched on(addon_name, addon_plan), consistent withdesign-mapping.md§ 2F.
Blocking
- [Completeness] Q16 is never asked and its answer has nowhere to land, so contract reconciliation can't run on any path.
clarify/clarify-interview.md:159
| Q16 — Heroku contract | \billing_profile.available == false` (no invoice data) | Billing data present, or user says "none" |`
Consequence:
- The batch table puts Batch 3 at "
Q11, Q12b, Q12–Q15" (:230), the batch rule says "Q12–Q15 always fire" (:234), and the Batch 3 template presents "[Present active questions Q11–Q15]" (:287). Q16 is active but in no batch, and questions are only presented by batch. - Even if asked,
clarify-assemble.mdStep 4 (the declaredpreferences.jsonshape perscripts/artifact-contracts.json) has nocommercialkey, sopreferences.commercial.heroku_contractis never written. - As a result,
estimate-cost-engine.md:151never fires,current_costs.contractis never produced, and the report table, caveat and workbook block are dead on every run. - The trigger also reads
available == false, while runs with no billing file carrybilling_profile: nullor no section (fixturesheroku-nonweb-scaling,heroku-workshop).
Fix:
- Batch 3 cell (
:230) →Q11, Q12b, Q12–Q16; rule at:234→ "Q16 fires when no invoice data";:287→[Present active questions Q11–Q16]. - Trigger → "
billing_profileabsent, null, oravailable == false". - In
clarify-assemble.mdStep 4, add"commercial": { "heroku_contract": { … } }(omitted when Q16 is answered none or defaulted) and"Q16"tosources.
Regression: a contract test asserting every question in the Applicability table appears in exactly one batch row (or the "Not in any batch" list), and that clarify-assemble.md names commercial.
Should-fix
-
[Completeness] Discover's workbook step tells a shell-less worker to run
python3.discover/discover-assemble.md:64–65.discover.mddeclares_exec: _agent: rw, andINTERPRETER.md:307/314gives that tier Read/Grep/Glob/Write/Edit only ("rwdeliberately excludes shell/Bash"). On dispatching hosts the Discover-time workbook, sold in the body as "review what was found before any AWS design exists", is never written, and the step invites a fabricatedESTATE_OK. Fix: remove step 11 from the assembler and add a main-window step todiscover.mdafterHANDOFF_OK, mirroringgenerate-assemble.md:123. Generate (rwx) and Estimate (inline) are fine as written. -
[Generated artifacts] The writer prints
ESTATE_OKfor a workbook that won't open.scripts/emit-estate-workbook.py:44–47,:75–76. Reproduced at head:
confidence_note: "bad\x01char\x0bhere"givesESTATE_OK, thensheet2.xmlis not well-formed and openpyxl raisesParseError.monthly_price_usd: NaN(accepted byjson.load) givesESTATE_OK, then openpyxl reports "Unable to read workbook … invalid XML".- A lone surrogate in an app name gives
ESTATE_SKIP, but leavesheroku-estate.xlsx.tmpin$MIGRATION_DIR(:123–131).
Free-text fields copied verbatim are the realistic carriers (confidence_note, gap_explanation, order-form notes pasted from Word, where \x0b is a soft line break). Mutation probe: replacing escape() with return text still passes all 5 tests, so escaping has no coverage. Fix: strip [\x00-\x08\x0b\x0c\x0e-\x1f\ufffe\uffff\ud800-\udfff] in escape(), emit an empty cell for non-finite floats, and tmp.unlink(missing_ok=True) on any exception during the zip write. Regression: a test with &<>", \x0b and NaN that asserts ESTATE_OK and that every worksheet part parses.
-
[Contradictions] The like-for-like rule pulls against "Keep the full baseline", and the figure has two names.
estimate/estimate-cost-engine.md:182–184says the like-for-like figure is "the only Heroku figure that is scope-matched … Part 3 uses it". Part 3 (:433) says "Keep the full baseline — do not subtract or hide the deferred resource's cost." Part 3 also stores the figure asheroku_monthly_like_for_like(:402), notdiscovered_like_for_like_monthly, and step 5 re-derives Part 1b'sdeferred_resource_costby a different method. Fix: "Writecost_comparison.heroku_monthly_like_for_like= Part 1 baseline −current_costs.deferred_resource_cost. Show it beside, never instead of,heroku_monthly_baseline; everymonthly_differencestays against the full baseline." The same goes for step 2 (:158–161): the discovered add-on bucket has no contract category to land on, so per-linediscovered_usdwon't sum todiscovered_monthly. Map add-ons toaddon_credits. -
[Propagation / PR honesty] Consumers of #608-only fields ship without #608, and the body says "Independent of #608".
generate/generate-report.md:64(metadata.apps_by_environment_tier) andemit-estate-workbook.py:202–203, 218(environment_tier,environment_tier_source) have no producer on main or at this head; only #608'sdiscover-assemble.mdrule 5a writes them. If this lands first, the Apps tab carries two always-blank columns and Summary shows every app as "(not inferred)". Fix: state the dependency and land #608 → #609, or move these consumers into #608.
Nits
references/shared/schema-discover-heroku.md:73: "written by assemble rule 9" should read "rule 10" (discover-assemble.md:46).scripts/emit-estate-workbook.py:331–340: the Cost estimate tab omitsheroku_monthly_like_for_likeand the per-linediscovered_usd/gap_usd, so the workbook and report show the contract differently.SKILL.md:145Input Security lists "billing exports" but not the Enterprise order form that Q16 now invites. Add it to the untrusted-file list.
Heads-up (out of diff or pre-existing, not blocking)
- The new test file is not run by CI. The
mise run citesttask runspytest tests -qfrom the plugin root, soskills/heroku-to-aws/scripts/test_*.pyis never collected (the same is true of the existingtest_validate_heroku_migration_report*.py). Green CI says nothing about this script. - Unknown whether cohorts can be detected. Whether
heroku addons --all --jsonemitsattachments[]was not verified. The replay fixturefixtures/heroku-live-capture/live-capture/addons.jsonhas noattachmentskey, so no fixture exercises cohorts. Please link a captured sample or the CLI source. - A 40,012-character string is written unchecked; Excel's cell limit is 32,767. openpyxl reads it; Excel's behaviour is not verified.
- Landing order: #611 is stacked on this PR. Conflicts with #610 in
discover-assemble.mdandgenerate-report.md, and with #612 ingenerate-report.md; shares files with #608 and #614 without conflict. Suggested order: #608 → #609 → #611, then rebase #610 and #612.
Alternatives (max 2)
- Write the workbook at Estimate and Generate only. This removes the shell-less wiring point (Should-fix 1); the cost is no workbook before Design. Keep the Discover run only if it moves to the main window.
- Compute cohorts in code, not prose. A small helper for the connected components over
attached_apps, shared by the workbook and report and covered by a test. Follow-up.
Verdict
Comment. Blocking 1 would set Request changes from a non-author identity.
Commenting rather than approving: author == reviewer identity.
…ct gaps. Review findings on PR aws#609 - emit-estate-workbook.py: strip XML-illegal characters (C0 controls, non-characters, lone surrogates) before escaping; non-finite floats (NaN/Infinity accepted by json.load) emit an empty cell; the zip write removes its .tmp on any exception; Read Me states that the environment-tier columns populate only once the discovery that writes them is present. - test_emit_estate_workbook.py: three regressions (every worksheet part parses with xml.etree after \x01/\x0b/lone-surrogate input; no <v>nan</v>/<v>inf</v>; .tmp gone after a failed rename). - clarify-assemble.md: add commercial.heroku_contract and sources.Q16 to the preferences.json template (machine contract; validate-artifacts previously raised UNKNOWN_KEY on commercial). - estimate-cost-engine.md: contract reconciliation buckets discovered add-on spend under addon_credits, per-line discovered_usd must sum to discovered_monthly; the scope-matched figure is cost_comparison.heroku_monthly_like_for_like = full baseline minus current_costs.deferred_resource_cost, an additional field beside the unchanged full baseline (Part 3's "Keep the full baseline" now says so explicitly). - generate-report.md: apps_by_environment_tier line is explicitly "when present". - schema-discover-heroku.md: "assemble rule 9" -> "rule 10". Verified: mise run ci (lint:md 0 errors, lint:artifacts, lint:vendored-parity, lint:skill-refs, pytest 1475 passed, telemetry) green.
…ar production, not a regex range CodeQL flagged the control-character range in _XML_BAD as an overly permissive regular expression range. Replace the regex with an explicit per-character check against XML 1.0's Char production (TAB/LF/CR, U+0020-U+D7FF, U+E000-U+FFFD, U+10000-U+10FFFF). Same behaviour; the existing test that writes \x01, \x0b and a lone surrogate and parses every sheet still passes. The now-unused re import is removed. Review finding: github-advanced-security on PR aws#609.
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Reviewed 31f171c. The two previously reported issues are fixed: Q16 now handles absent billing profiles, and matched-cost fields use the same selected-baseline contribution.
Two additional P2 issues remain:
- The report promises to cross-check the new Heroku comparison anchor, but the validator skips that key. Checking the already-emitted anchor does not require new mandatory sections.
- A valid estate can have a zero matched baseline; the percentage formula and report need an explicit unavailable-percentage branch while keeping the absolute difference.
Both issues were present at the previously reviewed head, 803af2f. I missed them in that review; they were not introduced by these latest fixes. The inline comments give reproductions and bounded corrections.
Validation: 44 focused original-source tests passed; eight original-writer cases, five preference-contract cases and XML boundary probes passed. Six full report-validator CLI checks reproduced the missing Heroku cross-check with AWS mismatch controls. The zero-baseline input passes the original machine inventory contract; its arithmetic witness is not a live migration execution.
…e the zero-baseline percentage, mark incomplete attribution provisional. Review findings on PR aws#609 - validate-heroku-migration-report.py: register heroku_monthly_like_for_like in _COST_ANCHORS (cost_comparison.heroku_monthly_like_for_like) and require it inside exec-costs whenever cost_comparison.like_for_like_comparison exists; optional otherwise. A rendered full-baseline figure against a smaller matched JSON value now fails instead of being skipped as an unknown key. - estimate-cost-engine.md: difference_pct is omitted (null) with pct_note when the matched Heroku baseline is $0; monthly_difference is kept. New like_for_like_comparison.attribution ("complete" | "incomplete", incomplete iff unattributed_deferred_services is non-empty); only complete may be labelled decision-grade. Contract reconciliation step 5 sets it. - generate-report.md / report-decision-core.md: render "(percentage not applicable)" for the zero baseline and the provisional label naming the unattributed add-ons for incomplete attribution, in both report modes. - emit-estate-workbook.py: provisional heading for incomplete attribution, pct_note in the percentage cell when difference_pct is null. Tests: 5 new validator cases (match/mismatch/missing/optional, both modes, anchor outside exec-costs) and 3 new workbook cases (complete, incomplete, zero baseline). `mise run ci` passed: lint:md 0 errors, pytest 1475 passed.
…measured-data sizing basis Large Heroku estates carry Postgres databases on plans chosen years ago that hold a small fraction of their allocated storage. Sizing AWS from the plan name copies that waste into the AWS estimate, and the customer - who usually knows they are over-provisioned - distrusts the comparison. - Clarify Q6d (fires only when some database holds <= 25% of its plan's storage, shown as a table first): size AWS from measured data with 2x headroom (recommended) or from the current plan. Writes preferences.data.database_sizing_basis. - Design 2B step 2a: with "measured", the sizing row is looked up by required storage within the same tier family instead of by plan name - still a direct table lookup, keeps HA/compliance semantics, never applied to Advanced-tier add-ons. Records sizing_basis / sized_as_plan / source_plan in aws_config and warns per app. - Estimate Part 1: current_costs.rightsizing_candidates[] (plan storage, data, utilization, fitting plan in the same family, monthly saving) and rightsizing_monthly_saving_usd. The Heroku baseline is left as what they pay today; the saving is labelled as Heroku-side and available before migration. Followers and Advanced-tier add-ons are skipped; Redis is not assessed (plan memory, not used memory, is what discovery captures). - Report: "Right-size before you move" table under exec-costs. - Estate workbook: Right-sizing tab copied from the Estimate; test updated. Stacked on the estate-workbook branch (aws#609) because the workbook tab lives in scripts/emit-estate-workbook.py.
…cohorts-export-reconcile # Conflicts: # plugins/aws-startup-advisor/skills/heroku-to-aws/references/phases/discover/discover-assemble.md # plugins/aws-startup-advisor/skills/heroku-to-aws/references/phases/generate/generate-report.md
…cohorts-export-reconcile
|
The branch is now merged up to current main (head |
Problem
When
heroku-to-awsis run against a large Heroku Enterprise estate, three things the customer and account team need are missing, so the SA ends up rebuilding them by hand from the skill's JSON:Fix
Shared data stores (cohorts)
discover-live.md: recordattached_appsper add-on fromaddons.jsonattachments[]; limitation noted when the capture has no attachments.discover-assemble.mdrule 11 (rule 10 is the end-of-life stack rule from feat(aws-startup-advisor): flag end-of-life Heroku stacks in heroku-to-aws discovery #610, already on main) + schema:shared_data_stores.cohorts(connected components over owning app + attached apps, data-store add-ons only),apps_in_cohorts. Explicitly a reporting label: no edges, no ordering, nocluster_idon resources. The Philosophy "Flat resource model" bullet inSKILL.mdgains one sentence saying so.generate-report.md: conditionalshared-data-storessection (table + one sentence on what a cohort means for cutover; no sequencing recommendation).Estate workbook —
scripts/emit-estate-workbook.py(+test_emit_estate_workbook.py, 10 tests: tabs, exclusions, optional inputs, fail-open, stale-artifact exclusion, XML-illegal characters stripped and every worksheet part parses, NaN/Infinity → empty cell,.tmpremoved on write failure).xlsxwriter (zip + inline-string XML), so it runs anywhere the skill runs.aws-design.json.ESTATE_OK/ESTATE_SKIP, always exit 0), same pattern asemit-plan-json.py. Wired at the end of Discover (so the customer can review what was found before any design exists) and refreshed after Estimate and Generate.Contract reconciliation
clarify-interview.mdQ16 (asked whenbilling_profileis absent,null, or hasavailable != true; in Batch 3): order-form line totals and term dates intopreferences.commercial.heroku_contract. No unit prices, quantities or counterparty recorded.estimate-cost-engine.mdPart 1:current_costs.contractwith monthly run-rate, per-category contract vs discovered vs gap, agap_explanationthat must name mechanisms (negotiated vs list, headroom, apps outside CLI view, support/Connect invisible to discovery), with discovered add-on spend bucketed underaddon_creditsso per-linediscovered_usdsums todiscovered_monthly. Part 3 gainsheroku_monthly_like_for_like(= full baseline −current_costs.deferred_resource_cost, an additional field shown beside the unchanged full baseline) andheroku_monthly_contract.preferences.commercial.heroku_contract/sources.Q16are in theclarify-assemble.mdtemplate (machine contract).generate-report.md: reconciliation table underexec-costswith a fixed caveat; "contract minus AWS" is never the headline saving.Ported from awslabs/startups#365, which is closed in favour of this PR. Best landed after #608 (environment-tier columns and the report tier line read fields #608 produces; they are blank otherwise).
Scope
plugins/aws-startup-advisor/skills/heroku-to-aws/only:SKILL.md(one sentence),references/phases/clarify/clarify-interview.md,references/phases/discover/discover-live.md,references/phases/discover/discover-assemble.md,references/phases/estimate/estimate-cost-engine.md,references/phases/estimate/estimate-assemble.md,references/phases/generate/generate-assemble.md,references/phases/generate/generate-report.md,references/shared/schema-discover-heroku.md, and two new files underscripts/. No root or.githubchanges.Known limitations: the new report sections are conditional and not enforced by
validate-heroku-migration-report.py(the required section set is unchanged). Q16 is skipped when invoice data is present, so a customer with both an invoice and a contract is not asked about the contract.pg_roleon the workbook's Postgres tab, the Apps tab'sEnvironment tier/Tier sourcecolumns, the Summary's tier breakdown and the report'sapps_by_environment_tierline are populated only once #608 lands; without it those columns are blank, the data total includes all databases, and the report omits the tier line (fail-soft, "when present"). No live run against a customer account is included.Type of Change
Select at least one.
Startup Advisor review checklist
By submitting this PR, I confirm that:
mise -C plugins/aws-startup-advisor run ciand all checks pass.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.