[epic] Every territorial identity in whep carries a polity code string

[lbm364dl]

Goal

Every row, cell, coefficient and join key in whep that identifies a territory must carry a polity code string — ESP-1846-1914, SDN-2011-2025 — not a raw FAOSTAT area code, an ISO3 string, a country name, a COW code, a hand-maintained region table, or a static present-day grid.

This is deliberately stronger than "adopt polity_area_code repo-wide", which is what #290 proposed and why that issue is closed in favour of this one. polity_area_code is not a polity. It is coalesce(fabio_code, area_code), a numeric aggregation bucket. Measured on the shipped crosswalk:

• 544 distinct polity_code collapse into 201 polity_area_code
• (polity_area_code, year) recovers 194 of 201 buckets — the temporal folds are fine
• 7 buckets it cannot recover. Bucket 999 holds up to 17 territories simultaneously; bucket 206 holds Sudan and South Sudan simultaneously 2011-2024. The other five (15, 28, 101, 170, 273) are crosswalk overlapping-period defects, not designed folds
• Of the 62 area codes folded into 999, 16 already have their own real polity code — Bermuda BMU-1684-1968, Faroe Islands FRO-1800-2025, Eq. Guinea GNQ-1886-1968, French Guiana GUF-1816-1946, Guadeloupe, Guam, Cook Islands. The polity exists; the numeric fold discards it

So the numeric key is a legitimate FABIO-matrix-grain aggregation key, and it must stop being the thing rows are identified by. That makes #419 and #414 critical path rather than side quests: until the folds stop, attaching a polity string to that data just writes ROW-1850-2023, which is not a territory.

Gridded data is in scope. Where no historical boundary exists, resolving to the present-day polity is acceptable — but as an explicit, documented choice with a real polity_code column, never as an unstated side effect of joining against a present-day-only lookup.

Already polity-keyed on main — do not redo this

Confirmed by direct measurement / code reading; no PR needed:

• Core FAOSTAT ingestion: get_primary_production()/build_primary_production() and every
  builder downstream of .aggregate_to_polities() (R/read_raw_inputs.R:297-349,
  R/polities.R:37-219) — year-aware interval join, polity_area_code already renamed onto
  area_code. build_commodity_balances(), build_supply_use(), build_detailed_trade() all
  finish with .add_reporting_polity_columns().
• Crop/livestock spatialize output: build_gridded_landuse() / build_gridded_livestock()
  already call .add_reporting_polity_columns() (R/spatialize.R:257-258,
  R/spatialize_livestock.R:244-246); the compartment/fractional-cell machinery
  (R/spatialize_compartments.R) is unit-tested and already supports cell_area_frac/polity_frac.
• Trade matrix: build_detailed_trade()'s .aggregate_dtm_to_polities() resolves both
  reporter and partner sides via year-aware .add_polity_columns_dt(), with a documented
  aggregate-polity self-loop carve-out (R/build_trade.R:187-270, 256-267).
• Land area: R/arable_permanent_land.R and R/grassland_land_extension.R both route through
  whep::polity_area_crosswalk/add_polity_code() and explicitly drop the China aggregate (351).
• Aggregation/footprint: build_io_model(), compute_footprint(), constant_territory.R
  (reference-year boundary set comes from get_polity_geometries(), not a static grid),
  calculate_lmdi.R/decompose_weighted_ratio.R/consolidate_sources.R (generic, no embedded
  vocabulary).
• regions_full dataset: already carries polity_code/reporting_polity_code/
  polity_area_code columns derived from the crosswalk.
• Numeric engines with no territorial identity of their own (correctly so — do not add one):
  manner_model.R, soc_dynamics.R, soc_models.R, soc_climate.R, read_cru_climate(),
  read_hyde_population(), read_soil_ph(), read_soil_hydraulic().
• Feed: national-grain demand and the Bouwman-region lookup are already polity-keyed end to end
  (the old hardcoded "OECD Europe" Spain-specific region is gone); only the aggregate-polity edge
  (Give ROW/continent-"Other" aggregate polities a Bouwman feed-region fallback #467) is open.

Standing conflict: whep#424

PR #382 added tests/testthat/test_exported_polity_column_boundary.R, which deliberately pins
22 exported functions (including build_nitrogen_balance, build_water_balance,
build_carbon_balance, read_luh2_landuse, etc.) as not carrying reporting-polity columns, on
the grounds that "whether they should carry the columns is whep#424" and a "mechanical sweep would
get get_faostat_data wrong." Under the owner's stated rule — everything territorial routes
through polities — that pin is the wrong call: these 22 exports already carry a polity-derived
area_code (inherited from get_primary_production()/get_wide_cbs()/read_luh2_landuse()
upstream) but do not say so in their own schema. #424 inverts this test rather than porting
it: 19 of the 22 get the columns added; the remaining 2 (build_grazing_feed_footprint,
build_land_balance_footprint) need an owner decision on which year to resolve at, because they
aggregate over time and have no year column.

PR #382 is the source of several already-written fixes

Closing #382 does not lose this work — it is preserved by naming these files/functions in the
children below:

• R/polities.R — resolve_polity_label() (≈lines 653-729), the label→polity primitive → Three datasets carry area identifiers that resolve to no polity (latent, 779 rows) #389
• data-raw/table_mappings.R — polity_label_aliases loading/validation → Three datasets carry area identifiers that resolve to no polity (latent, 779 rows) #389
• R/sources.R — .add_reporter_polity() for the trade-sources catalog → Three datasets carry area identifiers that resolve to no polity (latent, 779 rows) #389
• R/energy_co2_extension.R — .warn_areas_gleam_cannot_group() → energy_co2_extension: countries missing from GLEAM crosswalk get NA intensity and are silently dropped; NA emission factors silently become zero #207
• R/build_production.R — .read_fodder_euadb() numeric-key fix, .canonicalise_gdp_pop_area(),
  .warn_residues_no_area(), .compute_stock_shares()/.carry_forward_shares() territory-key fix,
  LUH2 unmatched-warning sentinel/territory split → Port core-production-cbs's 5 small #382 fixes #470 (and partially 11.9% of 1961-62 production is not back-cast: the LUH2 bridge is keyed by modern ISO3 and four of the areas are dissolved federations #408, Year-validity check in build_water_balance()'s cell-polity aggregation #462)
• R/build_cbs.R — .warn_unmapped_codes(), call site for .canonicalise_gdp_pop_area() → Port core-production-cbs's 5 small #382 fixes #470
• R/footprint_paths.R — doc-only addition naming the 17-area legacy-code mismatch (code itself
  unfixed) → Fix compute_footprint_paths()'s legacy-area-code filter footgun #471
• R/constant_territory.R — doc/warning additions only, no behavior change (nothing to port)
• tests/testthat/test_exported_polity_column_boundary.R — documents but does not fix the
  whep#424 boundary; 22 of 32 area-keyed exported outputs carry no polity columns — extend, or document as intentional? #424 inverts it rather than ports it
• inst/extdata/cow_to_lpjml.csv — Comoros/Mayotte data commits (unrelated bug fix, not a
  vocabulary port)

---

What the sub-issue list cannot tell you

The children are tracked in the sub-issues panel above. Three things that live here instead:

Grid work: a totals diff proves nothing. For anything touching the cell grid, national totals conserve by construction — the share denominators are per-polity and R/spatialize.R:829-843 rescales back to the national target. So "totals unchanged" reads as success even for an arbitrarily wrong migration. The measurement that discriminates is the per-(country, crop/species, year) L1 distance between old and new normalised cell-share vectors, plus the change in how many distinct cells receive a non-zero allocation. (Thanks @eduaguilera — this corrects guidance that was previously wrong here.)

The grid unit is the polycell. polycell_id = f(cell_id, polity_code) — cell x polity over the polity's validity interval, geodesic-intersected against get_polity_geometries(), with polity_area_ha = land_area_ha + inland_water_ha + ice_area_ha. Delivered by branch edu/polycell-spatial-support; #460 and #461 track it and their original present-day-snapshot framing is withdrawn. #462 and possibly #459 are downstream of it.

Open decision — which footprint is canonical. The deployed cell_polity_fraction.parquet measures 68,527 rows / 64,438 cells / 191 polities and is the basis of every published number, while the current producer path yields a smaller footprint after #381 (~5,651 deployed cells absent from country_grid; Singapore, Malta and Bahrain affected). Whichever is chosen silently becomes the baseline for every before/after in this migration, so it needs deciding explicitly rather than by whichever PR lands first.

Which children move published values. Most of them do, so a PR that changes any of these needs a magnitude check against main, not just green tests (subject to the grid caveat above): #459 #460 #461 #462 #463 #464 #465 #466 #467 #468 #469 #470, plus #408 #419 #414 #299 #389. The ones that do not move numbers are #424 #471 #207 — those are contract and diagnostic changes.

Critical path. #419 and #414 gate everything touching rest-of-world and Sudan, including #467. Until those folds stop, attaching a polity string to that data just writes ROW-1850-2023, which is not a territory.

Real dependencies. #460 (the one large item, the cell↔area grid) benefits from #459 landing first; #462 is blocked by #460; #466 is blocked by #465. Everything else can start in parallel today.

Fastest wins. #404, #401 and #399 already have working implementations on PR #382's branch — recover the diff rather than rewriting. Each names its files.

Crosswalk defects found while scoping this

Five of the seven buckets where (polity_area_code, year) cannot separate polities are data defects, not designed folds — overlapping polity periods on areas 15, 28, 101, 170 and 273, plus a duplicated ROW-1850-2023 row on area 240. Three of the five enter because the crosswalk build derives its join key with sub("-.*", "", polity_code), which collapses MMR-LWR-1852-1885 onto prefix MMR. Note that AZE-SSR-1920-1991 reaches the crosswalk only through that same quirk, so naively fixing the regex would silently remove Azerbaijan's 1920-1990 coverage.

Confidence

Every finding rests on reading code and running example = TRUE paths on origin/main. No audit ran a full build against real pins, so per-dataset area domains are partial. The structural conclusions — vocabulary mismatches, the 9 invalid grid codes, the bucket-collapse counts, the 10/22 export split — do not depend on that.

Superseded #290, which aimed at the wrong key. Closed as verified-fixed while scoping: #237, #238, #255, #403, #38.