Skip to content

polity_area_crosswalk is built from a stale upstream map: 31 areas upstream now maps resolve through whep's Rest-of-World bucket instead #745

Description

@lbm364dl

Found while measuring #740 (PR #744). This is the cheapest half of #740 and it
needs no upstream work at all.

The drift

data-raw/table_mappings.R builds polity_area_crosswalk from
WHEP_POLITIES_FAOSTAT_MAP, i.e. data/final/faostat_area_polity_map.csv in
eduaguilera/whep-polities. The committed .rda and the current upstream file
disagree:

rows areas
eduaguilera/whep-polities@main (abbd237) 281 228
whep::polity_area_crosswalk, mapping_source == "upstream_map" 245 197

31 FAOSTAT reporting areas that upstream maps today have no upstream_map
row in the shipped crosswalk.
They fall back to mapping_source == "fabio_row_fold", whep's Rest-of-World bucket, which is what #740 counts as
"no upstream authority":

5 American Samoa; 6 Andorra; 17 Bermuda; 42 Christmas Island; 47 Cook
Islands; 61 Equatorial Guinea; 64 Faroe Islands; 65 Falkland Islands
(Malvinas); 69 French Guiana; 85 Greenland; 87 Guadeloupe; 88 Guam;
125 Liechtenstein; 135 Martinique; 140 Monaco; 142 Montserrat;
153 New Caledonia; 154 North Macedonia; 160 Niue; 161 Norfolk Island;
180 Palau; 182 Réunion; 187 Saint Helena, Ascension and Tristan da Cunha;
190 Saint Pierre and Miquelon; 192 San Marino; 205 Western Sahara;
209 Eswatini; 212 Syrian Arab Republic; 239 British Virgin Islands;
240 United States Virgin Islands; 299 Palestine

Why it matters now

These are not empty codes. #628 promoted the Rest-of-World members to their own
polity_area_code, so 22 of the 31 publish rows in a real
get_primary_production() — and every one of those rows currently resolves
through a whep-invented crosswalk row rather than upstream's:

area rows published
212 Syrian Arab Republic 57,751
209 Eswatini 24,735
153 New Caledonia 22,000
61 Equatorial Guinea 16,444
182 Réunion 14,233
87 Guadeloupe 14,088
135 Martinique 11,456
69 French Guiana 10,385
47 Cook Islands 10,353
154 North Macedonia 9,393
299 Palestine 8,215
160 Niue 7,551
64 Faroe Islands 2,716
+ 9 more, 346 rows or fewer each

That is 3.37% of a 6,310,390-row build resolving through fabio_row_fold,
measurable with polity_mapping_provenance() (#744). After a re-sync, most of
it becomes upstream_map.

Only two publishing areas would still be genuinely missing upstream after
the re-sync: 94 Holy See (268 rows) and 22 Aruba (244). Those are filed at
eduaguilera/whep-polities#200.

What to do

Re-run data-raw/table_mappings.R against the current upstream map and commit
the rebuilt data/*.rda, the way #470 and #530 did. Worth checking at the same
time whether the polities table itself has drifted (upstream's
polities_database.csv is ahead by a handful of rows on a feature branch, so
compare against main before concluding anything there).

This will move published values, because the reporting polity of those 22
areas changes from the ROW aggregate to their own territory — the same class of
change #628 costed. It needs the before/after measurement that convention
requires, not a blind rebuild. Related: #717 (a promoted ROW member still
carries the ROW aggregate polity) is probably the same root cause and may close
with this.

Part of the polity migration epic #458.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:regionsRegions, countries, polities: harmonization & modelingbugSomething isn't workingneeds-expertEmbeds a methodological/scientific decision. Do not merge on green CI alone; tag an expert.priority:highWe want to get this done soon

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions