Skip to content

CBS rice is mixed-basis: the paddy-to-milled conversion is name-keyed and never fires on the current FAOSTAT FBS vintage #751

Description

@eduaguilera

What is wrong

.fix_item_codes() converts paddy rice to milled equivalent by matching the
item name:

# R/read_raw_inputs.R:262-269
dt[
  item_cbs_code %in% c(2804L, 2807L) &
    item_cbs %in% c("Rice, paddy", "Rice (Paddy Equivalent)"),
  value := value * .rice_milled_extraction_rate()
]

The current FAOSTAT Food Balances vintage names item 2807 "Rice and
products"
, which is in neither string, so the conversion never fires on it.
The very next block (R/read_raw_inputs.R:271-284) then relabels rows to that
same name, so anything already carrying it passes straight through.

Why that is a defect and not a preference

FAOSTAT's new FBS publishes item 2807 on a paddy (rough-rice) basis.
Verified in the cached faostat-fbs-new pin, 2010 Production, element 5511:

Area 2807 (new FBS) 2805 "Rice (Milled Equivalent)" (historic FBS) ratio
China, mainland 195,761 kt 130,573 kt 0.667005
India 143,963 kt 96,023 kt 0.667001

Those are the countries' paddy tonnages, not 0.67 of them.

Meanwhile WHEP's own production path does convert
(R/build_production.R:2578-2629, .fix_rice_milled_equiv()), and WHEP states
milled equivalent as the invariant for this item:

the CBS harmonizer already converts rice to milled equivalent
R/build_production.R:2579-2583

So a single item currently mixes bases: milled production against paddy FBS
utilisation. The item's own stated contract is violated.

The historic-FBS path is fine: item 2805 is already milled, and relabelling it
to 2807 without rescaling is correct. Only the new-FBS rows are wrong.

Consequences

  • Nourishment axis. biomass_coefs row Rice is a milled density
    (N_kgN_kgFM = 0.011889908; 7.07% protein at N x 5.95, against FAO's implied
    milled 7.156%, agreeing within 1.2%). Applying it to paddy tonnage overstates
    rice protein: WHEP/FAOSTAT concentration ratio 1.550, falling to 1.0385
    once the basis is corrected. This is the single largest term in Residual per-capita protein overstatement after #361: rice and wheat account for all of it #500, and it
    inverts that issue's hypothesis — it is a milled density on a paddy mass, not
    a whole-grain density on a milled mass.
  • Supply-use balance. With production milled and utilisation paddy, the
    difference is absorbed by the residual stock_variation plug
    (R/build_cbs.R:3066), which is systematically negative for 2807.
  • Every downstream consumer of 2807 tonnes: feed redistribution, the IO
    A-matrix, land and footprint extensions.

Note on magnitudes

Absolute tonnage figures should be re-measured on a current build. The
C:/XL_files/whep CBS artifact predates the #291 fix, so world-level totals
taken from it are not reliable for this issue. The ratios above come from the
pin and the packaged coefficients directly and are unaffected.

Fix

Key the conversion on the item code plus source rather than on names the
current vintage no longer uses, and check the interaction with the
paddy/milled de-duplication at R/read_raw_inputs.R:238-259 and with the
FBS_Old-to-FBS_New level scaling at R/build_cbs.R:1946-1952.

Regression: a fixture carrying item_cbs = "Rice and products",
item_cbs_code = 2807 asserting the conversion fires; plus a balance invariant
that stock_variation for 2807 is not systematically negative at ~1/0.67 of
production.

Sourcing for the 0.67 (currently an uncited literal at
R/read_raw_inputs.R:224-226): FAO Technical Conversion Factors for
Agricultural Commodities
gives country paddy-to-milled rates with median 65%
(range 60-73; China mainland 67, India 66), and the FAO Food Balance Sheets: A
Handbook
worked example uses 67%.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:cbsCommodity balance sheets & supply-usearea:data-ioData inputs, pins, reading, scrapingarea:productionPrimary & livestock production buildbugSomething isn't workingmechanicalNo methodological decision; objectively correct. Reviewable on tests/CI, low-review to merge.priority:highWe want to get this done soon

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions