Skip to content

biomass_coefs: 72 rows have proximate mass above their dry matter, and one Edible_portion is 4.0 #752

Description

@eduaguilera

What is wrong

Two internal-consistency violations in
inst/extdata/harmonization/biomass_coefs.csv, measured on current main
(421 rows).

1. 72 rows carry more proximate mass than dry matter

Summing protein (N_kgN_kgFM x 6.25), Carbohydrates_g_kgFM,
Lipids_g_kgFM and Fiber_g_kgFM and comparing against
Product_kgDM_kgFM x 1000:

Name_biomass proximate sum (g/kg) dry matter (g/kg) excess
Urea 2846.2 990.0 1856.2
Carob 1245.5 718.0 527.5
Tigernuts 723.0 300.3 422.7
Figs 610.0 197.0 413.0
Lysine 1234.4 980.0 254.4
Honey 1000.0 785.0 215.0
Cocoa beans 919.0 718.0 201.0
Barley 1068.5 885.0 183.5
Hemp seed 1085.6 905.5 180.1

Some rows are feed additives where a composition block is simply not
applicable (Urea, Lysine). But ordinary foods are affected too — Barley, Figs,
Carob, Honey, Cocoa beans — so this is not only a not-applicable-row artifact.

The practical consequence is that the proximate columns cannot be used to
reason about a row's basis. Concretely, in #500 the argument "Fiber_g_kgFM =
18.5 is too low for hull-bearing rough rice, so the Rice row must be milled"
does not hold, because that row's own proximate columns sum to 961.8 g/kg
against 864 g/kg of dry matter. The conclusion happens to be right, but the
evidence is not usable.

2. One Edible_portion outside (0, 1]

Exactly one row: ANIMAL PRODUCTS, Edible_portion = 4.0.

This is not a leaked section header, and it must not be "cleaned" upstream.
In afsetools/inst/extdata/Biomass_coefs.xlsx that row is the VLOOKUP
column-index vector the whole Coefs sheet depends on by absolute address
(row 292: M=2, S=4, T=3, U=5, V=6, W=7, X=8). Deleting or editing it upstream
breaks the workbook. It should be filtered downstream at ingestion instead.

It is currently harmless — no item_cbs_code in items_full.csv maps to
Name_biomass == "ANIMAL PRODUCTS", so build_food_supply() cannot reach it —
but nothing asserts that, so a future mapping edit could silently pull a
4x edible fraction into a food calculation.

Suggested resolution

Add ingestion-time invariants rather than editing the upstream workbook:

  • 0 < Edible_portion <= 1, with ANIMAL PRODUCTS filtered at read and
    asserted unreachable from any item_cbs_code.
  • proximate sum <= dry matter, scoped to rows where a composition block is
    meaningful, so the exemptions are explicit rather than implied.

Deciding which of the 72 rows carry a wrong value (as opposed to a
not-applicable one) is a coefficient question for the data owner, which is why
this is needs-expert rather than mechanical.

Reproduced on main with inst/extdata/harmonization/biomass_coefs.csv; no
pin or network needed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:data-ioData inputs, pins, reading, scrapingbugSomething isn't workingneeds-expertEmbeds a methodological/scientific decision. Do not merge on green CI alone; tag an expert.no-data-neededFixable and verifiable with no pins, rasters or network; package data onlypriority:mediumThis is something to keep in mind

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions