What is wrong
Two internal-consistency violations in
inst/extdata/harmonization/biomass_coefs.csv, measured on current main
(421 rows).
1. 72 rows carry more proximate mass than dry matter
Summing protein (N_kgN_kgFM x 6.25), Carbohydrates_g_kgFM,
Lipids_g_kgFM and Fiber_g_kgFM and comparing against
Product_kgDM_kgFM x 1000:
| Name_biomass |
proximate sum (g/kg) |
dry matter (g/kg) |
excess |
| Urea |
2846.2 |
990.0 |
1856.2 |
| Carob |
1245.5 |
718.0 |
527.5 |
| Tigernuts |
723.0 |
300.3 |
422.7 |
| Figs |
610.0 |
197.0 |
413.0 |
| Lysine |
1234.4 |
980.0 |
254.4 |
| Honey |
1000.0 |
785.0 |
215.0 |
| Cocoa beans |
919.0 |
718.0 |
201.0 |
| Barley |
1068.5 |
885.0 |
183.5 |
| Hemp seed |
1085.6 |
905.5 |
180.1 |
Some rows are feed additives where a composition block is simply not
applicable (Urea, Lysine). But ordinary foods are affected too — Barley, Figs,
Carob, Honey, Cocoa beans — so this is not only a not-applicable-row artifact.
The practical consequence is that the proximate columns cannot be used to
reason about a row's basis. Concretely, in #500 the argument "Fiber_g_kgFM =
18.5 is too low for hull-bearing rough rice, so the Rice row must be milled"
does not hold, because that row's own proximate columns sum to 961.8 g/kg
against 864 g/kg of dry matter. The conclusion happens to be right, but the
evidence is not usable.
2. One Edible_portion outside (0, 1]
Exactly one row: ANIMAL PRODUCTS, Edible_portion = 4.0.
This is not a leaked section header, and it must not be "cleaned" upstream.
In afsetools/inst/extdata/Biomass_coefs.xlsx that row is the VLOOKUP
column-index vector the whole Coefs sheet depends on by absolute address
(row 292: M=2, S=4, T=3, U=5, V=6, W=7, X=8). Deleting or editing it upstream
breaks the workbook. It should be filtered downstream at ingestion instead.
It is currently harmless — no item_cbs_code in items_full.csv maps to
Name_biomass == "ANIMAL PRODUCTS", so build_food_supply() cannot reach it —
but nothing asserts that, so a future mapping edit could silently pull a
4x edible fraction into a food calculation.
Suggested resolution
Add ingestion-time invariants rather than editing the upstream workbook:
0 < Edible_portion <= 1, with ANIMAL PRODUCTS filtered at read and
asserted unreachable from any item_cbs_code.
- proximate sum
<= dry matter, scoped to rows where a composition block is
meaningful, so the exemptions are explicit rather than implied.
Deciding which of the 72 rows carry a wrong value (as opposed to a
not-applicable one) is a coefficient question for the data owner, which is why
this is needs-expert rather than mechanical.
Reproduced on main with inst/extdata/harmonization/biomass_coefs.csv; no
pin or network needed.
What is wrong
Two internal-consistency violations in
inst/extdata/harmonization/biomass_coefs.csv, measured on currentmain(421 rows).
1. 72 rows carry more proximate mass than dry matter
Summing protein (
N_kgN_kgFMx 6.25),Carbohydrates_g_kgFM,Lipids_g_kgFMandFiber_g_kgFMand comparing againstProduct_kgDM_kgFMx 1000:Some rows are feed additives where a composition block is simply not
applicable (Urea, Lysine). But ordinary foods are affected too — Barley, Figs,
Carob, Honey, Cocoa beans — so this is not only a not-applicable-row artifact.
The practical consequence is that the proximate columns cannot be used to
reason about a row's basis. Concretely, in #500 the argument "Fiber_g_kgFM =
18.5 is too low for hull-bearing rough rice, so the Rice row must be milled"
does not hold, because that row's own proximate columns sum to 961.8 g/kg
against 864 g/kg of dry matter. The conclusion happens to be right, but the
evidence is not usable.
2. One
Edible_portionoutside (0, 1]Exactly one row:
ANIMAL PRODUCTS,Edible_portion = 4.0.This is not a leaked section header, and it must not be "cleaned" upstream.
In
afsetools/inst/extdata/Biomass_coefs.xlsxthat row is the VLOOKUPcolumn-index vector the whole
Coefssheet depends on by absolute address(row 292: M=2, S=4, T=3, U=5, V=6, W=7, X=8). Deleting or editing it upstream
breaks the workbook. It should be filtered downstream at ingestion instead.
It is currently harmless — no
item_cbs_codeinitems_full.csvmaps toName_biomass == "ANIMAL PRODUCTS", sobuild_food_supply()cannot reach it —but nothing asserts that, so a future mapping edit could silently pull a
4x edible fraction into a food calculation.
Suggested resolution
Add ingestion-time invariants rather than editing the upstream workbook:
0 < Edible_portion <= 1, withANIMAL PRODUCTSfiltered at read andasserted unreachable from any
item_cbs_code.<=dry matter, scoped to rows where a composition block ismeaningful, so the exemptions are explicit rather than implied.
Deciding which of the 72 rows carry a wrong value (as opposed to a
not-applicable one) is a coefficient question for the data owner, which is why
this is
needs-expertrather thanmechanical.Reproduced on
mainwithinst/extdata/harmonization/biomass_coefs.csv; nopin or network needed.