rice-mut-model v0.3.0-dev

Genome-wide analytical N* design calculator for heavy-ion mutagenesis breeding in rice (8022). P0 uses a fixed-τ Poisson-Binomial criterion (labelled N_bayesian in the legacy schema for backward compatibility); a full posterior-integrating Bayesian hierarchy is planned for P1. The l9_interaction layer is implemented; l1_dose_response through l8_validation are P1 placeholder stubs (see module STATUS: headers).

Upgraded from BADH2 single-gene P0 (v0.1.0) to full genome-wide P0-ext (v0.2.0) per synthesis_v3.1 + synthesis_v4.1 + audit Wave R1 fixes (audit_report_20260418.md).

Plan References

• plans/学位论文/plan_phase4_p0_implementation_v1.md — P0 base implementation
• plans/学位论文/synthesis_v4.1.md — P0-ext genome-wide extension
• plans/学位论文/phase4_p0ext_completion_report_v1.md — P0-ext completion report
• audit_report_20260418.md — Wave R1 audit findings

Capabilities (v0.3.0-dev / P0 analytical design calculator)

• 全基因组 61k 基因 p_g 预测: genome-wide L_eff table covering 61,285 genes: 16,660 with explicit CDS+UTR (27.2%); 44,625 use CDS-only fallback (72.8% — systematic UTR annotation missing in 8022 GFF3). The CDS-only subset further splits into 15,368 MSU-liftover (≈25% UTR underestimate risk per median UTR fraction) and 29,257 8022 de novo genes whose UTR presence is unknown pending RNA-seq re-annotation. See data/gene_leff_table_provenance.md §7 for the full subset breakdown. Per-gene p_g_raw via AnalyticalPgEngine (μ_global=3.24e-7).
• 多目标育种 N 推荐*: co-fixation probability for K ≥ 1 genes via Poisson-Binomial DP + Clopper-Pearson lower bound
• Gene symbol resolver: BADH2/LOC_Os/OsEIL1/... → canonical LOC_Os ID with alias_8022 lookup (32,770 8022-local IDs supported)
• Portfolio interaction seed list: 138 curated candidate pair rows in a 138-slot scaffold, of which 3 (2.2%) currently have literature-backed interaction labels (1 lethal A + 2 synergy/antagonism B/C); the remaining 135 pairs (97.8%) default to neutral pending P1 curation. Treat as exploratory; see Methods §M10 and the UI warning banner on the Portfolio tab. (R2 OP1.9 framing clarification)
• Streamlit 5-tab UI: Gene Search / N* Calculator / Sensitivity Heatmap / Portfolio Matrix / Methodology; full zh/en i18n (153/153 key parity)
• 132/132 pytest pass (post-R4 Week 2 Module 1; was 124/124 post-R2 Fix, 93/93 post-R1, 53/53 at v0.2.0 baseline): mix of ~70 core formula + regression tests, ~25 schema/data-compliance, ~15 smoke/layer existence, ~10 Methods-docs grep, ~11 R4 reproducibility + safety guards. Raw test count is NOT a formal coverage claim; see docs/reports/methods_must_state_v1.md §M14.

Quick Start

# Install dependencies + dev extras (reproducible, frozen lockfile)
uv sync --frozen --extra dev

# Run Streamlit UI (port 8501)
uv run streamlit run calculator/app.py

# Run tests
uv run pytest tests/ -v

Reproducibility

Environment

• uv.lock pins every dependency (committed). uv sync --frozen reproduces the exact environment used for verification.
• Deliberate upgrade: uv lock --upgrade (produces a diff in the PR).

Rebuild external inputs

Scripts that depend on upstream / HPC data resolve each path via a three-step chain: CLI flag > environment variable > data/manifest.tsv row.

Script	CLI flags	Env vars	Manifest rows
build_gene_catalogs.py	--gff3, --cloned-csv	RICE_GFF3_PATH, RICE_CLONED_CSV_PATH	ref_8022_gff3, ref_cloned_csv
pfam_full_scan.sh	--pfam-db, --protein-fasta, --gene-pool	RICE_PFAM_DB, RICE_PROTEIN_FASTA, RICE_GENE_POOL	ref_pfam_db, ref_8022_protein_fa, gene_recurrence_4548

Examples:

# Rebuild genome-wide gene catalogs — pass paths on the CLI
uv run python scripts/build_gene_catalogs.py \
    --gff3 /local/path/802.final.gff3 \
    --cloned-csv /local/path/NIP_cloned_gene.csv

# Or, resolve via environment variables (handy for batch jobs)
RICE_GFF3_PATH=/local/path/802.final.gff3 \
RICE_CLONED_CSV_PATH=/local/path/NIP_cloned_gene.csv \
    uv run python scripts/build_gene_catalogs.py

# Pfam scan (bash; same resolution order — CLI > env > manifest)
bash scripts/pfam_full_scan.sh \
    --pfam-db /local/path/Pfam-A.hmm \
    --protein-fasta /local/path/802.pep.fa \
    --gene-pool /local/path/gene_recurrence.tsv

# Refresh manifest md5 for is_frozen=true rows
uv run python scripts/refresh_manifest.py

Manifest scope

data/manifest.tsv column is_frozen distinguishes in-scope vs out-of-scope reproducibility rows:

• is_frozen=true (5 rows): local repo-committed deliverables; md5 and size populated; covered by this repo's reproducibility contract.
• is_frozen=false (13 rows): upstream HPC / external resources; size=TBD and md5=TBD are intentional — the file lives outside this repo. External rebuild requires server access and is not part of the reproducibility contract.

scripts/refresh_manifest.py hashes only is_frozen=true rows and passes is_frozen=false rows through unchanged.

Error recovery

Running a script without providing a path (and without a manifest row resolvable on the current machine) produces a self-describing error:

FileNotFoundError: Required file not accessible: /home/h3036/database/Pfam/Pfam-A.hmm
This path is from manifest.tsv row 'ref_pfam_db' with is_frozen=false (external upstream resource on HPC).
To resolve: either (a) pass --pfam-db /local/path (b) export RICE_PFAM_DB=/local/path (c) mount HPC storage.

Use any of the three options listed. The is_frozen=false flag in the manifest row stays as-is because the logical file remains external.

Directory Structure

Directory	Purpose
core/	L_eff table engine, gene resolver, analytical pg_raw engine; l9_interaction implemented; l1-l8 are P1 placeholder stubs (see module headers)
calculator/	Analytical N* backend + Streamlit app
data/	Genome catalogs (gene_leff_table.tsv 61k genes, gene_pool_cloned.tsv ~4.5k genes)
priors/	tau dual-mode JSON (conservative/standard)
constraints/	parameter_freeze.yaml / data_compliance.yaml / mvp_scope.yaml
scripts/	build_gene_catalogs.py, refresh_manifest.py, sanity checks
tests/	132 pytest tests (formula/regression/schema/smoke/docs/repro mix — NOT a coverage claim)
outputs/	Per-run snapshots (auto-created)

Key Parameters (parameter_freeze.yaml)

Parameter	Value	Source
μ_global	3.24e-7 /bp/plant (zygote-level)	WGS 100 M1 strict (12,598 variants / 389 Mb / 100 plants)
tissue_fraction	DEPRECATED (Fix R1 2026-04-18 C.1)	Removed from main p_g_raw path; μ is now zygote-level
tau_conservative	0.30	priors/tau_dual_mode.json
tau_standard	0.50	priors/tau_dual_mode.json
p_per_plant_detect	0.98 (Fix R1 C.3)	Replaces legacy VAF_kasp=0.1; per-plant KASP detection
BADH2 p_g_raw	5.67e-4	3.24e-7 × 1751 bp (zygote-level; Fix R1 C.1)
f_region	1.0 (P0 baseline; 0.93 pessimistic bound via compute_f_region_sensitivity)	R11A ATAC descriptive overlap ≈ 0.93 (L2 enrichment p=0.147, non-significant; 0.93 is NOT a rate ratio — see R2 CR1)
median_fallback policy	reject by default (opt-in allow_median_fallback=True)	Fail-closed validator — LOC must be in data/gene_leff_table.tsv (Fix R2 v2 C.8)

Fix R1 Group A (2026-04-18) Summary

• C.1: μ is now interpreted as zygote-level; p_g_raw = μ × L_eff × f_region (tissue_fraction removed from main N* path).
• C.2: M3 same-line joint co-fixation p_seed_cofix(K, r) applied per PMF cell; line-level factors s_M2_survival × s_M2_pheno applied ONCE per plant after PMF (not K times inside q_j).
• C.3: VAF_kasp renamed to p_per_plant_detect (default retuned 0.1 → 0.98). Legacy VAF_kasp kwarg accepted with a DeprecationWarning; conflicting dual-key values raise ValueError.
• MUST-fix B: 27-cell heatmap third axis = n_pool_plants ∈ {1, 3, 5} (Pool size — breeding-actionable operational sensitivity).
• MUST-fix C: _validate_target_list rejects duplicate/unresolvable targets (ValueError) and warns on same-chromosome distinct targets.

Fix R2 Group B (2026-04-18) Summary

• C.7: dedicated compute_f_region_sensitivity(f_region_grid=(0.93, 1.0)) helper sweeps ATAC bias; compute_n_star(f_region=...) is rejected with a DeprecationWarning (baked-in-p_g_raw is the sole source of truth).
• C.8: compute_n_star rejects targets that trigger median_fallback by default. Opt in via allow_median_fallback=True for exploratory workflow. The validator also fails closed on engine_results missing keys.

Test Status

132/132 pytest pass (v0.3.0-dev post-R4 Week 2 Module 1)

Evolution
  53/53   v0.2.0 baseline
  93/93   post-R1 Fix
  124/124 post-R2 Fix R2 (+31; ~68% grep-style per R3 audit)
  125/125 +R4 Day 1 C2 live md5+size verify guard
  126/126 +R4 Day 2 C1 tau 87/127 non-estimator guard
  131/131 +R4 Day 3 M8 schema-v2 + M16 v4_split raise guards (5 tests)
  132/132 +R4 Week 2 Module 1 M1/M6 legacy↔canonical alias regression guard

Top-level categories (approximate; not a coverage claim — see §M14):
  ~70 core formula + regression (tests/test_calculator_smoke,
       test_genome_pg, test_interaction_matrix, test_layer_smoke)
  ~25 schema / data-compliance (tests/test_data_compliance)
  ~10 Methods-docs grep (tests/test_r5_methods, test_r2_critical_fixes)
  ~11 R4 reproducibility + safety (tests/test_repro)