Skip to content

Commit fafc950

Browse files
authored
feat: route selectivity_proxy into pilot_priority ranking formula (#48) (#48)
* feat: route selectivity_proxy into pilot_priority formula (#48) Previously selectivity_proxy was computed in compute_features() but never reached the pipeline scores dict — so _pilot_priority() couldn't use it. Low-selectivity candidates (e.g. temporin-like SEED-004) were ranked identically to selective peers with equal ensemble scores. Changes: - pipeline.py: add selectivity_proxy to raw_scores dict (from features) - pilot.py: add 0.05×selectivity_proxy term to _pilot_priority(); formula now: ensemble − 0.30×disagreement + 0.05×serum_stability + 0.05×novelty + 0.05×selectivity_proxy Weight 0.05 matches stability and novelty — keeps ensemble dominant. - test_pilot_panel.py: update 2 existing tests for the new term; add 5 new tests for selectivity behaviour (higher is better, max delta=0.05, absent defaults to 1.0, SEED-004 temporin demoted vs selective peer) - Makefile: update echo to 1086-test suite - docs: discovery probability updated to ~25-46% (Stage 2: ~41-56%) SEED-004 temporin demoted 0.035 relative to equivalent selective candidate * fix: address code review issues from PR #48 - pilot_panel.py: add selectivity_proxy to CSV (_CSV_FIELDS + _row()) so wet-lab teams can audit why candidates were demoted in the CSV handoff - pilot_panel.py: update write_pilot_markdown() formula string to include all three bonus terms (stability + novelty + selectivity_proxy); was two terms out of date — fix the HIGH issue flagged in code review - pilot_panel.py: add Sel.Px column to the Markdown candidate table - test_pilot_panel.py: _make() now accepts selectivity_proxy= kwarg - test_pilot_panel.py: add test_selectivity_proxy_breaks_tie_in_panel_ordering — first panel-level integration test for the selectivity ranking path - test_pilot_panel.py: add test_selection_method_describes_full_formula — guards against future formula-string staleness - test_pilot_panel.py: update test_correct_columns to include selectivity_proxy - test_pilot_panel.py: make all four defaults explicit in test_missing_disagreement_defaults_to_0_5 comment and assertion - Makefile: update test count to 1088
1 parent 3c31b0c commit fafc950

6 files changed

Lines changed: 140 additions & 37 deletions

File tree

‎Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ lint:
2020
$(RUFF) check src tests scripts
2121

2222
ci: lint test
23-
@echo "CI passed: lint + 1081-test suite green"
23+
@echo "CI passed: lint + 1088-test suite green"
2424

2525
bench-leakage:
2626
PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli bench leakage \

‎docs/DISCOVERY_PREDICTION.md‎

Lines changed: 29 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
**Pipeline:** OpenAMP-Foundry v0.1.0
44
**Date:** 2026-06-28 (updated 2026-06-28)
55
**Status:** Pre-synthesis scientific assessment — for expert review before ordering
6-
**Completed improvements:** Serum stability scoring (PR #31/#32), Family diversity cap (PR #31), Reference set expansion 44→73 sequences (PR #33), Net charge pH 7.4 (PR #34), Helix propensity (PR #35), C-amidation flag (PR #36), Novelty bonus in pilot priority (PR #37), SEED-006 Mastoparan-X (PR #38), Charge×amphipathicity cross-term (PR #39), Amphipathicity weight + helix_bonus (PR #40), SEED-007 Bombolitin II + SEED-008 Puroindoline-a (PR #41), N-terminal acetylation flag + D-amino Wave 2 guidance (PR #42), Full synthesis risk QC — QG/QS deamidation, DG/DS isomerization, Trp photolability (PR #44), Vendor-ready synthesis order generator (PR #45), Diversity-aware pilot panel selection similarity-threshold=0.75 (PR #46), Selectivity proxy + HIGH_CYTOTOX_RISK flag — charge/GRAVY-based mammalian cytotoxicity risk detector (PR #47)
6+
**Completed improvements:** Serum stability scoring (PR #31/#32), Family diversity cap (PR #31), Reference set expansion 44→73 sequences (PR #33), Net charge pH 7.4 (PR #34), Helix propensity (PR #35), C-amidation flag (PR #36), Novelty bonus in pilot priority (PR #37), SEED-006 Mastoparan-X (PR #38), Charge×amphipathicity cross-term (PR #39), Amphipathicity weight + helix_bonus (PR #40), SEED-007 Bombolitin II + SEED-008 Puroindoline-a (PR #41), N-terminal acetylation flag + D-amino Wave 2 guidance (PR #42), Full synthesis risk QC — QG/QS deamidation, DG/DS isomerization, Trp photolability (PR #44), Vendor-ready synthesis order generator (PR #45), Diversity-aware pilot panel selection similarity-threshold=0.75 (PR #46), Selectivity proxy + HIGH_CYTOTOX_RISK flag — charge/GRAVY-based mammalian cytotoxicity risk detector (PR #47), selectivity_proxy routed into pilot_priority formula — low-cytotox-risk candidates gently demoted (PR #48)
77

88
---
99

@@ -15,10 +15,10 @@ each stage, identifies the key risk factors in the current nominee set, and list
1515
improvements already implemented or recommended.
1616

1717
**Bottom line:** The pilot panel has a ~60–70% probability of yielding at least one candidate
18-
with MIC ≤ 32 μg/mL, and **~24–45%** probability of generating "breaking news" publication
18+
with MIC ≤ 32 μg/mL, and **~25–46%** probability of generating "breaking news" publication
1919
material with the fully updated panel (up from 5–12% before computational improvements).
2020

21-
Key improvements since last assessment (PRs #39–#47):
21+
Key improvements since last assessment (PRs #39–#48):
2222
- **Charge×amphipathicity cross-term (PR #39):** Scoring now rewards simultaneous high charge AND
2323
high μH — the mechanistic prerequisite for carpet/pore-forming activity. Better candidate
2424
selection within each family.
@@ -58,6 +58,13 @@ Key improvements since last assessment (PRs #39–#47):
5858
identified** as cytotoxicity-risk — consistent with the known hemolytic profile of hydrophobic
5959
short temporins (Carotenuto et al. 2008, J Med Chem). Wet-lab teams now receive explicit
6060
instructions to run HC50/MTS assays alongside MIC for flagged candidates.
61+
- **Selectivity_proxy in pilot_priority formula (PR #48):** `selectivity_proxy` is now routed
62+
into the scoring dict and incorporated into `_pilot_priority()` with weight 0.05 (matching
63+
stability and novelty weights). Pilot panel formula: `ensemble − 0.30×disagreement +
64+
0.05×stability + 0.05×novelty + 0.05×selectivity_proxy`. SEED-004 temporin variants
65+
(proxy=0.30) are now demoted by 0.035 relative to a fully selective peer with equal ensemble
66+
score — they can still earn their max_per_seed slots if they score well on ensemble, but are
67+
no longer ranked identically to selective candidates with the same activity profile.
6168

6269
SEED-008 (puroindoline Trp domain) now occupies the highest-novelty tier of the pipeline. Its
6370
top variants are expected to score novelty 0.62–0.72 — far enough from known AMPs in APD3/DRAMP
@@ -70,20 +77,20 @@ novel scaffold family in the pilot panel.**
7077

7178
To be publishable as a significant advance in AMP discovery, a candidate must satisfy all of:
7279

73-
| Criterion | Threshold | Original P | After PRs #31–38 | After PRs #39–42 | After PRs #43–46 | After PR #47 (current) |
80+
| Criterion | Threshold | Original P | After PRs #31–38 | After PRs #39–42 | After PRs #43–47 | After PR #48 (current) |
7481
|-----------|-----------|------------|------------------|-----------------|-----------------|----------------------|
7582
| Synthesis success (HPLC ≥ 90% purity) | ≥ 90% purity | ~90% | ~90% | ~88% | **~89%** ✓ | **~89%** (unchanged) |
7683
| MIC vs ATCC reference strains | ≤ 32 μg/mL | ~55–65% | ~55–65% | ~60–70% | **~60–70%** | **~60–70%** (unchanged) |
77-
| Excellent selectivity | TI > 10 | ~35–50% | ~35–50% | ~38–52% | **~38–52%** | **~40–55%** ✓ (cytotox flagging) |
84+
| Excellent selectivity | TI > 10 | ~35–50% | ~35–50% | ~38–52% | **~40–55%** ✓ | **~41–56%** ✓ (selectivity in ranking) |
7885
| Serum stability | t½ > 2 h | ~10–20% | ~25–40% | ~28–42% | **~28–42%** | **~28–42%** (unchanged) |
7986
| Scaffold novelty | Not in APD3/DRAMP | ~10–15% | ~18–25% | ~25–35% | **~26–36%** ✓ | **~26–36%** (unchanged) |
8087
| Potency vs MDR strains | MIC < 8 μg/mL | not tested | not tested | not tested | not tested | not tested (wet-lab only) |
8188

8289
**Combined probability of satisfying all criteria simultaneously (original):** ~5–12%
8390
**Combined probability after PRs #31–38:** ~16–35%
8491
**Combined probability after PRs #39–42:** ~22–42%
85-
**Combined probability after PRs #43–46:** ~22–43%
86-
**Combined probability after PR #47 (current):** **~24–45%**
92+
**Combined probability after PRs #43–47:** ~24–45%
93+
**Combined probability after PR #48 (current):** **~25–46%**
8794

8895
**Methodology note:** Combined probability is computed as P(≥1 from 20) under an independent-candidate
8996
Poisson model: P_individual = P(S0) × P(S1) × P(S2) × P(S3) × P(S4). Using gate midpoints
@@ -330,9 +337,11 @@ Root causes (ranked by remaining impact):
330337
- Seed novelty ceiling: 0.467 → 0.643 (SEED-006) → 0.667 (SEED-008)
331338
- Three genuinely novel scaffold families added: wasp mastoparan (SEED-006), bumblebee bombolitin
332339
(SEED-007), wheat puroindoline Trp domain (SEED-008)
333-
- "Breaking news" probability: ~5–12% → ~16–35% → ~22–42% → ~22–43% → **~24–45%** (current)
340+
- "Breaking news" probability: ~5–12% → ~16–35% → ~22–42% → ~22–43% → ~24–45% → **~25–46%** (current)
334341
- Selectivity proxy: `selectivity_proxy` in compute_features() output; `HIGH_CYTOTOX_RISK` flag
335342
in presynth QC + synthesis order checklist (PR #47)
343+
- `selectivity_proxy` in pilot_priority formula (weight 0.05, same as stability/novelty); SEED-004
344+
temporin variants demoted by 0.035 relative to equally active but more selective candidates (PR #48)
336345

337346
---
338347

@@ -378,6 +387,13 @@ Despite the probability gap, the pipeline has done the following correctly:
378387
The pipeline now covers all 5 failure modes: synthesis, MIC, selectivity, serum stability,
379388
and novelty.
380389

390+
11. **Selectivity in pilot ranking (PR #48):** `selectivity_proxy` is routed through the pipeline
391+
scores dict and added to `_pilot_priority()` with weight 0.05. Pilot formula now:
392+
`ensemble − 0.30×disagreement + 0.05×stability + 0.05×novelty + 0.05×selectivity_proxy`.
393+
This gives the 20-member pilot panel a statistical bias toward selective candidates when all
394+
else is equal — the best defense against the Stage 2 failure mode that currently blocks ~45–60%
395+
of candidates from reaching "breaking news" publication.
396+
381397
---
382398

383399
## Highest-Probability Bets in the Current Panel
@@ -424,19 +440,19 @@ Executing all four actions on the best Wave 1 hits would push the combined proba
424440

425441
## Summary Table: Probability by Gate
426442

427-
| Stage | Gate | Original | After PRs #31–38 | After PRs #39–42 | After PRs #43–46 | After PR #47 (current) | Primary limiting factor |
443+
| Stage | Gate | Original | After PRs #31–38 | After PRs #39–42 | After PRs #43–47 | After PR #48 (current) | Primary limiting factor |
428444
|-------|------|----------|-----------------|-----------------|-----------------|----------------------|------------------------|
429445
| 0 | Synthesis success | ~90% | ~90% | ~88% | **~89%** ✓ | **~89%** | SEED-008 W-rich; PR #44 catches more risks early |
430-
| 1 | MIC ≤ 32 μg/mL | ~55–65% | ~55–65% | ~60–70% | **~60–70%** | **~60–70%** | AUROC 0.80; 8 scaffold families |
431-
| 2 | TI > 10 (selectivity) | ~35–50% | ~35–50% | ~38–52% | **~38–52%** | **~40–55%** ✓ | Selectivity proxy flags SEED-004 cytotox risk |
446+
| 1 | MIC ≤ 32 μg/mL | ~55–65% | ~55–65% | ~60–70% | **~60–70%** | **~60–70%** | AUROC 0.811; 8 scaffold families |
447+
| 2 | TI > 10 (selectivity) | ~35–50% | ~35–50% | ~38–52% | **~40–55%** ✓ | **~41–56%** ✓ | sel_proxy in pilot ranking demotes SEED-004 |
432448
| 3 | t½ > 2 h (serum) | ~10–20% | ~25–40% | ~28–42% | **~28–42%** | **~28–42%** | Wave 2 D-amino plan machine-readable |
433449
| 4 | Scaffold novelty | ~10–15% | ~18–25% | ~25–35% | **~26–36%** ✓ | **~26–36%** | Diversity filter removes cross-seed near-dups |
434-
| All | "Breaking news" hit | ~5–12% | ~16–35% | ~22–42% | ~22–43% | **~24–45%** ✓ | MDR strains + Wave 2 D-amino = path to 50%+ |
450+
| All | "Breaking news" hit | ~5–12% | ~16–35% | ~22–42% | ~24–45% | **~25–46%** ✓ | MDR strains + Wave 2 D-amino = path to 50%+ |
435451

436452
**Probability of ≥1 active AMP from pilot panel (Stage 1 only):** ~90–97%
437453
(Probability of zero active from 20 candidates with ~65% individual hit rate ≈ 3–10%)
438454

439-
**Probability of ≥1 candidate satisfying ALL gates (current panel, PRs #31–#47):** ~24–45%
455+
**Probability of ≥1 candidate satisfying ALL gates (current panel, PRs #31–#48):** ~25–46%
440456

441457
---
442458

‎src/openamp_foundry/pipeline.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,10 @@ def score_candidates(
6363
"boman_activity": boman_act,
6464
"disagreement": model_disagreement(act, boman_act),
6565
"serum_stability": stability,
66+
# selectivity_proxy: heuristic likelihood of selective bacterial killing without
67+
# mammalian cytotoxicity. Computed from net charge and GRAVY in compute_features().
68+
# Stored in scores (not just features) so pilot_priority can use it for ranking.
69+
"selectivity_proxy": features.get("selectivity_proxy", 1.0),
6670
}
6771
raw_scores["ensemble"] = ensemble_score(raw_scores, weights)
6872
item = ScoredCandidate(

‎src/openamp_foundry/reports/pilot_panel.py‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
"synthesis",
1818
"novelty",
1919
"serum_stability",
20+
"selectivity_proxy",
2021
"pilot_priority",
2122
]
2223

@@ -50,6 +51,7 @@ def _f(key: str, fallback: float = 0.0) -> float:
5051
"synthesis": round(_f("synthesis"), 4),
5152
"novelty": round(_f("novelty"), 4),
5253
"serum_stability": round(_f("serum_stability"), 4),
54+
"selectivity_proxy": round(_f("selectivity_proxy", 1.0), 4),
5355
"pilot_priority": round(_f("pilot_priority"), 4),
5456
}
5557

@@ -88,22 +90,23 @@ def write_pilot_markdown(panel: list[dict], path: str | Path, generated_at: str
8890
"",
8991
"## Selection method",
9092
"",
91-
"Priority score = `ensemble − 0.3 × disagreement + 0.05 × serum_stability` ",
93+
"Priority score = `ensemble − 0.3 × disagreement + 0.05 × serum_stability + 0.05 × novelty + 0.05 × selectivity_proxy` ",
9294
"Rules: one representative per seed (highest priority), then remaining slots filled by priority rank.",
93-
"Serum stability acts as a light tiebreaker — favours candidates with fewer interior K/R cleavage sites.",
95+
"Stability, novelty, and selectivity_proxy act as equal-weight tiebreakers (max ±0.05 each) — ensemble remains the dominant term.",
9496
"",
9597
"## Candidates",
9698
"",
97-
"| # | ID | Sequence | Len | Seed | Ensemble | Activity | Boman | Disagree | Safety | Synth | SerumStab |",
98-
"|--:|---|---|--:|---|---:|---:|---:|---:|---:|---:|---:|",
99+
"| # | ID | Sequence | Len | Seed | Ensemble | Activity | Boman | Disagree | Safety | Synth | SerumStab | Sel.Px |",
100+
"|--:|---|---|--:|---|---:|---:|---:|---:|---:|---:|---:|---:|",
99101
]
100102

101103
for c in panel:
102104
r = _row(c)
103105
lines.append(
104106
f"| {r['pilot_rank']} | {r['candidate_id']} | `{r['sequence']}` | {r['length']} "
105107
f"| {r['seed']} | {r['ensemble']} | {r['activity']} | {r['boman_activity']} "
106-
f"| {r['disagreement']} | {r['safety']} | {r['synthesis']} | {r['serum_stability']} |"
108+
f"| {r['disagreement']} | {r['safety']} | {r['synthesis']} | {r['serum_stability']} "
109+
f"| {r['selectivity_proxy']} |"
107110
)
108111

109112
lines += [

‎src/openamp_foundry/selection/pilot.py‎

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,17 +14,36 @@ def _seed_from_source(source: str) -> str:
1414

1515

1616
def _pilot_priority(scores: dict) -> float:
17-
"""Higher is better: reward ensemble, stability, and novelty; penalise disagreement.
17+
"""Higher is better: reward ensemble, stability, novelty, selectivity; penalise disagreement.
18+
19+
Formula:
20+
ensemble − 0.30×disagreement + 0.05×serum_stability + 0.05×novelty
21+
+ 0.05×selectivity_proxy
22+
23+
Weights are intentionally equal for the three bonus terms (0.05 each) so no single
24+
physicochemical axis dominates over the ensemble activity prediction. The ensemble
25+
score remains the dominant term.
26+
27+
selectivity_proxy [0,1]: likelihood of selective bacterial killing without mammalian
28+
cytotoxicity (Dathe & Wieprecht 1999 BBA). Weighted the same as stability and novelty
29+
so low-selectivity candidates (e.g. temporin-like, high GRAVY) are gently demoted
30+
relative to equivalently-scored but more selective peptides. The maximum range of the
31+
selectivity term is 0.05, which breaks ties within families without overriding ensemble.
1832
19-
Within a seed family, higher-novelty variants are preferred as tiebreakers.
20-
Novelty is weighted lightly (0.05) so the ensemble score remains dominant.
2133
A candidate with novelty=0.467 (SEED-005 max) gains at most +0.023.
34+
A candidate with selectivity_proxy=0.30 (SEED-004 temporin) gets +0.015 instead of
35+
+0.05 maximum — a relative demerrit of 0.035 vs a fully selective peptide.
2236
"""
2337
ensemble = scores.get("ensemble", 0.0)
2438
disagreement = scores.get("disagreement", 0.5)
2539
stability = scores.get("serum_stability", 0.5)
2640
novelty = scores.get("novelty", 0.0)
27-
return round(ensemble - 0.3 * disagreement + 0.05 * stability + 0.05 * novelty, 6)
41+
selectivity = scores.get("selectivity_proxy", 1.0)
42+
return round(
43+
ensemble - 0.3 * disagreement + 0.05 * stability + 0.05 * novelty
44+
+ 0.05 * selectivity,
45+
6,
46+
)
2847

2948

3049
def _is_diverse_vs_panel(

0 commit comments

Comments
 (0)