Skip to content

Commit 1053848

Browse files
cschanhniemOmX
andcommitted
fix: broken Makefile continuation lines + benchmark status table
Three Makefile targets (bench-cluster-split, bench-expert-ablation, bench-selectivity) had missing backslashes on continuation lines, making them fail with "No rule to make target" or shell errors. Also adds a Status column to the BENCHMARKING.md required benchmarks table, marking all implemented benchmarks and adding the two missing entries (expert ablation, multi-class triage). Co-authored-by: OmX <noreply@omx.local>
1 parent c3f0a46 commit 1053848

2 files changed

Lines changed: 21 additions & 19 deletions

File tree

‎Makefile‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -138,20 +138,20 @@ validate-scoring-phase3:
138138
--out outputs/validate_scoring_report_phase3.json
139139

140140
bench-cluster-split:
141-
PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli bench cluster-split
142-
--amp-csv examples/validation/known_amps.csv
143-
--decoy-csv examples/validation/random_background.csv
141+
PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli bench cluster-split \
142+
--amp-csv examples/validation/known_amps.csv \
143+
--decoy-csv examples/validation/random_background.csv \
144144
--out outputs/cluster_split_report.json
145145

146146
bench-expert-ablation:
147-
PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli bench expert-ablation
148-
--amp-csv examples/validation/known_amps.csv
149-
--decoy-csv examples/validation/random_background.csv
147+
PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli bench expert-ablation \
148+
--amp-csv examples/validation/known_amps.csv \
149+
--decoy-csv examples/validation/random_background.csv \
150150
--out outputs/expert_ablation_report.json
151151

152152
bench-selectivity:
153-
PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli bench selectivity
154-
--hemolysis-csv examples/validation/hemolysis_reference.csv
153+
PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli bench selectivity \
154+
--hemolysis-csv examples/validation/hemolysis_reference.csv \
155155
--out outputs/selectivity_benchmark_report.json
156156

157157

‎docs/BENCHMARKING.md‎

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -15,17 +15,19 @@ Common artifacts:
1515

1616
## Required benchmarks
1717

18-
| Benchmark | Purpose |
19-
|---|---|
20-
| Random baseline | Prove the pipeline is not noise |
21-
| Simple physicochemical baseline | Prove models beat charge/hydrophobicity alone |
22-
| Cluster split | Prevent near-duplicate leakage |
23-
| Cluster-aware CI | Honest bootstrap when positives contain near-duplicate families |
24-
| Time split | Test future generalization where metadata allows |
25-
| Hidden-positive recovery | Check whether known actives are ranked high |
26-
| Toxicity down-ranking | Ensure predicted risky candidates are penalized |
27-
| Novelty stress test | Avoid near-clones of references |
28-
| Within-AMP selectivity | Test whether scorers distinguish hemolytic from selective AMPs |
18+
| Benchmark | Purpose | Status |
19+
|---|---|---|
20+
| Random baseline | Prove the pipeline is not noise | Implemented |
21+
| Simple physicochemical baseline | Prove models beat charge/hydrophobicity alone | Implemented |
22+
| Cluster split | Prevent near-duplicate leakage | Implemented (`bench cluster-split`) |
23+
| Cluster-aware CI | Honest bootstrap when positives contain near-duplicate families | Implemented (cluster-aware bootstrap in `bench cluster-split`) |
24+
| Time split | Test future generalization where metadata allows | Deferred (metadata not available) |
25+
| Hidden-positive recovery | Check whether known actives are ranked high | Implemented |
26+
| Toxicity down-ranking | Ensure predicted risky candidates are penalized | Implemented |
27+
| Novelty stress test | Avoid near-clones of references | Implemented |
28+
| Within-AMP selectivity | Test whether scorers distinguish hemolytic from selective AMPs | Implemented (`bench selectivity`) |
29+
| Expert ablation | Test whether expert composite adds value over ensemble | Implemented (`bench expert-ablation`) |
30+
| Multi-class triage | Test selective > hemolytic > decoy ranking in one panel | Implemented (`bench triage`) |
2931

3032
## Minimum report fields
3133

0 commit comments

Comments
 (0)