Skip to content

feat: Phase W W2 charge-matched challenge schema (CMC-) - #1004

Merged
cschanhniem merged 1 commit into
mainfrom
feat/w2-charge-matched-challenge
Jul 10, 2026
Merged

cschanhniem merged 1 commit into
mainfrom
feat/w2-charge-matched-challenge

Conversation

@cschanhniem

Copy link
Copy Markdown
Collaborator

Summary

  • Adds CMC- (charge-matched challenge) schema: formally documents the pipeline AUROC vs charge-only baseline comparison
  • Verdict auto-computed: gap_meaningful (≥0.05), gap_marginal (0.02–0.05), gap_absent (<0.02), challenge_not_run (n=0)
  • 4 baseline methods (charge_only_rank, charge_length_rank, charge_hydrophobicity_rank, logistic_charge_only)
  • AUROC gap auto-computed from pipeline_auroc - baseline_auroc; validated against stored value
  • Blocks performance claims when charge-only baseline explains the selection signal
  • dry_lab_only=True enforced; 60 tests

Test plan

  • python3 -m pytest tests/evidence/test_charge_matched_challenge.py -- 60 passed

…pipeline AUROC vs charge-only baseline; VALID_CMC_VERDICTS (4), MEANINGFUL_GAP_THRESHOLD=0.05, auroc_gap auto-computed; blocks performance claims when charge explains the gap; 60 tests
@cschanhniem
cschanhniem merged commit 431eec1 into main Jul 10, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant