Skip to content

feat: Phase W W3 similarity challenge harness schema (SCH-) - #1005

Merged
cschanhniem merged 1 commit into
mainfrom
feat/w3-similarity-challenge-harness
Jul 10, 2026
Merged

cschanhniem merged 1 commit into
mainfrom
feat/w3-similarity-challenge-harness

Conversation

@cschanhniem

Copy link
Copy Markdown
Collaborator

Summary

  • Adds SCH- (similarity challenge harness) schema: checks whether pipeline-selected candidates are systematically more similar to known AMPs than a random draw from the same sequence space
  • Verdict: selection_adds_value (gap≥0.10), marginal_improvement (0.03–0.10), proximity_driven (<0.03), challenge_not_run (n=0)
  • SimilarityGroupStats helper stores group label, mean similarity to known AMPs, and n_sequences
  • similarity_gap auto-computed and validated (pipeline_mean - random_mean)
  • 4 similarity metrics (sequence_identity, blosum62_score, edit_distance, physicochemical_distance)
  • Prevents "novel panel" claim when selection is proximity-driven
  • dry_lab_only=True enforced; 60 tests

Test plan

  • python3 -m pytest tests/evidence/test_similarity_challenge_harness.py -- 60 passed

…proximity-driven selection vs random draw; VALID_SCH_VERDICTS (4), SELECTION_VALUE_GAP_THRESHOLD=0.10, SimilarityGroupStats helper; gap auto-computed; dry_lab_only=True; 60 tests
@cschanhniem
cschanhniem merged commit dcabc17 into main Jul 10, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant