-
Notifications
You must be signed in to change notification settings - Fork 1
Add webhook notifications for evaluation failures #5
Copy link
Copy link
Open
Labels
effort: mediumHalf day, some familiarity neededHalf day, some familiarity neededenhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomers
Milestone
Description
When an evaluation fails (score below threshold), there's no way to get notified. Teams running Iris in production need alerting.
Goal: Add optional webhook notifications when evaluations fail.
Acceptance criteria:
- New config option webhooks in config.json
- POST to configured URL when any evaluation scores below threshold
- Webhook payload includes: trace_id, eval_type, score, failed rules, timestamp
- Retry logic (3 attempts with exponential backoff)
- Configurable threshold per eval type
Resources:
- Eval engine: src/eval/
- Custom rules guide
- API reference
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
effort: mediumHalf day, some familiarity neededHalf day, some familiarity neededenhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomers