Skip to content

intel: reduce false positives in owasp-top-10-web and secrets-management#1

Merged
kamalsrini merged 2 commits into
mainfrom
intel/benchmark-fp-fixes-2026-03-17
Mar 21, 2026
Merged

intel: reduce false positives in owasp-top-10-web and secrets-management#1
kamalsrini merged 2 commits into
mainfrom
intel/benchmark-fp-fixes-2026-03-17

Conversation

@kamalsrini

@kamalsrini kamalsrini commented Mar 17, 2026

Copy link
Copy Markdown
Contributor

Summary

A/B benchmark results identified high false positive rates in two skills. This PR adds precision controls.

Benchmark Results (Before)

Skill Detection FP Rate TP FP
owasp-top-10-web 0.57 0.86 4 24
secrets-management 1.0 0.38 8 5

Changes

  • owasp-top-10-web v1.0.1: Added precision requirements (confirmed code path + file:line reference, exploitability verification) and 5-point pre-classification checklist
  • secrets-management v1.0.1: Added false positive filtering (entropy check, known secret prefix patterns like AKIA*/sk-/ghp_, placeholder detection)

Test Plan

  • Re-run A/B benchmark on both skills to verify FP reduction
  • Verify detection rate maintained or improved
  • Check SKILL.md format compliance (injection-hardened, <500 lines)

🤖 Generated with Claude Code

Source: A/B benchmark framework at ~/.openclaw/workspace/test-corpus/

Ubuntu and others added 2 commits March 17, 2026 07:23
A/B benchmark results showed:
- owasp-top-10-web: 0.86 FP rate (24 false positives on Juice Shop)
- secrets-management: 0.38 FP rate (5 false positives on leaked-secrets)

Changes:
- owasp-top-10-web v1.0.1: Add precision requirements and 5-point
  pre-classification verification checklist
- secrets-management v1.0.1: Add false positive filtering section
  (entropy check, known prefix patterns, placeholder detection)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…erns to gitignore

- injection-scan.yml: Add discord.com/api/webhooks and hooks.slack.com/services patterns
- .gitignore: Add *.secrets, .env*, *.credentials, *.pem, *.key

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@kamalsrini kamalsrini merged commit 6b9f677 into main Mar 21, 2026
3 checks passed
@kamalsrini kamalsrini deleted the intel/benchmark-fp-fixes-2026-03-17 branch March 21, 2026 18:20
jddark62 pushed a commit to jddark62/SecuritySkills that referenced this pull request Jun 5, 2026
…s-2026-03-17

reduce false positives in owasp-top-10-web and secrets-management
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant