feat(comprehension): ingestion-time LLM digests so ask/chat answer structure + combination questions

[northdpole]

Stacked on #1155.

Why

#1155 answered structural questions (e.g. SAMM levels) from hand-written facts, which cannot scale and was partly wrong. This derives that knowledge at ingestion instead.

How

• application/utils/comprehension/: reads whole harvested documents, extracts claims that must include a verbatim quote found in the source (unverified claims dropped), builds a per-standard overview that must cite existing documents, and stores everything as embedded evidence docs in the existing agent channel.
• Query time: router hook synthesizes an answer from the top comprehension hits with citations; UNKNOWN falls through to existing paths. Flag: OWASP_COMPREHENSION_ENABLED (default on).
• scripts/comprehend_standard.py --repo OWASP/SAMM runs it per harvested repo.
• Design note: docs/oie/comprehension-design.md.

Verified on staging (homeserver)

SAMM/ASVS/MASVS/AI guide digested; e.g. "how many levels in owasp samm" returns three maturity levels per practice (also surfaces a v1.5 vs v2 conflict, to fix by preferring the latest version), with GitHub citations.

Not done yet

Generated source-grounded eval (Gemini questions, Claude judge), held-out standard test, version preference, removal of hand-written node_facts, people/talks layer, scheduling into the pipeline.

🤖 Generated with Claude Code

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}