-
Notifications
You must be signed in to change notification settings - Fork 339
[Docs] Proposal for a practical deep-search / retrieval debugging companion guide #50
Description
Hi, I’d like to suggest a practical companion resource that may be helpful for users dealing with deep-search and retrieval-heavy workflows.
The resource is the WFGY RAG 16 Problem Map, a framework-agnostic troubleshooting checklist focused on common downstream failure modes in LLM and retrieval pipelines.
Reference:
https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md
It focuses on the kinds of issues that often appear after the search system technically “works,” but results still feel unreliable, such as:
- relevant material not being surfaced
- stale or low-quality evidence dominating retrieval
- chunking and embedding mismatch
- ranking instability
- context overload
- grounded evidence present, but final answer still drifts
- multi-step reasoning becoming inconsistent
I think this may be useful as a practical companion reference because users often need a structured way to reason about “why a search-backed answer failed,” not just how to run the system.
This Problem Map has already been referenced or integrated by several public RAG / LLM ecosystem projects, including:
- RAGFlow
- LlamaIndex
- ToolUniverse (Harvard MIMS Lab)
- Rankify
- Multimodal RAG Survey (QCRI LLM Lab)
If this sounds useful, I’d be happy to open a small docs-only PR with concise wording and repo-style scope.
Thanks for your time.