Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Nov 16, 2025

概述 / Overview

Extends RAG capabilities with external knowledge bases via HTTP API. Built-in knowledge bases had limited retrieval quality; external KBs allow integration with superior third-party retrieval services (e.g., Dify-compatible APIs).

实现细节 / Implementation Details

Backend Architecture:

  • Abstract KnowledgeBaseInterface for polymorphic KB handling (internal/external)
  • ExternalKnowledgeBase implementation with async HTTP client, Bearer auth support
  • ExternalKnowledgeBase persistence table (uuid, name, api_url, api_key, top_k)
  • Full CRUD endpoints at /api/v1/knowledge/external-bases
  • RAGManager loads both KB types; pipelines retrieve from mixed sources seamlessly

Frontend UI:

  • Tab navigation: Built-in | External (styled to match plugins page design)
  • External KB card list with create/edit dialog
  • Form fields: name, description, API URL, API key (optional), top_k
  • HTTP client methods for external KB operations
  • i18n: EN, ZH-Hans, ZH-Hant, JA

API Contract:

POST {api_url}
Authorization: Bearer {api_key}
Content-Type: application/json

{"query": "text", "top_k": 5}

Response: {"records": [{"content": "...", "score": 0.95, "title": "...", "metadata": {}}]}

Backward Compatibility:

  • Existing internal KBs unchanged
  • Pipeline knowledge-bases array field already supported
  • Legacy single knowledge-base string still works
  • Auto-creates tables on startup

检查清单 / Checklist

PR 作者完成 / For PR author

  • 阅读仓库贡献指引了吗? / Have you read the contribution guide?
  • 与项目所有者沟通过了吗? / Have you communicated with the project maintainer?
  • 我确定已自行测试所作的更改,确保功能符合预期。 / I have tested the changes and ensured they work as expected.

项目维护者完成 / For project maintainer

  • 相关 issues 链接了吗? / Have you linked the related issues?
  • 配置项写好了吗?迁移写好了吗?生效了吗? / Have you written the configuration items? Have you written the migration? Has it taken effect?
  • 依赖加到 pyproject.toml 和 core/bootutils/deps.py 了吗 / Have you added the dependencies to pyproject.toml and core/bootutils/deps.py?
  • 文档编写了吗? / Have you written the documentation?
Original prompt

我们内置的知识库(RAG)检索算法比较简单,效果质量并不好;我们希望能通过“外部知识库”来扩展能力,外部知识库即通过http来检索文档,只需要使用到检索接口;我们假设外部知识库会提供与:https://docs.dify.ai/zh-hans/guides/knowledge-base/api-documentation/external-knowledge-api-documentation 文档定义的格式一样的接口,请你设计支持外部知识库;

请在知识库页面左上角加一个tab list,可以参考插件页的用法;分为两个tab:内置、外部知识库;当切换到 外部知识库时,显示外部知识库的列表,最左上角的卡片是 添加卡片 (可以参考其他页面的组件),点击添加卡片即弹出对话框,供用户输入外部知识库的地址;

后端中,请你做好抽象,在内置知识库和外部知识库上方加一个抽象层?在流水线配置的选择知识库时,需要支持同时选择多个内置知识库或外部知识库;流水线检索时,内部知识库就是使用内部的rag逻辑,外部知识库则直接访问外部知识库的检索api;

注意:上方那个文档仅供参考,请根据 LangBot 内部所需的字段和具体情况来对外部知识库的接口做出定义;


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@RockChinQ RockChinQ marked this pull request as ready for review November 16, 2025 14:46
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. eh: Feature enhance: 新功能添加 / add new features pd: Need testing pending: 待测试的PR / PR waiting to be tested labels Nov 16, 2025
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Nov 16, 2025
Copilot AI changed the title [WIP] Add support for external knowledge base integration Add external knowledge base support for HTTP-based RAG retrieval Nov 16, 2025
Copilot AI requested a review from RockChinQ November 16, 2025 14:56
Copilot finished work on behalf of RockChinQ November 16, 2025 14:56
@RockChinQ
Copy link
Member

@copilot 知识库页面的左上角tab list样式很难看,请参考插件页面的那个tab list样式实现

Copy link
Contributor Author

Copilot AI commented Nov 16, 2025

@copilot 知识库页面的左上角tab list样式很难看,请参考插件页面的那个tab list样式实现

Updated the tab list styling to match the plugins page. Changes in commit abe9795:

  • Added flex container with justify-between and px-[0.8rem] padding
  • Applied shadow-md, py-5, and bg-[#f0f0f0] dark:bg-[#2a2a2e] to TabsList
  • Added px-6 py-4 cursor-pointer to each TabsTrigger

The knowledge base tabs now have the same visual style as the plugins page tabs.

Copilot finished work on behalf of RockChinQ November 16, 2025 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eh: Feature enhance: 新功能添加 / add new features pd: Need testing pending: 待测试的PR / PR waiting to be tested size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants