Skip to content

feat(kb): support pluggable knowledge base retrieval backends - #9751

Draft
lxfight wants to merge 14 commits into
AstrBotDevs:masterfrom
lxfight:feat/knowledge-base-backend-api
Draft

feat(kb): support pluggable knowledge base retrieval backends#9751
lxfight wants to merge 14 commits into
AstrBotDevs:masterfrom
lxfight:feat/knowledge-base-backend-api

Conversation

@lxfight

@lxfight lxfight commented Aug 20, 2026

Copy link
Copy Markdown
Member

AstrBot plugins can already integrate external knowledge base systems through custom hooks, but there is no stable, shared contract for knowledge base discovery and retrieval.

This PR introduces a minimal public backend API so plugins can expose external knowledge bases through a consistent interface without depending on AstrBot's built-in storage implementation. The scope is intentionally limited to enabled knowledge base discovery and read-only retrieval.

Modifications / 改动点

  • Added public plugin contracts for knowledge base backends:

    • BaseKnowledgeBaseBackend
    • KnowledgeBaseRef
    • KnowledgeBaseInfo
    • KnowledgeBaseQuery
    • KnowledgeBaseHit
    • KnowledgeBaseResponse
    • Standard backend error types
  • Added backend registration and unregistration through the plugin Context.

  • Added an adapter that exposes the built-in knowledge base through the same contract without replacing the existing built-in Agent flow.

  • Added concurrent multi-backend retrieval with:

    • Per-backend timeouts
    • Failure isolation
    • Rank-based result merging
    • Global top_k truncation
    • Runtime validation of third-party responses
  • Added first-class knowledge base identity to every retrieval hit.

  • Integrated enabled external backends into the existing Agent knowledge base flow.

  • Defined list_knowledge_bases() as returning only knowledge bases that are enabled and accessible for the current session.

  • Added explicit plugin reload lifecycle behavior. Plugins unregister their backends in terminate() before releasing resources.

  • Added Chinese and English plugin development documentation with a complete backend example.

  • Added unit and adversarial tests for registration, discovery, retrieval, failure isolation, malformed responses, result identity, and reload behavior.

  • This is NOT a breaking change. / 这不是一个破坏性变更.

Scope

This PR intentionally does not standardize:

  • Knowledge base creation or deletion
  • Document upload or management
  • Chunk or index management
  • Backup and restore
  • Credential configuration
  • Unified WebUI management

Plugins may provide those capabilities through their own configuration, commands, or Plugin Pages. Optional management contracts can be considered separately when concrete cross-backend requirements are available.

The existing AstrBot knowledge base database, upload flow, Dashboard APIs, and backup behavior are unchanged.

Screenshots or Test Results / 运行截图或测试结果

Verification performed:

uv run ruff format .
503 files left unchanged

uv run ruff check .
All checks passed

Knowledge base test suite:
51 passed

Unit test suite:
906 passed

Two existing Dashboard log-capture tests fail locally because warnings are emitted through the configured logger instead of caplog. The same failures were reproduced on upstream/master and are unrelated to this PR.

The Chinese and English VitePress documentation build also completed successfully.


Checklist / 检查清单

  • 😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
    / 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。

  • 👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
    / 我的更改经过了良好的测试,并已在上方提供了“验证步骤”和“运行截图”

  • 🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in requirements.txt and pyproject.toml.
    / 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到 requirements.txtpyproject.toml 文件相应位置。

  • 😮 My changes do not introduce malicious code.
    / 我的更改没有引入恶意代码。

Summary by Sourcery

Enable plugins to provide external knowledge bases through a shared, resilient retrieval backend contract.

New Features:

  • Add a public plugin API for discovering and performing read-only retrieval from external knowledge bases.
  • Integrate enabled external backend results into the Agent knowledge base context alongside built-in results.

Enhancements:

  • Provide concurrent multi-backend retrieval with timeouts, failure isolation, rank-based merging, global result limits, and response validation.
  • Expose the built-in knowledge base through the shared backend contract while preserving its existing retrieval flow.
  • Add backend lifecycle registration through plugin context and preserve backend identity across discovery and retrieval results.

Documentation:

  • Add English and Chinese plugin documentation with a complete external knowledge base backend example and API usage guidance.

Tests:

  • Add unit and adversarial coverage for backend contracts, registration, discovery, retrieval orchestration, malformed responses, failure isolation, result identity, built-in adaptation, and plugin reload behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant