Skip to content

feat(privacy): support automatic privacy extraction and restore for sessions - #6278

Open
J-StrawHat wants to merge 24 commits into
mainfrom
feat/privacy-policy
Open

J-StrawHat wants to merge 24 commits into
mainfrom
feat/privacy-policy

Conversation

@J-StrawHat

@J-StrawHat J-StrawHat commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Session messages can contain sensitive values that would otherwise be stored as plaintext. This PR extends OpenViking's privacy extraction and restore capabilities to Sessions, following the existing Skill privacy workflow.

Sensitive values are replaced with placeholders before message persistence and tool output externalization. Registered privacy values are reused, with preference given to uniquely matching personal credentials; newly detected values are saved in the current Session's privacy config.

When reading Session content, placeholders are restored from the active privacy configs through the existing Privacy service. Default reads continue to apply standard privacy sanitization.

Human Involvement

  • A human participated in the implementation or review loop
  • This PR was generated entirely by AI agents without human participation in the loop

Related Issue

Extends the Privacy management capabilities introduced in PR #6161.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

  • Add rule-based privacy extraction for message text, context abstracts, and tool input/output, without additional model calls.
  • Integrate Session persistence with versioned privacy configs, preserving existing placeholders and saving newly extracted values before messages.
  • Support restore for Session messages, archived messages, and externalized tool output.
  • Update English and Chinese concept/API documentation and regression tests.

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

Focused validation for the retained changes:

  • Unit and regression tests: 199 Python tests and 6 Rust tests passed, covering Session extraction, credential reuse, placeholder restore, and persistence.
  • Compatibility and boundary tests: 3,500 Markdown/value-boundary cases passed; Python/Rust comparisons passed for 7,177 extraction-rule cases and 6,000 registered-value matching cases.
  • HTTP/storage validation: 81 assertions passed across 97 requests, covering repeated values, version history, persisted placeholders, default sanitized reads, and restore.
  • Real-model validation: 7 checks passed using VikingBot through the Studio proxy and isolated Session storage, verifying actual user input and model responses, clean extracted values, stable version counts, plaintext-free message storage, and read-time restore.

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Screenshots (if applicable)

Before:

screenshot-20261019-225202

After: (Sensitive credentials have been automatically replaced with placeholders)

Snipaste_2026-10-19_23-01-41

Additional Notes

@J-StrawHat J-StrawHat changed the title feat(privacy): protect session credentials with owner-bound references feat(privacy): support automatic privacy extraction and restore for sessions Oct 20, 2026
@ZaynJarvis ZaynJarvis added the kernel Core server, runtime, storage, retrieval, SDK, and CLI. label Oct 20, 2026
@J-StrawHat
J-StrawHat marked this pull request as ready for review October 20, 2026 02:42

@yeshion23333 yeshion23333 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

本次审查确认 4 个需要修复的问题:默认 JSONL 读取可能暴露已经保护的凭据;同批重复凭据会制造歧义并阻断后续消息写入;普通英文被误识别为 Basic 认证信息;此前允许的 Session ID 在首次保存凭据时被拒绝。

验证:130 项相关 Python 测试和 6 项 Rust 测试通过;额外 7 个回归用例复现了上述问题,使用真实的 Session、Privacy、FS 业务代码及原生匹配器,仅以内存适配器替换存储。完整服务测试受本地向量引擎版本限制未完成。具体触发条件和修复建议见 4 条行内评论。

Comment thread openviking/service/fs_service.py
Comment thread openviking/privacy/session_extractor.py
Comment thread crates/ragfs/src/privacy_extractor_rules.json Outdated
Comment thread openviking/privacy/session_extractor.py
@J-StrawHat

Copy link
Copy Markdown
Collaborator Author

根据 review,修复了以下四个问题:

  • 默认读取的凭据保护:默认读取保留存储中的 Privacy 占位符,仅在显式指定 raw=true 时还原,避免包含转义字符的字符串及敏感容器在 JSONL 读取时泄露。
  • 同批重复发现的引用复用:同批自由文本中重复发现的凭据复用同一引用,避免产生歧义配置并阻断后续消息。结构化输入中独立命名的字段保留各自的身份。
  • Basic 认证识别范围:要求明确的认证上下文,避免将普通英文误识别并登记为 Basic 认证信息。
  • Session ID 兼容性:为不符合 Privacy 标识符要求的 Session ID 引入稳定的哈希映射,提取与还原使用相同映射。

同时,还做了如下改动:

  • Skill 和 Session 的还原逻辑已统一,共享引用解析、归属校验和单次读取内的配置缓存,并减少重复正文扫描。
  • 所有 category 的缺失字段、null 和空字符串均保留占位符;
  • 工具输入按原始 JSON 类型还原,无效 Session JSONL 会被明确拒绝。
  • 顺便修复了 MCP 中英文文档的代码块转义问题

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kernel Core server, runtime, storage, retrieval, SDK, and CLI.

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

3 participants