Skip to content

feat: 大文本文件 lazy loading + 24h 文件缓存 - #237

Merged
lishuceo merged 4 commits into
mainfrom
feat/lazy-text-file-loading
May 17, 2026
Merged

feat: 大文本文件 lazy loading + 24h 文件缓存#237
lishuceo merged 4 commits into
mainfrom
feat/lazy-text-file-loading

Conversation

@lishuceo

Copy link
Copy Markdown
Owner

Summary

  • 大文本文件不再塞 prompt:>64KB 的文本附件落盘到 tmpdir/feishu-downloads/,prompt 只注入路径元数据,agent 用 Read 工具按需 offset/limit 分段读
  • 文本文件上限从 1MB → 30MB(与 PDF 一致),群里发 3.5MB 日志不再被拒
  • 抽取 src/feishu/file-cache.ts 共享 helper,feishu_download_message_file MCP 工具也复用
  • 周期 cleanup 接入 24h 过期清理(硬盘充裕,给 agent 留足回头读的窗口)

Test plan

  • src/__tests__/lazy-text-files.test.ts — file-cache 单测 + 历史文本附件阈值覆盖
  • 既有 lazy-history-files.test.ts / message.test.ts 全部通过
  • 全量 vitest:1427 passed,仅 memory 质量集成测试失败(需 DASHSCOPE_API_KEY,与本 PR 无关)

🤖 Generated with Claude Code

lishuceo and others added 2 commits May 17, 2026 09:49
- 抽取 src/feishu/file-cache.ts: saveMessageFileToCache + cleanupOldDownloads,复用 tmpdir/feishu-downloads
- feishu_download_message_file MCP 工具改为复用该 helper
- 当前消息文本附件: ≤64KB 内嵌 prompt;>64KB 落盘并注入路径元数据,让 agent 用 Read 工具按需 offset/limit 分段读
- 历史消息文本附件: 同样小文件内嵌、大文件落盘
- 文本文件上限从 1MB 提升到 30MB(与 PDF 一致)
- index.ts 周期 cleanup 接入 24h 过期清理

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- saveMessageFileToCache 路径/扩展名 sanitize
- cleanupOldDownloads 24h 过期清理 + 缺失目录 no-op
- 历史文本附件 ≤64KB 内嵌、>64KB 落盘注入 Read 提示
- 30MB 硬上限丢弃

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions

github-actions Bot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Claude finished @lishuceo's task in 2m 12s —— View job


⚠️ Review: Issues Found (1 minor)

  • Checked previous claude review comments (none)
  • Read CLAUDE.md and PR diff
  • Reviewed src/feishu/file-cache.ts
  • Reviewed src/feishu/event-handler.ts changes
  • Reviewed src/feishu/tools/message.ts changes
  • Reviewed src/index.ts cleanup wiring
  • Reviewed test coverage
  • Posted inline + summary

总评

整体质量不错:抽取共享 file-cache.ts 消除重复,路径 sanitize(SAFE_NAME_RE + 扩展名白名单)、cleanup 容错(ENOENT/stat/unlink 都吞掉只记 debug)、24h TTL 配合每 30 分钟周期清理这套组合都做得到位。测试覆盖了 happy path、过期文件清理、目录缺失、64KB inline 阈值与 30MB 硬上限。

Issues

🟡 引用回复(quoted parent message)路径未同步升级 — confidence 80

  • src/feishu/event-handler.ts:3507-3517parseMessage 中处理 message.parent_id 的分支仍保留 MAX_TEXT_SIZE = 1 * 1024 * 1024 且只 inline embed,没用新的 saveMessageFileToCache
  • 用户先发 3.5MB 日志、再引用回复 @bot 这种 case 仍会被 silent skip,与 PR 描述的"群里发 3.5MB 日志不再被拒"语义不一致。
  • 见上方 inline comment。

Nits(不阻塞)

  • cleanupOldDownloads is a no-op when dir does not exist 这个测试用例(src/__tests__/lazy-text-files.test.ts:77-83)实际上 mkdtemp 出来的临时目录跟 DOWNLOAD_DIR 没关系,断言只验证了"返回 number",并不能真正覆盖 ENOENT 分支——上一条用例可能已把文件写进 DOWNLOAD_DIR,所以测的根本不是空目录。可以在这个 case 里把 DOWNLOAD_DIRrmSync 掉再调用,覆盖才有意义。
  • file-cache.ts:24safeExt 正则 /^\.[a-z0-9]{1,8}$/:上限 8 字符已经覆盖 .config.tsx 等常见情况,没问题;如果以后要支持 .markdown(8 字符)边界刚好,更长的就会被裁掉,对当前需求够用。

验证项

  • ✅ ESM .js import 后缀齐全
  • cleanupOldDownloads 错误路径完整(readdir/stat/unlink 都有兜底)
  • ✅ Promise.all 内 cleaned += 1 在单线程 Node 中安全
  • ✅ 路径注入 prompt 时 messageId + fileKey 已 sanitize,无逃逸风险
  • ✅ 测试新增覆盖 PR 行为,且兼容现有测试套件
    · branch feat/lazy-text-file-loading

Comment thread src/feishu/event-handler.ts
- 移除未使用的 writeFileSync import
- 字符类中的 / 不需要转义

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions

github-actions Bot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Reviewing PR updates

  • Check previous claude review comments & status
  • Read current state of src/feishu/event-handler.ts (quoted reply path)
  • Review latest diff vs origin/main
  • Verify tests cover the fix (if any)
  • Resolve fixed inline threads & post summary

View job run · branch feat/lazy-text-file-loading

第三处文本附件入口(引用回复父消息的文件)此前仍走 1MB silent-skip
的旧逻辑。现在与直接上传和历史消息路径行为一致:30MB 上限、>64KB
落盘 + 注入 Read 提示。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions

github-actions Bot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Claude finished @lishuceo's task in 1m 9s —— View job


✅ Approved — Previous issue resolved

  • Checked previous review threads
  • Verified src/feishu/event-handler.ts quoted-reply path (L3507-3523)
  • Reviewed latest diff vs origin/main
  • Reviewed tests
  • Posted summary

Verification

Previously flagged in inline #3253872451: the quoted-parent-message text-file branch in parseMessage still used the legacy 1MB silent-skip path.

Fix in 39db424 correctly addresses this. Current event-handler.ts:3507-3523:

} else if (isTextFile(fileName)) {
  const MAX_TEXT_SIZE = 30 * 1024 * 1024;
  const INLINE_THRESHOLD = 64 * 1024;
  const buf = await feishuClient.downloadMessageFile(parent.message_id, fileKey);
  if (buf.length > MAX_TEXT_SIZE) {  warn  }
  else if (buf.length <= INLINE_THRESHOLD) {  inline embed  }
  else {  saveMessageFileToCache + lazy-load metadata  }
}

Three text-file entries (direct upload parseMessage, history downloadHistoryFiles, quoted-reply parent) now share the same 30MB cap + 64KB inline threshold + saveMessageFileToCache lazy-load pattern. ✅

The lint fix in fbfe024 is trivial and correct (unused writeFileSync, redundant / escape in character class).

Summary

  • 1 previous issue fixed (quoted-reply lazy load)
  • No new issues found
  • The earlier non-blocking nit about cleanupOldDownloads is a no-op when dir does not exist (test L77-83 doesn't actually force the dir to be missing) still stands but was already flagged as non-blocking — leaving as-is.
    · branch feat/lazy-text-file-loading

@lishuceo
lishuceo merged commit ea98975 into main May 17, 2026
6 checks passed
@lishuceo
lishuceo deleted the feat/lazy-text-file-loading branch May 17, 2026 08:05
lishuceo added a commit that referenced this pull request May 17, 2026
* feat: 新增 feishu_download_message_image MCP 工具

按需下载飞书消息图片,参照 PR #237 的 file-cache 模式,为父群图片
lazy loading 与工作区切换后图片落盘提供按需读取能力。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: 任务队列透传 currentImagePaths

为工作区切换重启时图片路径注入做准备,让本次消息内已落盘的图片
路径能从 parseMessage 经队列流到 executeClaudeTask。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: 工作区切换后注入历史图片落盘路径

setup_workspace 触发的 restart query 不重传多模态 images(避免
重复消耗 token),导致之前加载的图片在新工作区上下文中丢失。

修复:
- downloadHistoryImages 在下载历史图片时同步将原图落盘到 file-cache
- parseMessage 将当前消息图片落盘路径收集到 currentImagePaths
- 工作区切换重启时将所有落盘路径以文本提示注入 prompt,agent
  可按需通过 Read 工具加载查看

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: 引用图片同步落盘 + 注入 restart 路径

injectQuotedMessage 下载引用消息中的图片时一并 saveMessageFileToCache,
在 executeClaudeTask 调用处把落盘路径合并进 restartImagePaths,
修复 reviewer 指出的:工作区切换 restart 后 quoted parent image 同样
会丢失的问题。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test: 覆盖 quoted 图片落盘行为

为 96292bc 修复加针对性测试:
- 验证下载 quoted 图片后调用 saveMessageFileToCache 并返回 savedImagePath
- 验证落盘失败时 images 与 prompt 仍正常返回(落盘是 non-fatal)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant