Skip to content

feat(desktop): 生成过程中播放 Token 驱动的背景音乐#4175

Closed
ttmouse wants to merge 6 commits into
esengine:main-v2from
ttmouse:pr/generative-music
Closed

feat(desktop): 生成过程中播放 Token 驱动的背景音乐#4175
ttmouse wants to merge 6 commits into
esengine:main-v2from
ttmouse:pr/generative-music

Conversation

@ttmouse

@ttmouse ttmouse commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

背景

AI 生成过程通常是一个静默等待的状态——用户在屏幕前看着文本逐字出现,缺少听觉上的反馈。这让等待显得漫长而枯燥,也削弱了「正在工作」的沉浸感。

设计思路

借鉴生成式音乐(Generative Music)的理念,将 AI 的 token 输出流实时映射为音符:

  • Token 驱动:每次 token(文本/推理/工具调用)到达时触发一个音符
  • 速率自然映射:模型生成快则音符密集,生成慢则音符稀疏,直观反映生成节奏
  • 五声音阶:C 大调五声音阶(C D F G A),无半音,确保任意顺序播放都悦耳
  • 双重节流:每 4 个 token 采样一次 + 100ms 最小间隔,适应 DeepSeek 的高速输出
  • 信号链:振荡器 → ADSR 包络 → 低通滤波器 → 混响 → 输出

新增配置

设置面板 → 通用 → 生成式背景音乐

预设 波形 风格
关闭 不播放
经典 三角波 温暖圆润
空灵 ★(默认) 正弦波 柔和飘渺
数字 方波 清脆锐利
复古 锯齿波 厚实怀旧

改动文件

6 个文件,+432 行:

  • desktop/frontend/src/lib/generative-music.ts — 音乐引擎(信号链、ADSR、混响)
  • desktop/frontend/src/App.tsx — 接入 state.running + token 事件
  • desktop/frontend/src/components/SettingsPanel.tsx — 设置面板 UI
  • desktop/frontend/src/locales/{en,zh}.ts — 国际化
  • desktop/frontend/src/styles.css — 设置行样式

Plays ambient musical notes in real-time as AI tokens arrive.
Token rate naturally reflects generation speed — faster output = denser notes.
Includes 4 sound presets (classic/ethereal/digital/retro) in settings panel.
Default: ethereal (sine wave, gentle reverb).
@github-actions github-actions Bot added v2 Go rewrite (1.x) — main-v2 branch, active development desktop Wails desktop app (desktop/**) labels Jun 12, 2026
@ttmouse

ttmouse commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author
image

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6cfb0b6a17

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread desktop/frontend/src/App.tsx
@taibai233

Copy link
Copy Markdown
Contributor

你这东西要上了嘛哈哈哈哈哈😂

@SivanCola

Copy link
Copy Markdown
Collaborator

评审了一轮,整体实现比较集中,build 和现有前端测试都能过。不过有一个交互边界建议修一下:

SettingsPanel 里切换生成音乐只写了 localStorage,而 App 里启动/停止音乐的 effect 只依赖 state.running。所以如果用户在一次较长生成过程中把音乐切到 Off,当前 generativeMusic 仍然是 running,后续 token 事件会继续触发音符;切换 preset 也要等下一轮生成才会生效。

相关位置:

  • desktop/frontend/src/components/SettingsPanel.tsxsetGenerativePreset(next) 后没有同步当前 engine 状态
  • desktop/frontend/src/App.tsx 的 effect 只在 state.running 改变时重新判断 isGenerativeMusicEnabled()

建议在设置变更时直接同步 engine,比如 next === "off" 时调用 generativeMusic.stop(),非 off 时如果正在生成则更新/重建 preset;或者把 preset 做成可订阅状态,让 App 的 effect 能响应设置变更。

另外确认一下产品预期:目前无历史设置时默认是 ethereal,等于所有升级用户默认开启生成音乐。如果这不是有意设计,建议默认 off,避免升级后突然出声。

我本地验证过:

  • wails generate module
  • npm run build
  • npm test

@SivanCola

Copy link
Copy Markdown
Collaborator

补充一下默认值要求:生成音乐需要默认关闭,不能在升级或首次启动后自动播放。请把无历史设置时的默认值改成 off,只有用户在设置里主动选择某个 preset 后才启用播放。

ttmouse and others added 5 commits June 12, 2026 18:22
- 默认值从 ethereal 改为 off,避免升级后突然出声
- 设置中切到关闭时立即 stop() 引擎
- 切到非关闭预设时即时 start()/setPreset(),不用等下一轮生成
参考 notification-sound PR (esengine#4176) 的 SoundSelect 样式,
将原生 <select> 替换为 AnchoredPopover + trigger 按钮 +
勾选图标的自定义下拉组件。
…ve-music-lifecycle

# Conflicts:
#	desktop/frontend/src/App.tsx
#	desktop/frontend/src/components/SettingsPanel.tsx
#	desktop/frontend/src/styles.css
@SivanCola SivanCola enabled auto-merge June 12, 2026 11:16
@SivanCola

Copy link
Copy Markdown
Collaborator

approve

@esengine

Copy link
Copy Markdown
Owner

I rebased this onto current main-v2 and landed it as #4192 — the branch here had merged main-v2 back in and #4188 had since moved SettingsPanel.tsx, so I squashed it forward and did a small house-style comment pass on the way (trimmed the file-header block and the // ── … ── section banners; the code is byte-identical). Credit kept to you for the feature and to @SivanCola for the review fixes. I'll merge #4192 once CI is green and close this in favour of it. Thanks — the engine is cleanly built, and default-off is exactly right.

auto-merge was automatically disabled June 12, 2026 13:27

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

desktop Wails desktop app (desktop/**) v2 Go rewrite (1.x) — main-v2 branch, active development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants