voice: introduce Voice Mode with a first-run card above the chat input - #327853
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds first-run Voice Mode onboarding above chat inputs, including voice previews, accessibility behavior, persistence, and support for both workbench and Agents window hosts.
Changes:
- Adds the onboarding card, waveform, voice samples, and styling.
- Registers connection-state triggering and one-time storage.
- Adds host integrations, fixtures, and unit tests.
Show a summary per file
| File | Description |
|---|---|
voiceModeOnboarding.fixture.ts |
Adds responsive visual fixtures. |
chatInputPart.ts |
Registers the workbench chat host. |
voiceModeOnboarding.test.ts |
Tests display, dismissal, and host selection. |
agentsVoiceColors.ts |
Registers waveform colors. |
agentsVoice.ts |
Adds the first-run storage key. |
voiceModeOnboarding.ts |
Implements the card and service. |
voiceModeOnboarding.css |
Styles interactions and animations. |
agentsVoice.contribution.ts |
Triggers onboarding from connection state. |
newChatInput.ts |
Registers the Agents window host. |
vscode-known-variables.json |
Allowlists referenced CSS variables. |
Review details
Comments suppressed due to low confidence (4)
src/vs/workbench/contrib/agentsVoice/browser/voiceModeOnboarding.ts:524
- Escape bypasses
finish(), unlike the close button. If the banner paused an already-listening session, Escape removes the card without handing hands-free mode back or announcing the resulting state.
this.options.onDismiss();
src/vs/workbench/contrib/agentsVoice/browser/voiceModeOnboarding.ts:578
- These custom radio controls only listen for
click. In the touch-capable Agents window, iOS relies on VS Code's gesture system, so taps may not activate them. RegisterGesture.addTarget, handleTouchEventType.Tapalongside click, and addtouch-action: manipulation, or render them with the shared button widget.
this._register(dom.addDisposableListener(option, dom.EventType.CLICK, () => this.selectVoice(voice)));
src/vs/workbench/contrib/agentsVoice/browser/voiceModeOnboarding.ts:620
- The custom close control also only handles
click, so it can be non-functional on iOS in the Agents window. Give it the sameGesture.addTargetplusTouchEventType.Taphandling andtouch-action: manipulation, or use the shared button widget.
this._register(dom.addDisposableListener(close, dom.EventType.CLICK, () => this.finish()));
src/vs/workbench/contrib/agentsVoice/browser/voiceModeOnboarding.ts:676
- Hands-free mode is only resumed after choosing a voice. Dismissing the card without selecting one is explicitly supported, but once onboarding correctly suppresses connect-time listening, this branch leaves the connected session idle instead of handing it back as described. Coordinate this condition with the connect-time suppression fix.
if (this.selectedVoice !== undefined && this.configurationService.getValue<boolean>('agents.voice.handsFree') === true) {
- Files reviewed: 10/10 changed files
- Comments generated: 11
- Review effort level: Medium
Voice Mode can speak back, and it does so in one of four model voices - but nothing in the product says so. The setting exists, the voices are only names in a dropdown, and nobody hears one before committing to it. Anyone who does not want to be spoken to at all is never told that dictation is the alternative. Show a small card above the chat input the first time a Voice Mode session starts. It says what Voice Mode is, that it costs nothing, and offers the four voices to audition. Clicking a voice plays a short sample recorded with that exact model voice and adopts it in the same gesture, so what you hear is what you get and there is nothing to confirm afterwards. The leading icon carries that story: play before the click, animating bars while it speaks, a check once it is yours. The card is deliberately no taller than the input it sits above, and survives down to a 240px panel. Its waveform is drawn with Voice Mode's own instrument - the same thin bars and rounded caps as the toolbar equalizer, just many more of them - so the introduction and the feature it introduces read as one thing. It appears exactly once, gated on an application-scoped storage key that is only written once a card is actually shown, so a session started with no chat panel open does not silently burn the one appearance. The trigger watches the voice session's connection state rather than any single entry point, because Voice Mode can be started from the input-mode pill, a command, a keybinding or the Agents window. While the card is up, Voice Mode is connected but not listening: nobody should be recorded while still reading. Dismissing hands the session back - hands-free starts listening, push-to-talk waits for the mic. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two problems with the card's sentence. It ended on dictation - "or dictate with the mic" - which answers a question nobody asked while standing in a card about picking a voice. It made the paragraph longer and the ask less clear. And nothing told anyone the choice was reversible. Clicking a voice writes `agents.voice.voice` immediately, so a first-run card that offers no way back reads as more of a commitment than it is. Replace the dictation clause with the reassurance, inline in the same sentence rather than as a line of its own: a note that says "you can undo this" belongs in the same breath as the ask, not stacked underneath it as though it were a second instruction. The link is marked in the localized string with `[[...]]` and rendered with `renderFormattedText`, so translators can move it inside the sentence instead of receiving a fixed phrase concatenated onto the end. That renderer only wires up a click handler, so the anchor also gets a tab stop, a button role and Enter/Space handling - it is a control, and should behave like one for people not using a mouse. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The card promised Voice Mode was connected but not listening while the
user read it, and did not deliver. It goes up on `isConnecting`, before
a session exists, so its `stopListening()` was a no-op - that method
returns early while `!isConnected`. Session init then called
`_enterAutoListen('connect')` and opened the microphone with the card
still on screen, which is the one thing it said it would not do.
Add `setAutoListenHeld()`: a hold that can be taken before the session
connects and survives the handshake. It is a separate flag from
`_autoListenSuppressed` on purpose - pttDown, playback prep and
disconnect all clear that one as part of normal turn-taking, so a hold
riding on it would be released by the first thing that happened. The
card takes the hold for its whole lifetime and releasing it is what
resumes hands-free, replacing the synthetic press `finish()` used to
fake.
Also from review:
- The one-time key was written before the card was built, so a card that
threw on the way up would consume an appearance nobody saw. It is now
written last, once the card is attached.
- Dismissing from the keyboard dropped focus on the document body. Hosts
now pass a focus callback - they know where their caret belongs, and a
container element is not a control - and it is only used when focus is
still inside the card, so clicking away is not undone.
- `phase` was left out of the signature morph. It is a static per-
component offset, so Kevin and Daniel never actually reached the
signatures declared for them.
- The resting trace claimed to be a blend of all four voices and was
quietly Victoria's. It is now the mean it always said it was.
- Reduced motion stopped the keyframes but not the transitions, so the
icon slot still slid open and its glyphs still crossfaded.
- Both 12px icons now use the compact glyphs drawn for that size.
- Chat accessibility help says nothing about a new interactive surface,
so it now describes reaching, using and dismissing the card.
- The spacing note claimed 6px chip gaps on an "8px scale"; both were
wrong. Named the tokens instead.
- Dropped an allowlisted custom property that nothing declares or uses.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eli-w-king
force-pushed
the
eli/agents/voice-mode-first-run
branch
from
July 28, 2026 17:49
fa0aa49 to
bf370d6
Compare
Three things about the introduction's waveform and its sentence. **It was not moving.** Measuring the resting trace: a bar changed height by about 1px per second, which is animating in the same sense that a clock's hour hand is. The signatures are written as a *relative* set - component 1 drifts against component 0, and so on - which reads well, but taken literally the dominant component turns once every 17 seconds. Anchor the tempo to the 2.6s cycle of `chat-voice-input-mode-wave`, the keyframe the toolbar waveform already idles on, and derive it from the resting signature so editing a signature cannot quietly put the trace back to sleep. Measured in the running build: every sampled frame now differs, where before the strip was effectively still. Worth noting this was latent, not new: the previous idle was ~1px/s and the blended resting signature introduced last commit made it ~0.74px/s. That did not cause the problem so much as cross the threshold where it became visible. **It was the loudest thing on the card.** `voiceInputMode.css` states the rule for this instrument directly - *bars are strokes, not shapes* - and keeps them at the same visual weight as the codicon glyphs beside them so the waveform never reads as bolder than the mic. Ours was drawn at twice the stroke width and filled with a blue-violet gradient, in a card where nothing else carries a colour of its own. Match the toolbar: 1px bars, and a single colour taken from the canvas's own computed `color`, which is the same `currentColor` arrangement the toolbar uses. CSS now owns the tier - `descriptionForeground`, since the trace is ambient - and theme overrides work for free. The colour is cached and re-read on theme change rather than per frame, so the animation loop does not force a style recalculation every tick. That leaves `agentsVoice.waveFrom` and `agentsVoice.waveTo` with no users; both were added for the gradient, so both go. **The sentence was long, and pointed at only half of what is configurable.** Shorten it, and give the second half a link to the `voice.md` customization: settings decide whether the agent speaks and in which voice, while `voice.md` shapes what it says back. The link reads "how it responds" rather than naming the file, because that is what a first-time reader is actually looking for. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 678d0200-f25e-426b-88ca-a13e124c87be
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 678d0200-f25e-426b-88ca-a13e124c87be
Contributor
Screenshot ChangesBase: Added (6)
|
Clarify that Voice Mode pauses listening while its introduction is open and add a context-menu action to show the card again. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 678d0200-f25e-426b-88ca-a13e124c87be
Collaborator
|
Pushed
Validated with client type-check/compile, 26 targeted onboarding tests, lint/hygiene, and all 894 component fixtures rendering without errors. |
Put the default voice first, clarify how to continue, and focus the introduction for screen reader users. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 678d0200-f25e-426b-88ca-a13e124c87be
Collaborator
|
Pushed
|
meganrogge
marked this pull request as ready for review
July 28, 2026 21:55
meganrogge
enabled auto-merge (squash)
July 28, 2026 21:55
meganrogge
approved these changes
Jul 28, 2026
roblourens
approved these changes
Jul 28, 2026
This was referenced Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
Voice Mode can speak back, in one of four model voices — but nothing in the product says so. The setting exists, the voices are only names in a dropdown, and nobody hears one before committing to it. Nor is there any sign that the choice is reversible, or that what the agent says back can be customized at all.
What this adds
A small card above the chat input, the first time a Voice Mode session starts.
It says what Voice Mode is, that it costs nothing, and offers the four voices to audition. Clicking a voice plays it and adopts it in the same gesture — the sample is pre-recorded with that exact model voice, so what you hear is what you get, and there is nothing to confirm afterwards. The leading icon carries that whole story:
The icon slot is collapsed to
width: 0until hover or selection. Four named buttons plus permanent icons need 261px; the default panel row is 206px, so the slot slides open instead of forcing a wrap.Details worth calling out
It uses Voice Mode's own instrument, to the pixel.
voiceInputMode.cssstates the rule for this shape directly — bars are strokes, not shapes, kept at the same visual weight as the codicon glyphs beside them — so the waveform matches it exactly: 1px bars, 2px gaps, rounded caps, and a single colour taken from the canvas's own computedcolor, the samecurrentColorarrangement the toolbar uses. CSS owns the tier, so theme overrides work for free. Bars are drawn on canvas rather than as DOM spans — ~70 spans would mean ~70 style writes per frame — and the colour is cached and re-read on theme change rather than per frame, so the loop never forces a style recalculation.Its tempo is borrowed too. The signatures are written as a relative set, which reads well but taken literally turns the dominant component once every 17 seconds — about 1px of movement per second, which is animating in the same sense that a clock's hour hand is. The tempo is anchored to the 2.6s cycle of
chat-voice-input-mode-wave, the keyframe the toolbar already idles on, and derived from the resting signature so editing a signature cannot quietly put the trace back to sleep.Two ways to change your mind. The sentence links to both: settings decide whether the agent speaks and in which voice, while
voice.mdshapes what it says back. The second link reads "how it responds" rather than naming the file, because that is what a first-time reader is looking for.It appears exactly once, gated on an application-scoped storage key. The key is written only once a card is actually shown, so a session started with no chat panel open doesn't silently burn the one appearance.
The trigger watches connection state, not any single entry point. Voice Mode can be started from the input-mode pill, a command, a keybinding or the Agents window; all of them land on
isConnecting/isConnected.Nobody is recorded while reading. While the card is up Voice Mode is connected but not listening. Dismissing hands the session back — hands-free starts listening, push-to-talk waits for the mic.
Hosts. Both the workbench chat input and the Agents window register a container; the most recently focused visible one wins.
Accessibility
role="region"with a label; the voices are aradiogroupofradios witharia-checkedand a single tab stop, arrows moving between them. Selection is announced withstatus(). Escape dismisses. The canvas isaria-hidden, and the animation honours reduced motion.Review follow-ups
All eleven review comments were verified against the code and addressed in
bf370d6. The substantive one:The card said Voice Mode was not listening, and it was. It goes up on
isConnecting, before a session exists, so itsstopListening()was a no-op (that method early-returns while!isConnected).session_initthen called_enterAutoListen('connect')and opened the microphone with the card still on screen.Fixed with
setAutoListenHeld()— a hold that can be taken before the session connects and survives the handshake. It is a separate flag from_autoListenSuppressed, whichpttDown,_prepareForPlaybackand disconnect all clear as part of normal turn-taking; a hold riding on that would be released by the first thing that happened. Releasing the hold now also replaces the synthetic key-press pairfinish()used to fake.Also addressed: the one-time key is written last (a card that threw would otherwise consume an unseen appearance); dismissal restores focus to the chat input via an explicit host callback, but only when focus is still inside the card;
phaseis eased with the rest of the signature, so Kevin and Daniel actually reach their declared traces; the resting trace is now the blend it always claimed to be rather than Victoria's; reduced motion disables the slot slide and glyph crossfade, not just the keyframes; both 12px icons use the compact glyphs; chat accessibility help describes the card; and an unused stylelint allowlist entry is gone.Validation
npm run typecheck-client,npm run valid-layers-check, eslint, stylelint, hygiene — all cleanVoiceSessionControllertests covering the new holdNote
npm run compilecurrently fails onextensions/markdown-language-features/markdown-editor-src/editor.ts(Property 'revealComment' does not exist). That is pre-existing onmainfrom aac8a5a and untouched by this branch, which changes nothing underextensions/.Follow-up: shared onboarding, clarity, and engagement
This branch now consumes the reusable
ChatInputOnboarding/ChatInputOnboardingCardinfrastructure extracted with Dictation, rather than maintaining a second host and lifecycle implementation.The card now puts the default Maya voice first and adds a separate “Close this when you’re ready to speak” line beneath the introduction. Its corner action is a checkmark rather than an X. Closing still hands control back to hands-free or push-to-talk as before, and screen-reader mode moves focus to the completed card so its introduction is announced.
Both Voice Mode and Dictation onboarding now emit action telemetry for card display and interactions (voice or microphone selection, settings/instructions links, start/close, and cancel/Escape), including whether the card was shown automatically or manually. The Voice Mode microphone context menu also includes Show Introduction, backed by a command that can reopen the card without resetting first-run state.
Fixes #327203