chat: add a first-run dictation introduction with microphone picker - #327869
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a first-run chat dictation card for microphone selection and live input-level preview.
Changes:
- Introduces the onboarding card, preview pipeline, picker, and styling.
- Integrates onboarding with standard and Agents-window chat inputs.
- Adds reopening support and lifecycle tests.
Show a summary per file
| File | Description |
|---|---|
dictationOnboarding.test.ts |
Tests device normalization and card lifecycle. |
chatInputPart.ts |
Registers standard chat input hosts. |
dictationOnboarding.css |
Styles the onboarding card and spectrum. |
dictationOnboarding.ts |
Implements preview, picker, banner, and service. |
chatSpeechToTextActions.ts |
Routes dictation through onboarding and adds reopening. |
newChatInput.ts |
Integrates onboarding into the Agents composer. |
Review details
- Files reviewed: 6/6 changed files
- Comments generated: 10
- Review effort level: Medium
meganrogge
reviewed
Jul 28, 2026
| @@ -0,0 +1,861 @@ | |||
| /*--------------------------------------------------------------------------------------------- | |||
Collaborator
There was a problem hiding this comment.
I suggest making a generic onboarding part that is then extended by dictation and voice mode, passing in the options to reduce code #327853
Collaborator
There was a problem hiding this comment.
So I would think:
- address copilot comments on both
- in dictation pr, make the part generic, use it for dictation
- merge dictation pr
- adopt generic part for voice mode pr
Collaborator
There was a problem hiding this comment.
We talked and I will take care of this
The "Chat: Dictate: Select Microphone" command is not discoverable, so there is no sign that a machine has more than one microphone and no way to tell whether the one in use is picking anything up. A disabled bluetooth headset left as the system default looks exactly like dictation being broken (microsoft/vscode-internalbacklog#8602). Add a card that takes over the first dictation started from a chat input. It names the device, shows a live spectrum driven by that device, and starts dictating when the user confirms. Nothing is recorded while it is up, so "is my microphone connected" can be answered before any words are transcribed rather than during. - The picker writes the microphone shared with Voice Mode (`AgentsVoiceStorageKeys.MicrophoneDevice`), so the choice made here is the one dictation records from, and it agrees with the quick pick. - The spectrum runs its own preview stream, independent of the dictation pipeline. It is released before the hand-off: two concurrent captures of one device make the audio service hand one of them silence. - Bands are mapped logarithmically across a calibrated -70..-25dB window. `getByteFrequencyData` normalizes against the Web Audio defaults, which put a silent room at a third of the range - a resting meter that never settles undermines the one question the card exists to answer. - Chat inputs only. Editor and terminal dictation have nowhere to dock the card and are untouched. - `Chat: Dictate: Show Introduction` reopens it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three things the card was getting wrong. It drew a *chart*. Each stroke was an independent FFT band, so neighbours jumped independently and the row read as a spectrum analyser rather than as a voice. Replace that with the instrument Voice Mode already uses: a travelling sine composite under a centre taper, with the microphone driving amplitude rather than per-band height. Dropping the spectrum takes the logarithmic band mapping, the noise floor, the response curve and the dB window with it - all of which only ever existed to make `getByteFrequencyData` behave. It was painted in an accent colour that ignored the theme. The strokes now inherit `currentColor` at 1px with 2px gaps, matching both the toolbar pill and Voice Mode's own card, and the row's opacity rides the same level that drives its height - quiet at rest, brighter as you speak, dimmer still when there is no microphone to read. The bar count follows the measured width so that 1px/2px rhythm survives a resize. It was a dead end. Neither `dictation.enabled` nor the `dictation.md` customization added in #327418 was reachable from the one surface a first-time user is actually looking at. The description now carries two links, in the same `renderFormattedText` shape the Voice Mode card uses so translators get a whole sentence rather than concatenated fragments. Both anchors are promoted to real controls; the callback distinguishes them by index, so the localization comment pins their order. Voice Mode is no longer mentioned: it introduces itself. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eli-w-king
force-pushed
the
eli/dictation-onboarding-banner
branch
from
July 28, 2026 18:53
a12eddb to
0d87616
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 678d0200-f25e-426b-88ca-a13e124c87be
meganrogge
previously approved these changes
Jul 28, 2026
meganrogge
marked this pull request as ready for review
July 28, 2026 19:42
meganrogge
enabled auto-merge (squash)
July 28, 2026 19:45
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 678d0200-f25e-426b-88ca-a13e124c87be
Contributor
|
Base:
|
hediet
previously approved these changes
Jul 28, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 678d0200-f25e-426b-88ca-a13e124c87be
hediet
approved these changes
Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes microsoft/vscode-internalbacklog#8602
Problem
Chat: Dictate: Select Microphoneis not discoverable. There's no sign that a machine has more than one microphone, and no way to tell whether the one in use is picking anything up. A disabled bluetooth headset left as the system default looks exactly like dictation being broken — which is how the issue was filed.What this does
Adds a card that takes over the first dictation started from a chat input. It names the device, shows a live spectrum driven by that device, and starts dictating when the user confirms.
This also adds reusable
ChatInputOnboardingandChatInputOnboardingCardparts for chat-input onboarding. They centralize host selection, first-run persistence, card ownership and dismissal, focus restoration, and accessible card actions so Voice Mode can reuse the same infrastructure.Nothing is recorded while the card is up, so "is my microphone connected" can be answered before any words are transcribed rather than during.
Chat: Dictate: Show IntroductionNotes for reviewers
Microphone selection is shared. The picker writes
AgentsVoiceStorageKeys.MicrophoneDevice— the same key Voice Mode and the existing quick pick use — so the choice made here is the one dictation actually records from, and the three never disagree. Device normalization (dropping the virtualdefault/communicationsentries, de-duplicating bydeviceId) matchesSelectSpeechToTextMicrophoneActionand is unit-tested.The preview stream is independent of the dictation pipeline, which is the point — a level can't be shown from a pipeline that hasn't started. It is explicitly released before the hand-off, and the hand-off is deferred by a beat: two concurrent captures of one device make the audio service hand one of them silence. This was a real bug during development, not a theoretical one.
Two Web Audio details that are easy to get wrong, both commented at the call site:
AudioContextsuspended without sticky user activation. A suspended graph reports silence — a dead meter indistinguishable from a dead microphone. The preview resumes it.getByteFrequencyDatanormalizes againstminDecibels/maxDecibels, and the Web Audio defaults (−100…−30 dB) put a silent room at a third of the range. Anchored to −70…−25 dB so silence reads as silence.Bands are mapped logarithmically. Linearly, a voice piles into the first few bars — its fundamental and low harmonics are all in the bottom bins — and leaves the rest of the row dead no matter how loudly anyone talks.
Scope is chat inputs only. Editor (
Cmd+I) and terminal dictation have nowhere to dock the card and are untouched —onboardingServiceis optional onIDictationShortcutContext, and those paths simply don't pass it.Gating is inherited from the existing dictation preconditions, which already include
ChatContextKeys.enabled, so nothing appears when AI features are disabled.Design
Follows the Voice Mode onboarding card's anatomy (host-registration service,
MutableDisposable-owned card, squared bottom edge tucked behind the rounded chat input) so the two read as one family.The spectrum is the one thing that leads; the copy is two lines; there is no standing instruction text — the moving waveform is the instruction, and the hint row only appears when the microphone can't be read. Sizes, radii and type all come from design tokens. Reduced motion is honoured by slowing the repaint to a readable step rather than stopping it, since the meter is feedback rather than decoration.
Testing
npm run typecheck-client,npm run valid-layers-check,npm run compile, hygiene — cleandictationOnboarding.test.ts— device normalization, host lifecycle, deferral, Escape, re-showNot verified on this machine: the exact liveliness of the bars at typical speaking volume. Audio output here is routed to AirPods, so synthetic playback never reaches the built-in mic. The pipeline is verified end-to-end (ambient levels, live transcription); if the bars read too flat in practice, the
MIN_DECIBELS/MAX_DECIBELSwindow is the single knob.