Skip to content

[repo-study-facetime-bridge-raw-mic-capture] Open pitch-analysis mics with browser audio processing off and report what was actually applied #5303

Description

@atomantic

Provenance

Studied repo: kingbootoshi/facetime-bridge (MIT). Studied 2026-08-28. Clean-room: technique described, no code copied.

Rationale

facetime-bridge opens its capture device with echo cancellation, noise suppression, and automatic gain control explicitly disabled, then reads the track's applied settings back and refuses to proceed if the browser silently ignored a constraint — because processed audio is the wrong input for signal analysis. PortOS's three pitch-analysis surfaces all call getUserMedia({ audio: true }) and take whatever the browser hands back: client/src/components/songs/PitchTuner.jsx (~line 139), client/src/components/songs/SongTraining.jsx (~line 106), and client/src/hooks/useSingToVerify.js (~line 111). With defaults on, Chrome/Safari apply AGC (pumps the level the tuner's clarity gate reads), noise suppression (chews sustained vowels and soft onsets — exactly the singing input pitchDetect.js scores), and AEC (can gate the mic while the reference melody plays back during sing-to-verify). The tuner hysteresis work in #2109 papered over some of this downstream; the fix belongs at capture.

Fix

  • Add openAnalysisMic() to client/src/lib/audioRecorder.js (or a new client/src/lib/analysisMic.js — register in the client/src/lib/index.js barrel + README row): calls getUserMedia with { audio: { echoCancellation: false, noiseSuppression: false, autoGainControl: false } } (plain values, not { exact } — a browser that can't honor one must still open the mic rather than fail closed), then reads track.getSettings() and returns { stream, processing: { echoCancellation, noiseSuppression, autoGainControl } } reporting what was actually applied.
  • Replace the three getUserMedia({ audio: true }) calls with it; keep each site's existing re-entrancy/unmount guards untouched.
  • Surface the applied-settings report in the tuner and sing-to-verify UI as a small hint when any processing stayed on ("Browser audio processing is on — pitch readings may drift"), so a Safari/Firefox user knows why accuracy differs.
  • Tests: extend useSingToVerify.test.js (already mocks getUserMedia) to assert the constraint object passed and the processing report when the mocked getSettings() reports a constraint was ignored; one jsdom test for the helper's fallback when getSettings is missing.

Scope

One shared helper, three call-site swaps, one hint element in two components, two test additions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:songsSong/vocal/score systemeffort:mediumRecommended reasoning effort: mediummodel:lightRecommended implementation capability: lightplanTracked by /do:replanrepo-studyProposed from a study of a captured reference repository

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions