Add echo cancellation to loopback audio capture#27
Open
paterkleomenis wants to merge 3 commits intodesktop-app:masterfrom
Open
Add echo cancellation to loopback audio capture#27paterkleomenis wants to merge 3 commits intodesktop-app:masterfrom
paterkleomenis wants to merge 3 commits intodesktop-app:masterfrom
Conversation
Contributor
|
The PR has conflicts and commits from other PRs |
paterkleomenis
force-pushed
the
loopback-echo-cancellation
branch
from
0038528 to
d3d14ef
Compare
Contributor
|
The PR still has commits from other PRs, please fix |
paterkleomenis
force-pushed
the
loopback-echo-cancellation
branch
from
d3d14ef to
d906c1c
Compare
Contributor
|
Did you squash this one too? 😭 |
paterkleomenis
force-pushed
the
loopback-echo-cancellation
branch
2 times, most recently
from
17987a9 to
a7e9ed7
Compare
Introduces infrastructure to mix system-audio loopback into the outgoing
microphone stream during a call, on both Linux and Windows.
New types (in webrtc_create_adm.cpp, anonymous namespace):
LoopbackCollector
Thread-safe mono ring buffer (max 2 s at 48 kHz). Loopback capture
threads call pushSamples(); MixingAudioTransport calls readAndMix()
to saturating-add loopback audio into the mic frame.
DirectLoopbackCapture (Linux only, guarded by WEBRTC_LINUX)
Background std::thread that opens the PulseAudio monitor source via
alcCaptureOpenDevice (stereo preferred, mono fallback), feeds decoded
frames into LoopbackCollector, and stops cleanly on destruction.
findMonitorDevice() prefers the monitor that matches the current
default playback sink.
MixingAudioTransport
webrtc::AudioTransport decorator. When mixing is enabled it copies the
mic buffer, calls LoopbackCollector::readAndMix, then forwards the
blended frame to the inner transport. Disabled path is zero-overhead.
MixingAudioDeviceModule (details namespace)
webrtc::AudioDeviceModule wrapper. Installs MixingAudioTransport in
RegisterAudioCallback(), starts/stops DirectLoopbackCapture (Linux) or
a dedicated loopback ADM + LoopbackAdmTransport (Windows) when
setLoopbackEnabled() is toggled via MixingAudioControl.
New public API (webrtc_create_adm.h):
MixingAudioControl
Shared handle that survives ADM recreation. setLoopbackEnabled() /
loopbackEnabled() let callers toggle mixing at any time; the control
re-applies pending state whenever a new MixingAudioDeviceModule
attaches itself.
MixingAudioDeviceModuleCreator(innerCreator, control)
Returns a creator lambda that wraps any ADM (e.g. the OpenAL ADM)
inside a MixingAudioDeviceModule.
…odule
When screen audio is being shared in a call, muting the microphone should
not kill the system-audio stream. The two new runtime controls address this:
setMicrophoneMuted(bool)
Zeroes out the mic samples inside MixingAudioTransport::RecordedDataIsAvailable
before they reach WebRTC, while keeping the audio channel open so that
loopback (system audio) continues to flow through unchanged.
setPlaybackVolume(float)
Scales every decoded playback sample by the given factor (0.0-1.0) in
MixingAudioTransport::NeedMorePlayData, providing per-call software volume
control independent of the system mixer.
Both controls are exposed on the shared MixingAudioControl handle with their
respective atomic fields, so changes take effect immediately without any
lock held in the hot path.
State is persisted in MixingAudioControl and re-applied whenever a new
MixingAudioDeviceModule attaches (e.g. after an ADM restart) or whenever
RegisterAudioCallback() installs a fresh MixingAudioTransport.
When sharing system audio in stereo, playback leaks back through the
capture path and remote participants hear an echo. This adds a
WebRTC AudioProcessing (AEC) stage to the OpenAL capture loop inside
MixingAudioDeviceModule:
- Create an AudioProcessing instance with echo_canceller enabled
(mobile_mode) when the capture device opens in stereo.
- Feed far-end (rendered) audio into ProcessReverseAudioFrame via
LoopbackCaptureTakeFarEndLinux, which returns the estimated delay.
- Run the captured microphone + loopback mix through
ProcessAudioFrame before pushing samples to the collector.
- Track capture activity with SetLoopbackCaptureActiveLinux so the
far-end buffer is only maintained while capture is running.
Falls back to the existing non-AEC path when stereo capture is not
available or no far-end reference frame is ready.
paterkleomenis
force-pushed
the
loopback-echo-cancellation
branch
from
a7e9ed7 to
471f0c8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When sharing system audio in stereo, playback leaks back through the capture path and remote participants hear an echo. This adds a WebRTC AudioProcessing (AEC) stage to the OpenAL capture loop inside MixingAudioDeviceModule.
Changes
AudioProcessinginstance withecho_cancellerenabled (mobile mode) when the capture device opens in stereo.ProcessReverseAudioFrameviaLoopbackCaptureTakeFarEndLinux, which returns the estimated delay.ProcessAudioFramebefore pushing samples to the collector.SetLoopbackCaptureActiveLinuxso the far-end buffer is only maintained while capture is running.Falls back to the existing non-AEC path when stereo capture is not available or no far-end reference frame is ready.