Repository navigation
PlatformAudio.Dispose leaves iOS audio session deactivated #331
Description
Activity
Just an update here. I ended up caching the iOS audio session state prior to PlatformAudio being created. Following a full Disposal of all livekit related items, I then restored those audio session values, assuming the user is not routing things through some other chosen external output (headphones, airplay, bt speaker, etc)
// Captures the current category/mode/options before LiveKit's own audio pipeline // reconfigures the session for a call. Call this as the very first thing when starting // a LiveKit connection, so _LiveKit_RestoreCachedAudioSessionState() can put things back // exactly as they were once the call ends — whatever that was (music already playing in // some other category, a prior recording session, plain defaults) — rather than guessing // a single "safe" target state for every experience that might run next. void _LiveKit_CacheAudioSessionState() { AVAudioSession *session = [AVAudioSession sharedInstance]; cachedCategory = session.category; cachedMode = session.mode; cachedOptions = session.categoryOptions; hasCachedAudioSessionState = YES; NSLog(@"LiveKitAudioSessionUtils: Cached audio session state (category=%@, mode=%@, options=%lu)", cachedCategory, cachedMode, (unsigned long)cachedOptions); } // Restores the audio session to whatever was captured by _LiveKit_CacheAudioSessionState() // and reactivates it. No-ops (with a log) if nothing was cached. Must be called after the // LiveKit room has fully torn down — the room's own teardown can touch the session and // undo a restore called too early. void _LiveKit_RestoreCachedAudioSessionState() { if (!hasCachedAudioSessionState) { NSLog(@"LiveKitAudioSessionUtils: RestoreCachedAudioSessionState called with no cached state — skipping"); return; } AVAudioSession *session = [AVAudioSession sharedInstance]; NSError *error = nil; // Force the physical route back to the speaker while the session is still in // whatever PlayAndRecord-derived state the WebRTC ADM left it in, before switching // away below. overrideOutputAudioPort: only takes effect under // AVAudioSessionCategoryPlayAndRecord — it's a harmless no-op error otherwise (e.g. // if the cached category isn't PlayAndRecord). Doing this BEFORE the category switch // ensures CoreAudio actually renegotiates the output port away from the earpiece left // by the prior VoIP call, since most non-PlayAndRecord categories have no equivalent // forcing option. // // Skip entirely if the user has headphones/Bluetooth/AirPlay connected — this // override is only meant to break a route stuck on the built-in earpiece, and // should never yank audio away from an external device the user chose. if (isRoutedToExternalOutput()) { NSLog(@"LiveKitAudioSessionUtils: External output active — skipping speaker override"); } else { [session overrideOutputAudioPort:AVAudioSessionPortOverrideSpeaker error:&error]; if (error) { NSLog(@"LiveKitAudioSessionUtils: Failed to override output port to speaker: %@", error.localizedDescription); error = nil; } } [session setCategory:cachedCategory mode:cachedMode options:cachedOptions error:&error]; if (error) { NSLog(@"LiveKitAudioSessionUtils: Failed to restore cached category/mode/options: %@", error.localizedDescription); error = nil; } [session setActive:YES error:&error]; if (error) { NSLog(@"LiveKitAudioSessionUtils: Failed to reactivate audio session after restore: %@", error.localizedDescription); return; } NSLog(@"LiveKitAudioSessionUtils: Restored cached audio session state (category=%@, mode=%@, options=%lu)", cachedCategory, cachedMode, (unsigned long)cachedOptions); hasCachedAudioSessionState = NO; }Hey, that sounds good, but I have not been able to get it working.
My caching seems to work, these values are used to restore after I tore down the LiveKit room and PlatformAudio:
LiveKitAudioSessionUtils: Restored cached audio session state (category=AVAudioSessionCategoryAmbient, mode=AVAudioSessionModeDefault, options=1)
To make sure the teardown is finished I even added a 5s delay, but the sound does not come back.One thing I noticed with the Meet sample, that even disconnecting from the room is enough to break the Unity audio output, even if you don't dispose PlatformAudio. I still tested your code with both cases, with and without disposing platform audio on top.
What Unity version are you using? Today I only tested with 2022.3.62f1.
I also assume, you integrate this into your own project? I always use the Meet sample as our reference test project.If possible, can you share more details on your test code and setup?
One more thing I noticed btw is that with Unity 6 builds, backgrounding and foregrounding the app breaks our PlatformAudio. I have a ticket for that as well and it is related to Unity also trying to reconfigure the session.
My work is all on 2022.3.62f2
So my Restoration code goes like this:
- Restore cached audio session info (ios native plugin call)
- AudioSettings.Reset(AudioSettings.GetConfiguration());
- Reapply native mute switch override (ios native plugin call) - this sets the current ios audio session to Playback/PlayAndRecord depending on the cached category, and then makes sure that it is active via
[session setActive:YES error:&error];) Setting the category to Ambient resulted in audio only playing if the hardware mute switch was off, which our app is set to ignore.
The key I found here, is that Unity's AudioSettings.Reset can stop the audio session, so the mute switch override call is making sure the restored session becomes active as well. You probably don't need to override the mute switch, but you do need to make sure the session is setActive.
re: Unity 6. I tried one build from 6.3.14, and none of the livekit incoming audio was working, so I decided to come back to it later. Unity made a number of audio session management changes in Unity 6, so I suspect many of my 2022.3 work-arounds would need to be changed.
I still could not repro your fix, but have another fix for some of the issues:
https://github.com/livekit/client-sdk-unity/pull/346/changesI want to make sure that we can fully support Platform Audio on iOS long term and therefore we are also looking into the native implementation again and try to align more with what we learned in our Swift SDK.
Are you unblocked for now or is there any immediate help needed? I will keep this issue open until we found a cleaner solution we can merge into main.
We've figured out a workaround for now on iOS to keep things moving forward.
I have a larger PR prepared to fix a lot of these lifecycle issues with Platform Audio. If you want to give it a try please let me know if this works for you as well:
#378Short update here:
There are a lot of difficulties getting the Platform Audio setup right for all platforms. The above mentioned PR is a combination of a lot of fixes, but still does not solve all cases.
We have been able to introduce echo cancellation into the Unity audio path though and I will investigate more if the Unity audio path could in the end be the better overall offering for developers.
SDK: io.livekit.livekit-sdk (Unity) v1.3.8, iOS
When PlatformAudio is created, LiveKit_ConfigureAudioSessionForVoIP() correctly configures the AVAudioSession to PlayAndRecord category with VoiceChat mode and calls setActive:YES. However, when PlatformAudio.Dispose() is called, the underlying Rust WebRTC ADM tears down and internally calls setActive:NO — but the session is left in PlayAndRecord + VoiceChat configuration. No restore or reactivation occurs.
Any audio playback attempted after disposal (e.g. switching from a voice conversation to a listening to music in the same app session) is silent because Unity's audio engine cannot play through a deactivated VoIP-mode session.
To reproduce
Expected behavior
PlatformAudio.Dispose() restores the AVAudioSession to a neutral state (e.g. SoloAmbient with setActive:YES) so the host application's audio pipeline can resume normally.
Note: LiveKit_RestoreDefaultAudioSession() already exists in LiveKitAudioSession.mm but is never called by the SDK during teardown, and it does not call setActive:YES — so even if called manually it would leave the session deactivated.