Gapless Looping Issue on iOS 17+

[DeclanShine]

Which API doesn't behave as documented, and how does it misbehave?

On apple devices above iOS 17, a clicking noise can be heard on each loop, when loopMode set to one.

On android devices and apple devices running iOS 15.5 no clicking noise can be heard. This is an iOS 17+ specific issue.

Minimal reproduction project

The example.

As per the instructions for contributing:
https://github.com/ryanheise/just_audio/blob/minor/CONTRIBUTING.md

" If the only change to the example necessary to reproduce the bug is the audio source URL, it will be sufficient to supply that URL without forking the repository, as long as you clarify in your bug report what we should do with that URL."

AUDIO SOURCE:

https://firebasestorage.googleapis.com/v0/b/frequencies-44e38.appspot.com/o/assets%2Faudio%2F174.mp3?alt=media&token=928ab936-12f5-4321-b17a-4c4942bc6b58

The audio source linked above can be looped, with loopMode set to one.

await player.setLoopMode(LoopMode.one);
await player.play();

To Reproduce (i.e. user steps, not code)
Steps to reproduce the behavior:

1. Play the audio file with loopMode set to one
2. Observe clicking sound on each loop

Error messages

None

Expected behavior
Gapless looping without clicking noises. We are using this plugin for an app with the purpose of relaxation, clicking noises are somewhat disturbing to the app experience.

Screenshots
None

Desktop (please complete the following information):
N/A

Smartphone (please complete the following information):

• Device: iPhone 12 Pro Max
• OS: iOS 17

Flutter SDK version

[✓] Flutter (Channel stable, 3.10.6, on macOS 13.5.1 22G90 darwin-x64, locale en-CH)
[✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
[✓] Xcode - develop for iOS and macOS (Xcode 15.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2021.1)
[✓] VS Code (version 1.85.1)
[✓] Connected device (2 available)
[✓] Network resources

Additional context
I have attempted to find a workaround or a fix for three weeks without any success. I appreciate your assistance and look forward to hearing from you. If there are any further questions I am happy to help.

[ryanheise]

Hmm, when I just listen to the audio file by clicking the link, it sounds like there is a sharp click at the end. I wonder if there's any artifact in the audio file, or whether that is just what it sounds like when you abruptly cut off a sine wave. Also, depending on the frequency of the sound wave, if you cut off a sine wave at a certain phase and then loop to start the sine wave where the beginning of the clip doesn't match in phase, I wonder whether that discontinuity might also explain the perceived click. If you want to just play a continuous sine wave, you might consider sound_generator.

[DeclanShine]

Thank you for the recommendation to use sound_generator, however we are looping a variety of audio files, which are not all continuous sine waves.

Just checked with the audio engineer for the project. He states that the audio files are seamlessly looping, no problem with the file itself. The end of the clip matches the beginning in terms of phase.

We are also experiencing this clicking noise on the loop of our other audio files, which are not continuous sine waves.

It is important to mention that the audio files do not click when looping on android devices or iPhones running iOS 15.

It seems that with the new iOS update, something has broken in the iOS loop implementation of just_audio.

Can you attempt to replicate the issue and confirm?

Many thanks.

[ryanheise]

I think I have actually seen something like this before but it was on the Simulator rather than a real device.

If you go through the iOS implementation code, it will be hard to point the finger at anything in the code since it is just letting AVQueuePlayer auto-advance to the next item in the queue. When looping a single item, the iOS implementation achieves this by making a copy of the first item so that it smoothly auto-advances to the second item (which is the same as the first item). It is possible perhaps for there to be a gap if the audio file being looped is too short and it doesn't have time to prepare the next item just in time, but I suspect what you're seeing is not that. It would be helpful if you could confirm if 1. you are using a real physical device, 2. the issue happens even when not looping and just concatenating multiple audio files together.

[DeclanShine]

1. Yes we are using a range of real physical devices to test.

2. I will have to create a test for this purpose and get back to you, as we are always looping the audio files in our use cases.

If you have any ideas in the meantime, anything is greatly appreciated. Many thanks.

[DeclanShine]

To follow up on point 2:

I have created a test where we are concatenating multiple of the same audio file together in a playlist using just_audio.
A clicking noise can still be heard each time the audio file plays within that playlist, while testing using testflight on a physical device.

[ryanheise]

That makes sense. Unfortunately, this means that it comes down to the mysterious behaviour of Apple's AVQueuePlayer. Fixing it may require black magic, or guidance from Apple (both of which are equally unlikely). What I can hypothesise is that some unique combination of factors causes it to happen, e.g. a vanilla usage of AVQueuePlayer might work, but then once you start adding on different key value observers, you might trigger this unexpected behaviour, which shouldn't happen in the first place, but it is impossible to see under the hood of AVQueuePlayer to see why it may sometimes not in fact produce gapless playback from one item to the next. Unless others want to perform experiments on enabling/disabling lines of code, this is likely to be an issue that remains open for quite some time.

P.S. Happy New Year (from Australia)

[DeclanShine]

Thank you Ryan!

Would you recommend getting in touch with apple support and submitting a TSI? What are the steps we could possibly take to help resolve this issue sooner?

Thanks a bunch.

Wishing you a happy new year too! :)

[DeclanShine]

I would like to add that we found a workaround. The issue is fixed when replacing the .mp3 file with a .wav file.

This however means that our filesizes are much larger which will result in higher firebase storage costs in the long run. Do you think it's possible to take a look at making a fix with this new knowledge?

[ryanheise]

That's interesting, that would suggest that the gap is caused by a delay in decoding. That's unexpected since AVQueuePlayer should start decoding the next queue item in advance. Although I don't think there is anything that can be done within the plugin to fix that, the API exposes preferredForwardBufferDuration which you might try playing with, but then if it's a case of the behaviour changing in iOS 17+, it sounds like maybe a bug introduced in iOS 17. Looking on Google, I found another report of the same issue:

https://developer.apple.com/forums/thread/738293

One other alternative you could experiment with is dynamically generating an HLS playlist and see if Apple's player can string a sequence together without gaps.

[DeclanShine]

Thanks Ryan. In the meantime I contacted apple and they had the following to say:

"You mentioned that the developer of the just_audio library conjectured the issue might be associated with AVQueuePlayer. Once this developer has identified the specific issue, they should either submit a TSI, if they require code-level assistance with their implementation, or use Feedback Assistant to submit a bug report.

You also mentioned that the issue seems to be related to iOS 17, which is an indication that submitting a bug report via Feedback Assistant might be the correct route, especially if the underlying implementation has not changed.

Of course, you are welcome to submit a bug report yourself, but if the engineering team evaluating your bug report determines the issue is being caused by an implementation error and not by a bug with AVQueuePlayer, then the developers of just_audio or Flutter would need to fix the error on their side. That is why we recommend that the TSI and/or bug report be submitted by the party implementing AVQueuePlayer."

Would it be possible for you to submit a bug report on this issue?

[wrunk]

@DeclanShine I've experienced this issue with another non-flutter swift/swiftUI app and unfortunately had to move away from avqueueplayer and just use avaudioplayer and pass previously downloaded mp3 files which of course is not ideal (luckily in our case our looping sound files are small so this is OK for now).

One possible solution I just thought of based on your comments about .wav files is to try other similar formats such as ogg or flac that might be smaller than wav. That might cut down on storage and bandwidth costs (if either of those work).

Anyway I'll post again if I find a better solution and please do as well.

[gitmole7]

Running into the exact same problem - .mp3 files loop seamlessly on a simulator, but have a gap when played back on a physical device running iOS 17.

Using .wav files has worked for me too, but of course they are much bigger files. I’ve found the best workaround to be using .m4a files - these loop seamlessly and are smaller than .wav.

I've found just one issue with using .m4a - after calling .pause() and .play(), the file plays back with a gap on the first loop, then without gaps thereafter. Again, this is only present on a physical device running iOS 17, and .m4a files loop seamlessly right away and regardless of toggling pause/play on Android and iOS simulator.

[ryanheise]

It's very interesting that .m4a works, as that will be a viable option for many people. I suspect that most developers who have gapless looping requirements probably use their own assets and can transcode them.

Regarding non-AVQueuePlayer implementations, #334 is another avenue that could be explored more long term (although since this is likely a temporary bug, it may be that by the time such an alternative implementation is ready for prime time, Apple has already fixed the issue...)

So I think I would rank these options in priority of:

1. Try transcoding assets to m4a if that's an option
2. Submit a bug report to Apple (see below)
3. Build a non-AVQueuePlayer implementation

It may be worth writing an extremely stripped-back example of using AVQueuePlayer without any bells and whistles, to see if it still exhibits these gaps. On my end, that's the next thing I would be interested to explore before reporting this to Apple.