Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AudioDecoder should let you specify an format for resulting AudioData #859

Open
jyavenard opened this issue Dec 11, 2024 · 0 comments
Open

Comments

@jyavenard
Copy link
Member

The User Agent is free to choose how the decoding will be performed and what format it will use when creating the AudioData.

In theory, how the data is stored should indeed be an implementation details.

However, what format is chosen will impact on how AudioData.copyTo will behave and how it will interpret the planeIndex parameter.
Additionally, there's a few WPTs that relies on the decoded AudioData to be interleaved and error otherwise (such as https://wpt.live/webcodecs/audio-encoder.https.any.js and its channelNumberVariationTests)

AudioData::copyTo
"The output AudioSampleFormat for the destination data. If not provided, the resulting copy will use this AudioData’s [[format]"

With the default option, if we were to call copyTo with a planeIndex value of 0 and the AudioData was interleaved, we would copy all planes.
While if the AudioData was planar, copyTo with a planeIndex of 0 would copy only that particular plane.

That the internal UA choice on what to use impact the default behaviour for later created objects and change the behaviour means that it cannot be considered an implementation detail, it's not transparent and is directly observable.
And real-life experience already shows web compatibility between Chrome and Firefox (such as https://w3c.github.io/webcodecs/samples/audio-video-player/audio_video_player.html where the code expects the decoded AudioData to be planar and will have an exception as it attempts to copy the planar data in an array of float) as it doesn't specify what format to use with copyTo)

So we should specify what the default format the AudioDecoder should use and provide the ability to override it in the configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant