Skip to content

Whisper API not compatible with OpenAI #5327

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mutschler opened this issue May 6, 2025 · 0 comments
Open

Whisper API not compatible with OpenAI #5327

mutschler opened this issue May 6, 2025 · 0 comments
Labels
bug Something isn't working unconfirmed

Comments

@mutschler
Copy link

mutschler commented May 6, 2025

LocalAI version:
localai: stable 2.28.0 (bottled), HEAD (via brew)

Environment, CPU architecture, OS, and Version:
Darwin MacBook-Pro.local 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:59 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6030 arm64

Describe the bug
The whisper api endpoint (/v1/audio/transcriptions) isn't compatible to OpenAI's implementation.

While support for response_format (and other request parameters) would be amazing i think at least the default response should be compatible with OpenAI's verbose_json.

To Reproduce
spin up localai, install whisper-1 model, send any audio file to /v1/audio/transcriptions and check the response

Expected behavior
Get a response compatible with https://platform.openai.com/docs/api-reference/audio/verbose-json-object

Additional context
i know that OpenAi defaults to json for response_format but since the parameter isn't supported and the current answer is already close to the verbose_json format changing it so one could use official libraries or use localai as drop in replacement

The main differences i spotted:

task, language and duration are missing from the top level
seek, temperature, avg_logprob, compression_ratio and no_speech_prob are missing from each segment and the start and stop values seem to be (rounded?) Nanoseconds instead of Seconds (5000000000 instead of 5.119999999999998 for around 5 seconds)

@mutschler mutschler added bug Something isn't working unconfirmed labels May 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working unconfirmed
Projects
None yet
Development

No branches or pull requests

1 participant