AI-186: Evaluate faster-whisper's performance on device

[itsPronay]

Jira - https://mifosforge.jira.com/browse/AI-186

This PR introduces the main benchmarking script for evaluating Faster Whisper models on mobile devices.

Changes:

• Profiles both encoder and decoder on the selected device(s).
• Extracts and logs key metrics, including latency and memory usage.
• Supports logging results to Weights & Biases (online/offline modes).
• Modularizes device selection for easy extension.

Results

[biplab1]

@itsPronay You could improve legibility of the graphs by using larger font sizes and darker colors for titles, labels, and scales. Also, consider reducing the thickness of the bars and using thicker x-axis line in the graph for a cleaner appearance.

For example:

Use a Logarithm graph for graphs where few values are much larger than the others with zoomed in inset linear graph for the smaller values. For example:

[itsPronay]

@biplab1 Thanks for your suggestions.

@itsPronay You could improve legibility of the graphs by using larger font sizes

We cannot freely modify font sizes in Weights & Biases. We are limited to predefined options: small, medium, and large.
The screenshot below uses the largest available option.

Regarding the titles (e.g., estimated_latency_ms), these are dictionary keys. In Python, the recommended convention is to use lowercase with underscores, so the titles naturally follow that format.

While it is possible to modify titles in the web UI, those changes are only local and will not persist if someone else reruns the notebook.

and darker colours for titles, labels, and scales. Also, consider reducing the thickness of the bars and using thicker x-axis line in the graph for a cleaner appearance. Use a logarithmic graph for cases where a few values are much larger than others, along with a zoomed-in inset for smaller values.

Most of these aspects are not configurable in Weights & Biases (except for scale colors, those RGB colours in the image).

That being said, while visual clarity is important but the primary focus here is on effective experiment tracking and comparison using weights & biases.

[biplab1]

@itsPronay Ah got it, I didn’t realize these were auto-generated in Weights & Biases. Also, I meant only title size, not content, examples were just for illustration.

[staru09]

what's the need of qualcomm ai hub here?

[itsPronay]

what's the need of qualcomm ai hub here?

@staru09 , Qualcomm AI Hub is needed to run models on mobile devices (qualcomm powered mobile devices.)

The target devices you see here (e.g., S25 Ultra, S24, Pixel 3a) are provided by Qualcomm AI Hub for benchmarking purposes.

After running the evaluations, it provides detailed benchmarking results such as latency, memory usage, and overall performance, which helps in analyzing how the model behaves across different hardware configurations.

See sample output of Qualcomm AI Hub here

[DavidH-1]

CLA check = Passed