This software is provided “as is,” without any express or implied warranties. The authors and contributors shall not be held liable for any damages arising from its use. The code may be incomplete or insufficiently tested. Users are solely responsible for evaluating its suitability and assume all associated risks.
Note: Contributions are welcome. Please ensure thorough testing before deploying in critical systems.
The Quick AI Application Builder (this repository) is also referred to as "QAI AppBuilder" in the source code and documentation.
QAI AppBuilder is an extension of the Qualcomm® AI Runtime SDK, which is used to simplify the deployment of QNN models. Some libraries from the Qualcomm® AI Runtime SDK are required to use QAI AppBuilder.
QAI AppBuilder is designed to help developers easily use the Qualcomm® AI Runtime SDK to execute models on Windows on Snapdragon (WoS) and Linux platforms. It encapsulates the model execution APIs into a set of simplified interfaces for loading models onto the NPU/HTP and performing inference. It substantially lowers the complexity of model deployment for developers.
Qualcomm® AI Runtime SDK is designed to provide unified, low-level APIs for AI development. It can be downloaded from Qualcomm software center:
https://softwarecenter.qualcomm.com/#/catalog/item/Qualcomm_AI_Runtime_SDK
Or from QPM [this option expected to be deprecated soon]
https://qpm.qualcomm.com/#/main/tools/details/Qualcomm_AI_Runtime_SDK
Developers can use QAI AppBuilder in both C++ and Python projects
• Support both C++ & Python
• Support both Windows & Linux
• Support Genie(Large Language Model) [NEW!]
• Support Multi Graph [NEW!]
• Support LoRA
• Support multiple models
• Support multiple inputs & outputs
• Easier for developing apps
• Faster for testing models
• Plenty of sample code
** Support ARM64 Windows, Linux and Ubuntu (e.g.: X Elite Windows, QCS8550 Linux and QCM6490 Ubuntu).
** Support OpenAI Compatible API Service(GenieAPIService) on WoS, Android and Linux.
Refere to python.md for instructions on setting up the Python(x64 version) environment to use QAI AppBuilder on Windows on Snapdragon (WoS) platforms.
You can also run the batch file Setup_QAI_AppBuilder.bat to setup the environment automatically.
We have developed several WebUI AI applications based on QAI AppBuilder, allowing you to experience them quickly.
All these applications run on a local PC, requiring no internet connection and are completely free.
You can run these WebUI AI applications througn the batch file Start_WebUI.bat automatically.
| App | Description |
|---|---|
| ImageRepairApp | An image restoration tool designed to repair old or damaged photographs. |
| StableDiffusionApp | A text-to-image generation tool that creates images based on user input. |
| GenieWebUI | A large language model (LLM) interface that enables interactive conversations. |
Considering that the current mainstream method for invoking LLMs is based on OpenAI-compatible APIs, we have implemented such interfaces in both C++ and Python. This allows application developers to interact with the local large language model running on NPU in a familiar way.
Many third-party applications that support the OpenAI API can seamlessly switch to the local NPU-based model by simply changing the API IP endpoint.
We have also implemented the client sample code of Genie API Service through both C++ and Python for the reference of developers.
- Python based service: Guide to run OpenAI compatible API services developed with python.
- C++ based service: Guide to run OpenAI compatible API services developed with C++.
We have many samples which can be run directly. The models used in these samples are from AI-Hub.
We suggest that you try to run these sample code before you actually start writing your own code.
We have LangFlow Package for running QNN model in 3rd party LangFlow framework.
There're two ways to use QAI AppBuilder:
Download prebuild binary package QAI_AppBuilder-win_arm64-{Qualcomm® AI Runtime SDK version}-Release.zip to get these files: https://github.com/quic/ai-engine-direct-helper/releases
Download Python extension qai_appbuilder-{version}-cp312-cp312-win_amd64.whl and install it with the command below: https://github.com/quic/ai-engine-direct-helper/releases
pip install qai_appbuilder-{version}-cp312-cp312-win_amd64.whl
Refere to User Guide on how to use QAI AppBuilder to program AI application.
Refer to tutorial.ipynb to setup and run a cv model step by step.
You can use the pre-compiled version directly and download the version you need from Release. But if you really want to compile it yourself, you can refer to the BUILD.md.
QAI AppBuilder is licensed under the BSD 3-clause "New" or "Revised" License. Check out the LICENSE for more details.