[release/v1.21.0]: Updating docs for 1.21.0 #683
Draft
+324
−36
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Abukhoyer Shaik <[email protected]>
abukhoy
requested review from
ochougul,
quic-amitraj,
quic-hemagnih and
quic-rishinr
as code owners
Signed-off-by: Abukhoyer Shaik <[email protected]>
quic-rishinr
requested changes
Dec 22, 2025
docs/source/release_docs.md
Outdated
|
|
||
| - **Mistral 3.1 (24B)** | ||
| - Executable via [`QEffAutoModelForCausalLM`](#QEffAutoModelForCausalLM) | ||
| - Production-ready deployment |
docs/source/release_docs.md
Outdated
| - **Olmo2** | ||
| - Executable via [`QEffAutoModelForCausalLM`](#QEffAutoModelForCausalLM) | ||
| - Full CausalLM support with optimizations | ||
| - Bug fixes included |
Signed-off-by: Abukhoyer Shaik <[email protected]>
Signed-off-by: Abukhoyer Shaik <[email protected]>
Contributor
|
We are missing a few more models, Olmo, Molmo and Wave2Vec2 @abukhoy please add these models as well |
Signed-off-by: Abukhoyer Shaik <[email protected]>
Signed-off-by: Abukhoyer Shaik <[email protected]>
Signed-off-by: Abukhoyer Shaik <[email protected]>
Signed-off-by: Abukhoyer Shaik <[email protected]>
Signed-off-by: Abukhoyer Shaik <[email protected]>
Signed-off-by: Abukhoyer Shaik <[email protected]>
Signed-off-by: Abukhoyer Shaik <[email protected]>
quic-rishinr
requested changes
Dec 24, 2025
docs/source/release_docs.md
Outdated
| ## Key Features & Enhancements | ||
|
|
||
| - **Framework Upgrades**: Transformers `4.55`, PyTorch `2.7.0+cpu`, Torchvision `0.22.0+cpu` | ||
| - **Python Support**: Now requires Python `>=3.9` |
Contributor
There was a problem hiding this comment.
better to keep python support as 3.10
| @@ -0,0 +1,84 @@ | |||
| # Diffuser Classes | |||
Contributor
There was a problem hiding this comment.
Can we follow the similar approach like qeff_autoclasses.html? Add small examples and keep only the user exposed classes? @quic-amitraj can you suggest
| ``` | ||
|
|
||
| (QEFFAutoModelForCTC)= | ||
| ## `QEFFAutoModelForCTC` |
Contributor
There was a problem hiding this comment.
please add an example here
docs/source/supported_features.rst
Outdated
| - Enabled for GPT-OSS models, allowing for flexible deployment of large language models across different hardware configurations. | ||
| * - `ONNX Sub-Functions <https://github.com/quic/efficient-transformers/pull/621>`_ | ||
| - Feature enabling more efficient model compilation and execution on hardware. | ||
| * - `Continuous Batching (VLMs) <https://github.com/quic/efficient-transformers/pull/610>`_ |
Contributor
There was a problem hiding this comment.
you can remove Continuous Batching (VLMs)
docs/source/supported_features.rst
Outdated
| - Implements a blocked K/V cache layout so attention reads/processes the cache blockbyblock, improving longcontext decode performance. | ||
| * - `Memory Profiling Tool <https://github.com/quic/efficient-transformers/pull/674>`_ | ||
| - Adds scripts to profile memory during export/compile/infer (peak usage, cache footprint) for quicker diagnosis. Refer `sample scripts <https://github.com/quic/efficient-transformers/tree/main/scripts/memory_profiling>`_ for more **details**. | ||
| * - `ONNX transform, memory & time optimizations <https://github.com/quic/efficient-transformers/pull/640>`_ |
Contributor
There was a problem hiding this comment.
remove it. it's an optimization and it's not a standalone feature
docs/source/supported_features.rst
Outdated
| - Adds scripts to profile memory during export/compile/infer (peak usage, cache footprint) for quicker diagnosis. Refer `sample scripts <https://github.com/quic/efficient-transformers/tree/main/scripts/memory_profiling>`_ for more **details**. | ||
| * - `ONNX transform, memory & time optimizations <https://github.com/quic/efficient-transformers/pull/640>`_ | ||
| - Adds periodic memory cleanup (e.g., to FP16ClipTransform / SplitTensorsTransform) during largetensor processing, and avoids redundant external data loading when already present | ||
| * - Onboarding Guide |
docs/source/supported_features.rst
Outdated
| - Extended to Vision Language Models with multi-image handling capabilities, optimizing throughput and latency by dynamically batching requests with varying image counts. Refer `sample script <https://github.com/quic/efficient-transformers/blob/main/examples/image_text_to_text/models/granite_vision/continuous_batching.py>`_ for more **details**. | ||
| * - `BlockedKV attention in CausalLM <https://github.com/quic/efficient-transformers/pull/618>`_ | ||
| - Implements a blocked K/V cache layout so attention reads/processes the cache blockbyblock, improving longcontext decode performance. | ||
| * - `Memory Profiling Tool <https://github.com/quic/efficient-transformers/pull/674>`_ |
| | Architecture | Model Family | Representative Models | [vLLM Support](https://quic.github.io/cloud-ai-sdk-pages/latest/Getting-Started/Installation/vLLM/vLLM/index.html) | | ||
| |-------------------------|--------------------|--------------------------------------------------------------------------------------|--------------| | ||
| | **FalconForCausalLM** | Falcon** | [tiiuae/falcon-40b](https://huggingface.co/tiiuae/falcon-40b) | ✔️ | | ||
| | **MolmoForCausalLM** | Molmo① | [allenai/Molmo-7B-D-0924](https://huggingface.co/allenai/Molmo-7B-D-0924) | ✕ | |
Contributor
There was a problem hiding this comment.
There are some characters coming in these. Check for all the newly added models
| | Architecture | Model Family | Representative Models | [vLLM Support](https://quic.github.io/cloud-ai-sdk-pages/latest/Getting-Started/Installation/vLLM/vLLM/index.html) | | ||
| |-------------------------|--------------------|--------------------------------------------------------------------------------------|--------------| | ||
| | **FalconForCausalLM** | Falcon** | [tiiuae/falcon-40b](https://huggingface.co/tiiuae/falcon-40b) | ✔️ | | ||
| | **MolmoForCausalLM** | Molmo① | [allenai/Molmo-7B-D-0924](https://huggingface.co/allenai/Molmo-7B-D-0924) | ✕ | |
Contributor
There was a problem hiding this comment.
Check with @quic-vargupt and confirm if these models are supported on vllm and update it accordingly
Signed-off-by: Abukhoyer Shaik <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is created for updating documentation for 1.21.0 release.
Note: First Draft