Merge v0.20.0 - #6
Open
billycao wants to merge 58 commits into
Open
Conversation
Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> (cherry picked from commit 43a013c)
…llm-project#33093) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com> (cherry picked from commit a2393ed)
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn> Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com> Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> Signed-off-by: youkaichao <youkaichao@gmail.com> Signed-off-by: Roger Wang <hey@rogerw.io> Co-authored-by: wanglinian <wanglinian@stu.pku.edu.cn> Co-authored-by: wangln19 <96399074+wangln19@users.noreply.github.com> Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn> Co-authored-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: youkaichao <youkaichao@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> (cherry picked from commit b539f98)
Signed-off-by: NickLucche <nlucches@redhat.com> (cherry picked from commit 1f3a2c2)
…oject#33090) Signed-off-by: NickLucche <nlucches@redhat.com> (cherry picked from commit 492a798)
Signed-off-by: Nick Hill <nickhill123@gmail.com> (cherry picked from commit 0cd259b)
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com> (cherry picked from commit a97b5e2)
…roject#30207)" (vllm-project#33241) Signed-off-by: Or Ozeri <oro@il.ibm.com> Co-authored-by: Kevin H. Luu <khluu000@gmail.com> (cherry picked from commit 2e8de86)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com> (cherry picked from commit f9d0359)
…M100 (vllm-project#33285) Signed-off-by: mgoin <mgoin64@gmail.com> (cherry picked from commit 1bd47d6)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> (cherry picked from commit abb34ac)
…oject#33129) Signed-off-by: khluu <khluu000@gmail.com> (cherry picked from commit 2284461)
…t#32831 (vllm-project#33366) Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com> (cherry picked from commit 31aedfe)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Michael Goin <mgoin64@gmail.com> (cherry picked from commit 0a3c71e)
…ng kv cache update to splitting ops (vllm-project#33441) Signed-off-by: Luka Govedič <lgovedic@redhat.com> Co-authored-by: Richard Zou <zou3519@gmail.com> (cherry picked from commit 15f40b2)
…oE kernels (vllm-project#33417) Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> (cherry picked from commit 0797811)
Signed-off-by: greg pereira <grpereir@redhat.com> Co-authored-by: Michael Goin <mgoin64@gmail.com> (cherry picked from commit d6416fd)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: i-zhangmingming <i-zhangmingming@stepfun.com> Co-authored-by: xiewuxun <xiewuxun@stepfun.com> Co-authored-by: zetaohong <i-hongzetao@stepfun.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> (cherry picked from commit c3b40dc)
Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> (cherry picked from commit 318b120)
…project#33524) Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu> (cherry picked from commit a01ef3f)
…ect#33602) Signed-off-by: zhewenli <zhewen@inferact.ai> Co-authored-by: zhewenli <zhewen@inferact.ai>
Signed-off-by: zhewenli <zhewen@inferact.ai> Co-authored-by: zhewenli <zhewen@inferact.ai>
Signed-off-by: Zachary Aristei <zaristei@nvidia.com> Co-authored-by: Zachary Aristei <zaristei@nvidia.com>
Signed-off-by: Zachary Aristei <zaristei@nvidia.com> Co-authored-by: Zachary Aristei <zaristei@nvidia.com>
Signed-off-by: kiersten-stokes <kierstenstokes@gmail.com> (cherry picked from commit 9e138cb)
…e is speculative decoding (vllm-project#33624) Signed-off-by: Richard Zou <zou3519@gmail.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> (cherry picked from commit 5eac9a1)
…LLM per-tensor FP8 MoE (vllm-project#33620) Signed-off-by: mgoin <mgoin64@gmail.com> (cherry picked from commit e346e2d) Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
…lm-project#33729) Signed-off-by: Nick Hill <nickhill123@gmail.com>
Handle models that consider tool-open tags to also be reasoning-close tags
* Add --max-request-secs to vllm serve * Fix import location and SIM105 errors in serving.py files Signed-off-by: Billy Cao <the.billy.c@gmail.com> * Remove unused _timeout_watcher function from engine/serving.py Signed-off-by: Billy Cao <the.billy.c@gmail.com> * Move duplicated functions to base class Signed-off-by: Billy Cao <the.billy.c@gmail.com> * Add files to synthetic Dockerfile Signed-off-by: Billy Cao <the.billy.c@gmail.com> * Update comment Signed-off-by: Billy Cao <the.billy.c@gmail.com> --------- Signed-off-by: Billy Cao <the.billy.c@gmail.com>
…ose thinking via tool start tags
…se-kimi Handle the case in which there's no opening think tag for Kimi autoclose thinking via tool start tags
Handle embedded tools in reasoning more robustly
Fix tool call parsing for Kimi K2.5
… tool parser, adopt upstream reasoning parser
…project#37861)" This reverts commit d2e2e85.
…ng bound (vllm-project#40946) Signed-off-by: Dao Le <Dao007forever@gmail.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> (cherry picked from commit 7b1bc0a)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> (cherry picked from commit 706a04d)
Signed-off-by: Zhewen Li <zhewenli@inferact.ai> Co-authored-by: Zhewen Li <zhewenli@inferact.ai> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> (cherry picked from commit 6232fb4)
Signed-off-by: Billy Cao <the.billy.c@gmail.com>
Signed-off-by: Billy Cao <the.billy.c@gmail.com>
Signed-off-by: Billy Cao <the.billy.c@gmail.com>
Signed-off-by: Billy Cao <the.billy.c@gmail.com>
Signed-off-by: Billy Cao <the.billy.c@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Also update
cu130.Dockerfileto preserve built library files (eg. .so files) which are not part of the git checkout.