Skip to content

Support MLA for nixl_connector #1749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 18 commits into
base: skaul_nixl
Choose a base branch
from
Open

Support MLA for nixl_connector #1749

wants to merge 18 commits into from

Conversation

srajabos
Copy link

@srajabos srajabos commented Aug 12, 2025 •
edited by github-actions bot
Loading

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results

Purpose

Test Plan

Test Result

srajabos and others added 16 commits August 14, 2025 14:11
@srajabos
fix typo
a168509
@bukeao @srajabos
move the interface functions from nixl to the their appropriate locat…
b2ba25f 
…ions within hpu_model_runner
@skaulintel @srajabos
temporarily comment out asserts
ca02051
@bukeao @srajabos
temperary code to fix the input_batch.req_type has the right flag; te…
5984d93 
…sted in high concurret requests
@libinta @srajabos
Fix D node processing prompt
0264e11
@libinta @srajabos
change the chunked prefill max to 8k
7a48a04
@libinta @srajabos
add requirement and modification to accuracy.sh
3aaf907
@libinta @srajabos
improve copy func and revert get_prompts_and_decode
43791b2
@libinta @srajabos
revert benchmark test with more prompts
ebe97e3
@libinta @srajabos
modify log
fc202ec
@libinta @srajabos
change input to 8k
a608a57
@libinta @srajabos
fix run_accuracy issue with allowing prefill on decode for case that …
fa6235c 
…prompt is not a multipleir of block size
@skaulintel @srajabos
support pin memory
1d216d2
@libinta @srajabos
rewrite copy function and add benchmark profile
89b759e
@srajabos
rebase
fcd0db3
@libinta @srajabos
rebase
c1c2e40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kzawora-intel kzawora-intel Awaiting requested review from kzawora-intel kzawora-intel is a code owner

@madamczyk-intel madamczyk-intel Awaiting requested review from madamczyk-intel madamczyk-intel is a code owner

@michalkuligowski michalkuligowski Awaiting requested review from michalkuligowski michalkuligowski is a code owner

@mgawarkiewicz-intel mgawarkiewicz-intel Awaiting requested review from mgawarkiewicz-intel mgawarkiewicz-intel is a code owner

@vivekgoe vivekgoe Awaiting requested review from vivekgoe vivekgoe is a code owner

@afierka-intel afierka-intel Awaiting requested review from afierka-intel afierka-intel is a code owner

@xuechendi xuechendi Awaiting requested review from xuechendi xuechendi is a code owner

@jikunshang jikunshang Awaiting requested review from jikunshang jikunshang is a code owner

@mswiniarsk mswiniarsk Awaiting requested review from mswiniarsk mswiniarsk is a code owner

@PatrykWo PatrykWo Awaiting requested review from PatrykWo PatrykWo is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@srajabos @bukeao @skaulintel @libinta