Skip to content

Commit 8444c86

Browse files
fix(agentx): keep the measured LMCache message queue timeout on MI300X
The role connector is now the lmcache-mp preset written out plus lmcache.mp.mq_timeout 6000, so the point keeps its measured timeout instead of falling back to LMCache's 300 s default.
1 parent cd2dbd5 commit 8444c86

1 file changed

Lines changed: 3 additions & 2 deletions

File tree

  • inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm3/vllm/mi300x-fp8-mtp

‎inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm3/vllm/mi300x-fp8-mtp/agentic.yaml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -155,8 +155,9 @@ override_tp8_c16_lmcache:
155155
agg:
156156
args:
157157
max-num-seqs: 32
158-
# LMCacheMPConnector to the lmcache-server service on this node (port 8750).
159-
connector: lmcache-mp
158+
# The lmcache-mp preset (the node's lmcache-server on port 8750) plus the
159+
# measured message queue timeout.
160+
connector: '{"kv_connector":"LMCacheMPConnector","kv_connector_module_path":"lmcache.integration.vllm.lmcache_mp_connector","kv_role":"kv_both","kv_connector_extra_config":{"lmcache.mp.host":"tcp://localhost","lmcache.mp.port":8750,"lmcache.mp.mq_timeout":6000.0}}'
160161
benchmark:
161162
env:
162163
CONC: '16'

0 commit comments

Comments
 (0)