Skip to content

Commit 7ffbae7

Browse files
authored
[Bugfix] Fix KV Cache Spec CI Failure (#787)
1 parent 303ab6f commit 7ffbae7

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

tpu_commons/worker/tpu_worker_jax.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -255,6 +255,9 @@ def get_kv_cache_spec(self) -> dict[str, KVCacheSpec]:
255255
# and the vLLM side should be updated to handle the translation.
256256
kv_cache_specs = self.model_runner.get_kv_cache_spec()
257257

258+
if len(kv_cache_specs) == 0:
259+
return kv_cache_specs
260+
258261
# TODO(kyuyeunk): Instead of checking page_size_bytes here, introduce
259262
# feature that allows overriding page_size_bytes of KVCacheSpec.
260263
vllm_page_size_bytes = get_uniform_page_size(kv_cache_specs)

0 commit comments

Comments
 (0)