Repository navigation

pull

[executorch][cuda] Add a batched-sequence CUDA KV cache #101619

Run time

Learn about OS pricing on GitHub Actions

Job	Run time
CI run decision / decide	31s
unittest / Resolve CI docker image / resolve	7s
Get changed files / get-changed-files	5s
unittest / windows / windows-job	1h 48m 30s
Resolve CI docker image / resolve	6s
android / build-android	10m 7s
unittest-buck / macos / macos-job	10m 15s
unittest / macos / macos-job	44m 12s
unittest-buck / Resolve CI docker image / resolve	14s
unittest-buck / windows	0s
test-arduino-library / Resolve CI docker image / resolve	6s
nxp-mcuxpresso-test	-1s
unittest-editable	-1s
test-qnn-testsuite-linux	-1s
unittest / linux / linux-job	53m 10s
test-pybindings-no-aten / linux-job	7m 12s
test-arm-backend-no-driver	-1s
test-minimal-wheel-linux	-1s
test-arm-backend-public-api-backward-compatibility	-1s
test-qnn-wheel-packages-linux	-1s
test-qnn-delegate-linux	-1s
test-mcu-cortex-m-backend	-1s
test-qnn-models-linux	-1s
unittest-nxp-neutron	0s
test-arm-backend-docgen	-1s
unittest-wasm-bindings	0s
test-qnn-python-imports-linux	0s
test-qnn-buck-build-linux	0s
test-qnn-passes-linux	-1s
test-qnn-direct-build-linux	0s
nxp-build-test	0s
test-arm-backend-vkml	0s
test-vulkan-models-linux	0s
test-build-wasm-linux	0s
test-vulkan-operators-linux	0s
test-mediatek-models-linux	0s
test-openvino-linux	0s
test-models-linux-basic (mv3, portable, buck2, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job	7m 0s
test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, mt-l-arm64g4-16-62, executo... / linux-job	5m 54s
test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, mt-l-x86iavx512-8-64, execu... / linux-job	10m 26s
test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, mt-l-x86iavx512-8-64, execu... / linux-job	9m 32s
test-parakeet-xnnpack-linux / linux-job	14m 39s
test-models-linux-basic (vit, portable, buck2, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job	8m 45s
test-models-linux-basic (vit, portable, cmake, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-... / linux-job	7m 27s
test-models-linux-basic (mv3, portable, cmake, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-... / linux-job	5m 31s
test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, mt-l-x86iavx512-8-64, execu... / linux-job	7m 45s
test-selective-build-linux / linux-job	9m 24s
test-models-linux-basic (mv3, portable, cmake, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job	7m 57s
test-models-linux-basic (vit, portable, cmake, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job	9m 32s
test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, mt-l-arm64g4-16-62, executo... / linux-job	6m 12s
test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, mt-l-x86iavx512-8-64, execu... / linux-job	8m 46s
test-setup-linux-gcc / linux-job	6m 3s
test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, mt-l-arm64g4-16-62, executorch-ubuntu-... / linux-job	9m 9s
test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, mt-l-x86iavx512-8-64, executorch-ubunt... / linux-job	12m 15s
test-llama-runner-linux (fp32, xnnpack+quantize_kv, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-g... / linux-job	8m 9s
test-llama-runner-linux (fp32, xnnpack+custom+qe, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-c... / linux-job	10m 34s
test-llama-runner-linux (fp32, xnnpack+quantize_kv, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04... / linux-job	8m 52s
test-llama-runner-linux (fp32, xnnpack+custom+qe, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc... / linux-job	9m 13s
test-llama-runner-linux (bf16, custom, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clang12) / linux-job	7m 28s
test-models-linux (mobilebert, portable, mt-l-x86iavx512-8-64) / linux-job	7m 18s
test-models-linux (mobilebert, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job	7m 23s
test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job	8m 23s
test-models-linux (linear, portable, mt-l-x86iavx512-8-64) / linux-job	6m 10s
test-models-linux (add, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job	8m 3s
test-models-linux (resnet18, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job	6m 29s
test-models-linux (resnet18, portable, mt-l-x86iavx512-8-64) / linux-job	6m 38s
test-models-linux (emformer_join, portable, mt-l-x86iavx512-16-128) / linux-job	13m 41s
test-models-linux (w2l, portable, mt-l-x86iavx512-16-128) / linux-job	6m 5s
test-models-linux (linear, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job	6m 25s
test-models-linux (mv2, portable, mt-l-x86iavx512-8-64) / linux-job	8m 4s
test-models-linux (emformer_transcribe, portable, mt-l-x86iavx512-8-64) / linux-job	6m 48s
test-models-linux (add_mul, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job	6m 50s
test-models-linux (ic4, portable, mt-l-x86iavx512-16-128) / linux-job	12m 18s
test-models-linux (phi_4_mini, portable, mt-l-x86iavx512-16-128) / linux-job	41m 52s
test-models-linux (ic4, xnnpack-quantization-delegation, mt-l-x86iavx512-16-128) / linux-job	6m 46s
test-models-linux (ic3, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job	7m 1s
test-models-linux (add, portable, mt-l-x86iavx512-8-64) / linux-job	7m 36s
test-models-linux (add_mul, portable, mt-l-x86iavx512-8-64) / linux-job	7m 38s
test-models-linux (mv2, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job	8m 12s
test-models-linux (llama3_2_vision_encoder, portable, mt-l-x86iavx512-16-128) / linux-job	14m 33s
test-models-linux (resnet50, portable, mt-l-x86iavx512-8-64) / linux-job	9m 35s
test-models-linux (emformer_join, xnnpack-quantization-delegation, mt-l-x86iavx512-16-128) / linux-job	6m 5s
test-models-linux (ic3, portable, mt-l-x86iavx512-8-64) / linux-job	7m 28s
test-models-linux (resnet50, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job	6m 27s
test-multimodal-linux (gemma3-4b) / linux-job	24m 17s
test-static-llama-qnn-linux (stories_260k_bc) / linux-job	13m 20s
test-static-llama-qnn-linux (stories_110m) / linux-job	21m 37s
test-custom-ops-linux / linux-job	5m 52s
test-arm-cortex-m-size-test (bare_metal) / linux-job	10m 7s
test-binary-size-linux-gcc / linux-job	6m 32s
test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job	27m 24s
test-llama-runner-linux-android / linux-job	13m 45s
test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job	20m 20s
test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job	13m 22s
test-voxtral-realtime-xnnpack-linux / linux-job	20m 49s
test-moshi-linux / linux-job	12m 56s
test-binary-size-linux / linux-job	4m 54s
test-quantized-aot-lib-linux / linux-job	8m 18s
test-eval_llama-wikitext-linux / linux-job	9m 25s
test-arm-cortex-m-size-test (zephyr-preset) / linux-job	9m 21s
test-llama_runner_eager-linux / linux-job	8m 43s
test-lora-linux / linux-job	27m 35s
test-lora-multimethod-linux / linux-job	19m 21s
test-phi-3-mini-runner-linux / linux-job	14m 4s
unittest-buck / linux / linux-job	8m 23s
test-arduino-library / run / arduino-library	15m 44s
test-coreml-bc-macos (${{ matrix.runner }})	0s
llm-server	0s
android / run-emulator	45m 34s
	17h 0m 27s

Navigation Menu

Repository navigation

[executorch][cuda] Add a batched-sequence CUDA KV cache #101619

Usage

[executorch][cuda] Add a batched-sequence CUDA KV cache #101619

Uh oh!

Run time