Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ The prompt system also recognizes `cuda2hip`; the current bundled task tree does
- Node.js 22+ and npm when using the alternative npm installation of Claude Code
(or another npm-installed agent CLI); DeepSeek Harness uses a dedicated
Node.js 24 prefix as described in [its setup guide](agents/deepseek_harness/README.md)
- For MI300/MI355X, use the GPU-specific SGLang image: `gfx942` uses `lmsysorg/sglang:v0.5.12-rocm720-mi30x`; `gfx950` uses `lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260705`
- For MI300/MI355X, use the GPU-specific SGLang image: `gfx942` uses `lmsysorg/sglang:v0.5.12-rocm720-mi30x`; `gfx950` uses the digest-pinned SGLang 0.5.20 / ROCm 10 runtime. See the [runtime compatibility matrix](docs/reference/compatibility-matrix.md) for image selection and task-specific requirements.
- For RDNA4 `gfx1201`, the runner automatically builds the default [pinned RDNA4 runtime](docker/rdna4/README.md) on first use if it is missing.
- A supported agent CLI installed and logged in on the host, or the dependencies required by a specialized agent

Expand Down
6 changes: 5 additions & 1 deletion agents/geak/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,11 @@ Docker provisions Claude for `geak` and its v2 aliases, mounts only the selected
GEAK checkout read-only, and forwards `GEAK_HOME`. The complete checkout is
needed for revision checks and private engine/knowledge copies. Preflight
installs the pinned SDK into the GEAK-only dependency directory when necessary
and verifies the clean upstream pin. These checks do not certify a live
and verifies the clean upstream pin. SDK dependencies are cached under
`.aka-pyuserbase/geak-sdk/<python-abi>` so changing the image's Python version
installs compatible wheels. The first run after an upgrade may reinstall the
SDK; existing caches remain available for their original interpreter.
These checks do not certify a live
Workflow invocation or a GPU task.

## Upstream compatibility and evaluation
Expand Down
52 changes: 35 additions & 17 deletions docker/eval-tools/gpu-asan/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,30 +1,48 @@
ARG BASE_IMAGE=lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260705@sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78
ARG BASE_IMAGE=lmsysorg/sglang:v0.5.20-rocm10-mi35x@sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69
FROM ${BASE_IMAGE}

LABEL org.opencontainers.image.title="AgentKernelArena ROCm GPU ASan tool runtime" \
org.opencontainers.image.base.digest="sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78" \
org.opencontainers.image.version="rocm-7.2.0-asan"
org.opencontainers.image.base.digest="sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69" \
org.opencontainers.image.version="rocm-10.0.0-asan"

COPY docker/eval-tools/gpu-asan/packages.sha256 /tmp/packages.sha256

RUN mkdir -p /tmp/gpu-asan-debs \
# Unpack checksum-locked ROCm 10 ASan packages into their own versioned prefix.
# Installing their metapackage would also mutate the base image's SDK selection.
RUN mkdir -p /tmp/gpu-asan-debs /tmp/gpu-asan-root \
&& cd /tmp/gpu-asan-debs \
&& apt-get update \
&& apt-get download \
rocm-core-asan=7.2.0.70200-43~22.04 \
comgr-asan=3.0.0.70200-43~22.04 \
hsa-rocr-asan=1.18.0.70200-43~22.04 \
hip-runtime-amd-asan=7.2.26015.70200-43~22.04 \
&& while read -r checksum package; do \
curl --fail --location --retry 3 -o "$package" \
"https://stable.repo.amd.com/rocm/core/packages-asan/ubuntu2404/pool/main/$package"; \
done < /tmp/packages.sha256 \
&& sha256sum --check /tmp/packages.sha256 \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends ./*.deb \
&& rm -rf /tmp/gpu-asan-debs /tmp/packages.sha256 /var/lib/apt/lists/*
&& for package in *.deb; do dpkg-deb --extract "$package" /tmp/gpu-asan-root; done \
&& mv /tmp/gpu-asan-root/opt/rocm/core-asan-10.0 /opt/rocm-asan-10.0 \
&& test -L /opt/rocm \
&& test -x /opt/rocm/bin/hipcc \
&& rm -rf /tmp/gpu-asan-debs /tmp/gpu-asan-root /tmp/packages.sha256

# PyTorch preloads these SDK libraries by absolute path. Point those paths at
# the instrumented copies too; LD_LIBRARY_PATH alone would load two LLVMs.
RUN /opt/venv/bin/python - <<'PY'
from pathlib import Path
import rocm_sdk

runtime = Path('/opt/rocm-asan-10.0/lib')
for name in ('amd_comgr', 'amdhip64'):
target = runtime / f'lib{name}.so'
assert target.is_file(), target
for path in rocm_sdk.find_libraries(name):
path.unlink()
path.symlink_to(target)
PY

COPY src/eval_tools /opt/aka-eval-tools/src/eval_tools

ENV AKA_EVAL_TOOL_FRAMEWORK_ROOT=/opt/aka-eval-tools \
PYTHONPATH=/opt/aka-eval-tools \
AKA_GPU_ASAN_RUNTIME_DIR=/opt/rocm-7.2.0/lib/asan \
AKA_GPU_ASAN_HIP_RUNTIME=/opt/rocm-7.2.0/lib/asan/libamdhip64.so \
AKA_GPU_ASAN_HOST_PRELOAD=/opt/rocm-7.2.0/lib/llvm/lib/clang/22/lib/linux/libclang_rt.asan-x86_64.so \
AKA_GPU_ASAN_HOST_LIB_DIR=/opt/rocm-7.2.0/lib/llvm/lib/clang/22/lib/linux \
AKA_GPU_ASAN_NORMAL_ROCM_LIB_DIR=/opt/rocm-7.2.0/lib
AKA_GPU_ASAN_RUNTIME_DIR=/opt/rocm-asan-10.0/lib \
AKA_GPU_ASAN_HIP_RUNTIME=/opt/rocm-asan-10.0/lib/libamdhip64.so \
AKA_GPU_ASAN_HOST_PRELOAD=/opt/rocm-asan-10.0/lib/llvm/lib/clang/23/lib/linux/libclang_rt.asan-x86_64.so \
AKA_GPU_ASAN_HOST_LIB_DIR=/opt/rocm-asan-10.0/lib/llvm/lib/clang/23/lib/linux \
AKA_GPU_ASAN_NORMAL_ROCM_LIB_DIR=/opt/rocm/lib
8 changes: 4 additions & 4 deletions docker/eval-tools/gpu-asan/packages.sha256
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
3bd5b98b3ae2cb8fbfd10c248682feb86d0f5136914e7a226b701340f9b90f83 rocm-core-asan_7.2.0.70200-43~22.04_amd64.deb
31118ea2dc79fe9d8c69ad7ff2176a2f1c822128ece604d4438b6c13a1aa3179 comgr-asan_3.0.0.70200-43~22.04_amd64.deb
c5d6e48846f6163b5c8d7168949e1c24dc41f69bd871705621b0972c2f1a03cc hsa-rocr-asan_1.18.0.70200-43~22.04_amd64.deb
fa56c2192f28adb022dc323d2836dfbe4f12211287575ea27978dc96b10a5300 hip-runtime-amd-asan_7.2.26015.70200-43~22.04_amd64.deb
030fc18229683369c77e7c9e124dd1b80114e7708a5ba8e86b985b031c2cda96 amdrocm-base-asan10.0_10.0.0-4_amd64.deb
3f45a0f26b4c9c01741fea8d42dec83f16dae4f754253d09fba32f1c32aec724 amdrocm-llvm-asan10.0_10.0.0-4_amd64.deb
c23ff5640c087457980d0d8e9edda49b3eb1c2114dd21fe437ffcf5e942fd15b amdrocm-runtime-asan10.0_10.0.0-4_amd64.deb
ff3c7cecf40d8133692116455266a7c0c02a2bdf5b695fdae746e47e50ff7147 amdrocm-sysdeps-asan10.0_10.0.0-4_amd64.deb
4 changes: 2 additions & 2 deletions docker/eval-tools/hip-fpsan/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
ARG BASE_IMAGE=lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260705@sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78
ARG BASE_IMAGE=lmsysorg/sglang:v0.5.20-rocm10-mi35x@sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69
FROM ${BASE_IMAGE}

ARG HIP_FPSAN_COMMIT=0ac9be8a1539a473ba21dfa686564c3be33c890e

LABEL org.opencontainers.image.title="AgentKernelArena HIP-FpSan tool runtime" \
org.opencontainers.image.base.digest="sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78" \
org.opencontainers.image.base.digest="sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69" \
org.opencontainers.image.revision="0ac9be8a1539a473ba21dfa686564c3be33c890e"

COPY src/eval_tools/probes/hip_fpsan_probe.hip /tmp/hip_fpsan_probe.hip
Expand Down
51 changes: 28 additions & 23 deletions docker/eval-tools/images.lock.yaml
Original file line number Diff line number Diff line change
@@ -1,36 +1,41 @@
schema_version: 1

# The scoring runtime remains unchanged. Every tool image is an immutable child
# of this verified MI355X/gfx950 image.
# Every tool runtime is a child of the same pinned MI355X/gfx950 scoring base.
# Native rocJITsu build dependencies have a separate builder lock below.
base:
gfx950:
reference: lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260705
digest: sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78
rocm: 7.2.0
reference: lmsysorg/sglang:v0.5.20-rocm10-mi35x
digest: sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69
rocm: 10.0.0

# The native C++ engines keep their GCC 13/Ubuntu 22.04 build toolchain.
# Only installed engines and their C++ runtime libraries enter the ROCm 10
# sidecars; HIP candidate/control compilation uses the new runtime SDK.
builders:
rocjitsu:
reference: lmsysorg/sglang-rocm@sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78

tools:
triton_fpsan:
triton:
version: 3.7.0+amd.rocm7.2.0.gitd0d77a509
sha256: 3a8acacdfb4723c8bb71844c427f4d3ce047658bf97fbdc23065c06205231687
triton_kernels:
version: 1.0.0+amd.rocm7.2.0.gitd0d77a509
sha256: df8b42ebf098767c0d31a3916913bd7557a623903aefa26602e7c6818c34b84a
version: 3.8.0+git4cff872c.rocm10.0.0
source: pinned_base_image

gpu_asan:
repository: https://stable.repo.amd.com/rocm/core/packages-asan/ubuntu2404
packages:
rocm-core-asan:
version: 7.2.0.70200-43~22.04
sha256: 3bd5b98b3ae2cb8fbfd10c248682feb86d0f5136914e7a226b701340f9b90f83
comgr-asan:
version: 3.0.0.70200-43~22.04
sha256: 31118ea2dc79fe9d8c69ad7ff2176a2f1c822128ece604d4438b6c13a1aa3179
hsa-rocr-asan:
version: 1.18.0.70200-43~22.04
sha256: c5d6e48846f6163b5c8d7168949e1c24dc41f69bd871705621b0972c2f1a03cc
hip-runtime-amd-asan:
version: 7.2.26015.70200-43~22.04
sha256: fa56c2192f28adb022dc323d2836dfbe4f12211287575ea27978dc96b10a5300
amdrocm-base-asan10.0:
version: 10.0.0-4
sha256: 030fc18229683369c77e7c9e124dd1b80114e7708a5ba8e86b985b031c2cda96
amdrocm-llvm-asan10.0:
version: 10.0.0-4
sha256: 3f45a0f26b4c9c01741fea8d42dec83f16dae4f754253d09fba32f1c32aec724
amdrocm-runtime-asan10.0:
version: 10.0.0-4
sha256: c23ff5640c087457980d0d8e9edda49b3eb1c2114dd21fe437ffcf5e942fd15b
amdrocm-sysdeps-asan10.0:
version: 10.0.0-4
sha256: ff3c7cecf40d8133692116455266a7c0c02a2bdf5b695fdae746e47e50ff7147

rocjitsu:
repository: https://github.com/ROCm/rocm-systems.git
Expand Down Expand Up @@ -70,5 +75,5 @@ tools:
image_probe: src/eval_tools/probes/hip_fpsan_probe.hip

verification:
gfx950: verified
gfx950: qualification_pending
gfx942: unverified
16 changes: 10 additions & 6 deletions docker/eval-tools/rocjitsu-sanitizers/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
ARG BASE_IMAGE=lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260705@sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78
FROM ${BASE_IMAGE} AS builder
ARG BASE_IMAGE=lmsysorg/sglang:v0.5.20-rocm10-mi35x@sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69
# Keep the native engine build toolchain pinned independently of its runtime.
ARG BUILDER_IMAGE=lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260705@sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78
FROM ${BUILDER_IMAGE} AS builder
ARG BUILD_JOBS=8

ARG ROCJITSU_SANITIZERS_COMMIT=ed35c0b54547c98bab359c8732529d9f5e8fd1ae
ARG GOOGLETEST_COMMIT=b514bdc898e2951020cbdca1304b75f5950d1f59
Expand Down Expand Up @@ -45,7 +48,7 @@ RUN curl -sSfL -o /tmp/zstd.tar.gz \
-DZSTD_BUILD_STATIC=ON \
-DZSTD_BUILD_PROGRAMS=OFF \
-DZSTD_BUILD_TESTS=OFF \
&& cmake --build /build/zstd --parallel \
&& cmake --build /build/zstd --parallel "${BUILD_JOBS}" \
&& cmake --install /build/zstd \
&& rm -f /tmp/zstd.tar.gz

Expand All @@ -58,7 +61,7 @@ RUN cmake -S /src/rocm-systems/emulation/rocjitsu -B /build/rocjitsu -G Ninja \
-DFETCHCONTENT_SOURCE_DIR_GOOGLETEST=/src/googletest \
-DFETCHCONTENT_SOURCE_DIR_FLATBUFFERS=/src/flatbuffers \
-DBUILD_TESTING=OFF \
&& cmake --build /build/rocjitsu --parallel \
&& cmake --build /build/rocjitsu --parallel "${BUILD_JOBS}" \
--target rj_waitcheck rocjitsu_waitcheck rocjitsu_dbi_hooks \
&& mkdir -p /opt/rocjitsu/include /opt/rocjitsu/lib \
&& cp -a /src/rocm-systems/emulation/rocjitsu/lib/rocjitsu/include/rocjitsu \
Expand All @@ -81,15 +84,15 @@ RUN g++-13 -std=c++17 -O2 -Wall -Wextra -Werror \

FROM ${BASE_IMAGE} AS runtime-common

LABEL org.opencontainers.image.base.digest="sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78" \
LABEL org.opencontainers.image.base.digest="sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69" \
org.opencontainers.image.revision="ed35c0b54547c98bab359c8732529d9f5e8fd1ae"

COPY --from=builder /opt/rocjitsu /opt/rocjitsu
COPY src/eval_tools /opt/aka-eval-tools/src/eval_tools

ENV AKA_EVAL_TOOL_FRAMEWORK_ROOT=/opt/aka-eval-tools \
PYTHONPATH=/opt/aka-eval-tools \
LD_LIBRARY_PATH=/opt/rocjitsu/lib:/opt/rocjitsu/runtime \
LD_LIBRARY_PATH=/opt/rocjitsu/lib:/opt/rocjitsu/runtime:${LD_LIBRARY_PATH} \
PATH=/opt/rocjitsu/bin:${PATH} \
AKA_ROCJITSU_SANITIZERS_COMMIT=ed35c0b54547c98bab359c8732529d9f5e8fd1ae

Expand All @@ -112,6 +115,7 @@ LABEL org.opencontainers.image.title="AgentKernelArena rocJITsu ConSan runtime"
ENV AKA_CONSAN_HOOK=/opt/rocjitsu/lib/librocjitsu_dbi_hooks.so

RUN test -f /opt/rocjitsu/lib/librocjitsu_dbi_hooks.so \
&& /opt/venv/bin/python -c "import ctypes; ctypes.CDLL('libamdhip64.so.7')" \
&& /opt/venv/bin/python -I \
/opt/aka-eval-tools/src/eval_tools/adapters/consan_entrypoint.py \
--help >/dev/null
11 changes: 7 additions & 4 deletions docker/eval-tools/rocjitsu/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
ARG BASE_IMAGE=lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260705@sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78
FROM ${BASE_IMAGE} AS builder
ARG BASE_IMAGE=lmsysorg/sglang:v0.5.20-rocm10-mi35x@sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69
# Keep the native engine build toolchain pinned independently of its runtime.
ARG BUILDER_IMAGE=lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260705@sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78
FROM ${BUILDER_IMAGE} AS builder
ARG BUILD_JOBS=8

ARG ROCJITSU_COMMIT=0bf561a0d8a4a6b88954f2c46bd3a50871cda140
ARG GOOGLETEST_COMMIT=b514bdc898e2951020cbdca1304b75f5950d1f59
Expand Down Expand Up @@ -40,7 +43,7 @@ RUN cmake -S /src/rocm-systems/emulation/rocjitsu -B /build/rocjitsu -G Ninja \
-DCMAKE_HIP_ARCHITECTURES=gfx950 \
-DBUILD_TESTING=ON \
-DRJ_INSTALL_TESTS=OFF \
&& cmake --build /build/rocjitsu --parallel \
&& cmake --build /build/rocjitsu --parallel "${BUILD_JOBS}" \
&& cmake --install /build/rocjitsu \
&& mkdir -p /opt/rocjitsu/runtime \
&& cp -a /usr/lib/x86_64-linux-gnu/libstdc++.so.6* /opt/rocjitsu/runtime/ \
Expand All @@ -49,7 +52,7 @@ RUN cmake -S /src/rocm-systems/emulation/rocjitsu -B /build/rocjitsu -G Ninja \
FROM ${BASE_IMAGE}

LABEL org.opencontainers.image.title="AgentKernelArena rocJITsu tool runtime" \
org.opencontainers.image.base.digest="sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78" \
org.opencontainers.image.base.digest="sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69" \
org.opencontainers.image.revision="0bf561a0d8a4a6b88954f2c46bd3a50871cda140"

COPY --from=builder /opt/rocjitsu /opt/rocjitsu
Expand Down
19 changes: 7 additions & 12 deletions docker/eval-tools/triton-fpsan/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,21 +1,16 @@
ARG BASE_IMAGE=lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260705@sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78
ARG BASE_IMAGE=lmsysorg/sglang:v0.5.20-rocm10-mi35x@sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69
FROM ${BASE_IMAGE}

LABEL org.opencontainers.image.title="AgentKernelArena Triton FpSan tool runtime" \
org.opencontainers.image.base.digest="sha256:b435b508b5aa696abb25c909341ce73e41574c4271cf716bed72418dcea86b78" \
org.opencontainers.image.version="triton-3.7.0+amd.rocm7.2.0.gitd0d77a509"
org.opencontainers.image.base.digest="sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69" \
org.opencontainers.image.version="triton-3.8.0+git4cff872c.rocm10.0.0"

COPY docker/eval-tools/triton-fpsan/requirements.lock /tmp/eval-tool-requirements.lock

RUN python -m pip uninstall -y \
triton pytorch-triton pytorch-triton-rocm triton-rocm amd-triton triton-kernels \
&& python -m pip install --no-cache-dir --no-deps --require-hashes \
-r /tmp/eval-tool-requirements.lock \
&& python -c "import importlib.metadata as m, triton, triton_kernels; assert m.version('triton') == '3.7.0+amd.rocm7.2.0.gitd0d77a509'; assert m.version('triton-kernels') == '1.0.0+amd.rocm7.2.0.gitd0d77a509'" \
&& rm -f /tmp/eval-tool-requirements.lock
# The digest-pinned ROCm 10 base already supplies FpSan-enabled Triton.
# Do not replace it with a ROCm 7.2 wheel or infer support from import success.
RUN python -c "import importlib.metadata as m; assert m.version('triton') == '3.8.0+git4cff872c.rocm10.0.0'"

COPY src/eval_tools /opt/aka-eval-tools/src/eval_tools

ENV AKA_EVAL_TOOL_FRAMEWORK_ROOT=/opt/aka-eval-tools \
PYTHONPATH=/opt/aka-eval-tools \
AKA_TRITON_FPSAN_VERSION=3.7.0+amd.rocm7.2.0.gitd0d77a509
AKA_TRITON_FPSAN_VERSION=3.8.0+git4cff872c.rocm10.0.0
4 changes: 0 additions & 4 deletions docker/eval-tools/triton-fpsan/requirements.lock

This file was deleted.

Loading
Loading