Summary
coze_studio/run.sh requires COZE_WORKFLOW_ID via bash parameter expansion, but coze_studio/spec.py only declares COZE_INSTANCE_URL and COZE_API_TOKEN in requires_credentials. A run with the two declared vars set — but COZE_WORKFLOW_ID missing — passes the matrix credential gate, launches the container, and then fails at runtime.
Root cause
coze_studio/spec.py (line 23):
requires_credentials=("COZE_INSTANCE_URL", "COZE_API_TOKEN"),
coze_studio/run.sh (line 7):
: "${COZE_WORKFLOW_ID:?COZE_WORKFLOW_ID must be set (Coze workflow that drives the browser)}"
COZE_WORKFLOW_ID is enforced in the shell but invisible to the Python credential gate.
Impact
harnessbench harnesses reports coze-studio as ready (credentials present) when only the two declared vars are set.
harnessbench matrix marks the run as eligible.
- The container starts, then immediately exits non-zero with a bash error.
- The leaderboard receives no result (silent failure).
Suggested fix
Add COZE_WORKFLOW_ID to requires_credentials in coze_studio/spec.py:
requires_credentials=("COZE_INSTANCE_URL", "COZE_API_TOKEN", "COZE_WORKFLOW_ID"),
Also update docs/cloud-harness-setup.md to document all three required variables.
Affected files
src/harnessbench/harnesses/coze_studio/spec.py:23
src/harnessbench/harnesses/coze_studio/run.sh:7
docs/cloud-harness-setup.md
Summary
coze_studio/run.shrequiresCOZE_WORKFLOW_IDvia bash parameter expansion, butcoze_studio/spec.pyonly declaresCOZE_INSTANCE_URLandCOZE_API_TOKENinrequires_credentials. A run with the two declared vars set — butCOZE_WORKFLOW_IDmissing — passes the matrix credential gate, launches the container, and then fails at runtime.Root cause
coze_studio/spec.py(line 23):coze_studio/run.sh(line 7):COZE_WORKFLOW_IDis enforced in the shell but invisible to the Python credential gate.Impact
harnessbench harnessesreportscoze-studioasready (credentials present)when only the two declared vars are set.harnessbench matrixmarks the run aseligible.Suggested fix
Add
COZE_WORKFLOW_IDtorequires_credentialsincoze_studio/spec.py:Also update
docs/cloud-harness-setup.mdto document all three required variables.Affected files
src/harnessbench/harnesses/coze_studio/spec.py:23src/harnessbench/harnesses/coze_studio/run.sh:7docs/cloud-harness-setup.md