Skip to content

bug: COZE_WORKFLOW_ID required at runtime but absent from coze-studio credential gate #6

@Infinite-Ng

Description

@Infinite-Ng

Summary

coze_studio/run.sh requires COZE_WORKFLOW_ID via bash parameter expansion, but coze_studio/spec.py only declares COZE_INSTANCE_URL and COZE_API_TOKEN in requires_credentials. A run with the two declared vars set — but COZE_WORKFLOW_ID missing — passes the matrix credential gate, launches the container, and then fails at runtime.

Root cause

coze_studio/spec.py (line 23):

requires_credentials=("COZE_INSTANCE_URL", "COZE_API_TOKEN"),

coze_studio/run.sh (line 7):

: "${COZE_WORKFLOW_ID:?COZE_WORKFLOW_ID must be set (Coze workflow that drives the browser)}"

COZE_WORKFLOW_ID is enforced in the shell but invisible to the Python credential gate.

Impact

  • harnessbench harnesses reports coze-studio as ready (credentials present) when only the two declared vars are set.
  • harnessbench matrix marks the run as eligible.
  • The container starts, then immediately exits non-zero with a bash error.
  • The leaderboard receives no result (silent failure).

Suggested fix

Add COZE_WORKFLOW_ID to requires_credentials in coze_studio/spec.py:

requires_credentials=("COZE_INSTANCE_URL", "COZE_API_TOKEN", "COZE_WORKFLOW_ID"),

Also update docs/cloud-harness-setup.md to document all three required variables.

Affected files

  • src/harnessbench/harnesses/coze_studio/spec.py:23
  • src/harnessbench/harnesses/coze_studio/run.sh:7
  • docs/cloud-harness-setup.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions