Replace the eval tasks with four multi-turn tasks #148
AshishKumar4workshop-evals-pr.yml
on: pull_request
Verify pull request
3s
Matrix: evals
Matrix: assemble
Review eval trajectories
1m 1s
Annotations
3 errors, 8 warnings, and 10 notices
|
Eval candidate change-calendar
Process completed with exit code 1.
|
|
Eval candidate worker-logs
Process completed with exit code 1.
|
|
Eval candidate chess
Process completed with exit code 1.
|
|
Eval candidate change-calendar
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Eval candidate incident-desk
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Eval candidate worker-logs
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Eval candidate chess
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Assemble candidate
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Assemble baseline
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Compare evals
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0, actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Review eval trajectories
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Verify pull request
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
|
|
Plan eval runs
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
|
|
Eval candidate change-calendar
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
|
|
Eval candidate incident-desk
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
|
|
Eval candidate worker-logs
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
|
|
Eval candidate chess
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
|
|
Assemble candidate
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
|
|
Assemble baseline
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
|
|
Compare evals
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
|
|
Review eval trajectories
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
workshop-evals-baseline
|
225 KB |
sha256:d74ed60d8838f672bb9fb9cf31c99bfe0875cecdf97a3bd444b0f1037ac473cb
|
|
|
workshop-evals-candidate
|
1.14 MB |
sha256:fb637dc37594cc29b8b95d69594a2d126adb7367be21c7862324ba103b31ef29
|
|
|
workshop-evals-candidate-task-change-calendar
|
204 KB |
sha256:ea589ff4d4a3f3a2668613e80dbc912ef541c3e1d2f127fc386faee98f87514b
|
|
|
workshop-evals-candidate-task-chess
|
346 KB |
sha256:f35fe5badfb65419d1e4e057fc52d5aafdd2f9149cf7811fcfc92918c5a8a0ac
|
|
|
workshop-evals-candidate-task-incident-desk
|
217 KB |
sha256:6d8cabe51764d4c2f795f6499197df9430bfd92a453b18be1a48d557ffa46472
|
|
|
workshop-evals-candidate-task-worker-logs
|
236 KB |
sha256:6d6b1b41de159f2e468ed3484a94976c11be231099c5ac0c52bd2523d9dd2e50
|
|
|
workshop-evals-comparison
|
1.06 KB |
sha256:fade8373276e90b1bd604c5c4e758de627d74956983200afe9226ded5fff9b5d
|
|