Conversation
added 6 commits
September 28, 2026 13:46
| @@ -0,0 +1,282 @@ | |||
| #!/usr/bin/env python3 | |||
Collaborator
There was a problem hiding this comment.
https://github.com/amd/skills/actions/runs/36598752509
Looks like failing tests are not showing any logs. Those are potentially part of a log file now, but this is not great for visibility. Is there any way in which we can show the logs here as they used to show on other machines?
Comment on lines
+130
to
+131
| uv tool install --system-certs git+https://github.com/amd/skillscope@v0.1.2 | ||
| SKILLSCOPE_SHA="$(sed -n 's/^ SKILLSCOPE_SHA: //p' .github/workflows/evals.yml)" | ||
| uv tool install --system-certs "git+https://github.com/amd/skillscope@$SKILLSCOPE_SHA" |
Collaborator
There was a problem hiding this comment.
Recommend simply pinning it to a version explicitly
|
|
||
| env: | ||
| SKILLSCOPE_REPOSITORY: amd/skillscope | ||
| SKILLSCOPE_SHA: 0f734b2dd344cd72b5e016ad6c3b1ac2ce36f6f3 |
Collaborator
There was a problem hiding this comment.
We shouldn't stick with sha versions. Only releases like it was before.
added 5 commits
September 29, 2026 16:03
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Run selected Linux and Windows Strix behavioral evals through OrchestrAI using the PR head commit and existing
run_behavioralgate. Preserveevals / results, routing, and Instinct jobs.Show bounded, sanitized grader output plus case/expectation totals, model/effort, and unmet-expectation details in GitHub. Keep full logs in ReportPortal. Wait for every requested session and require confirmed parent termination.
Validation: 90 OrchestrAI tests (161 repository unit tests), lint, and repository checks pass. Replaying the reported lemonade-router-builder output reproduces 2/3 cases and 20/21 expectations with the keyword-router failure explanation. New-head CI is pending.