-
Notifications
You must be signed in to change notification settings - Fork 1
feat!(cli): Switch to cyclopts for CI #55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR migrates the CLI framework from Typer to Cyclopts 4.0, which brings improved performance, faster tab completion (implemented in shell scripts rather than Python), and more flexibility. The migration introduces a breaking change in the run command's visualization flag (-v), which now accepts arguments directly instead of using a separate -s flag.
Key changes:
- Replaced Typer with Cyclopts 4.0 for CLI implementation with custom help formatting
- Refactored visualization options from
VisArgsdataclass toVisOptionsenum with updated argument handling - Consolidated type validation and completion logic into new converter functions
Reviewed Changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| pyproject.toml | Updated dependencies: added cyclopts 4.0, removed typer, upgraded rapidfuzz |
| python/evalio/cli/init.py | Complete rewrite to use Cyclopts App with custom help formatting and command registration |
| python/evalio/cli/types.py | New file containing type converters and validators for datasets and pipelines |
| python/evalio/cli/completions.py | File deleted - functionality moved to types.py |
| python/evalio/cli/ls.py | Migrated from Typer to Cyclopts, changed Kind enum to string literal |
| python/evalio/cli/dataset_manager.py | Converted commands to use Cyclopts parameters, renamed force flag to confirm |
| python/evalio/cli/run.py | Refactored to use Cyclopts with new visualization argument handling |
| python/evalio/cli/stats.py | Converted to Cyclopts with reorganized parameter groups and updated documentation |
| python/evalio/rerun.py | Changed VisArgs from dataclass to VisOptions enum, updated type hints |
| python/evalio/stats.py | Removed extra blank line |
| python/evalio/datasets/base.py | Fixed typo in warning message (set_data_dir → set_DATA_DIR) |
| tests/test_cli_ls.py | Updated test to use string literals instead of Kind enum |
| tests/test_cli_data.py | Updated tests to use app.meta() instead of direct function calls |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ), | ||
| w_meters: Ann[Optional[list[float]], Par(group=mg)] = None, | ||
| w_seconds: Ann[Optional[list[float]], Par(group=mg)] = None, | ||
| metric: Ann[stats.MetricKind, Par(alias="-m", group=mg)] = stats.MetricKind.sse, |
Copilot
AI
Oct 27, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] The metric parameter now uses alias -m but originally had no short alias (only --metric). While this adds convenience, it should be documented as it changes the CLI interface.
With the recent release of cyclopts 4.0 with tab completion, we can now switch from typer to cyclopts.
Cyclopts is significantly faster, easier to write, as well being more flexible. Further, the new completion is written fully in bash/zsh/fish rather than typer's that simply calls python under the hood. This means LOTS faster completion is available!
Once mkdocs plugin lands, this should be good to merge! Been waiting for a long time to do this:)
There are a few small breaking changes - see
evalio run -vcommand that now accepts arguments instead of a separate-sflag. I don't think there's too many others though!