Skip to content

Conversation

@contagon
Copy link
Owner

With the recent release of cyclopts 4.0 with tab completion, we can now switch from typer to cyclopts.

Cyclopts is significantly faster, easier to write, as well being more flexible. Further, the new completion is written fully in bash/zsh/fish rather than typer's that simply calls python under the hood. This means LOTS faster completion is available!

Once mkdocs plugin lands, this should be good to merge! Been waiting for a long time to do this:)

There are a few small breaking changes - see evalio run -v command that now accepts arguments instead of a separate -s flag. I don't think there's too many others though!

@contagon contagon requested a review from Copilot October 27, 2025 10:03
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR migrates the CLI framework from Typer to Cyclopts 4.0, which brings improved performance, faster tab completion (implemented in shell scripts rather than Python), and more flexibility. The migration introduces a breaking change in the run command's visualization flag (-v), which now accepts arguments directly instead of using a separate -s flag.

Key changes:

  • Replaced Typer with Cyclopts 4.0 for CLI implementation with custom help formatting
  • Refactored visualization options from VisArgs dataclass to VisOptions enum with updated argument handling
  • Consolidated type validation and completion logic into new converter functions

Reviewed Changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pyproject.toml Updated dependencies: added cyclopts 4.0, removed typer, upgraded rapidfuzz
python/evalio/cli/init.py Complete rewrite to use Cyclopts App with custom help formatting and command registration
python/evalio/cli/types.py New file containing type converters and validators for datasets and pipelines
python/evalio/cli/completions.py File deleted - functionality moved to types.py
python/evalio/cli/ls.py Migrated from Typer to Cyclopts, changed Kind enum to string literal
python/evalio/cli/dataset_manager.py Converted commands to use Cyclopts parameters, renamed force flag to confirm
python/evalio/cli/run.py Refactored to use Cyclopts with new visualization argument handling
python/evalio/cli/stats.py Converted to Cyclopts with reorganized parameter groups and updated documentation
python/evalio/rerun.py Changed VisArgs from dataclass to VisOptions enum, updated type hints
python/evalio/stats.py Removed extra blank line
python/evalio/datasets/base.py Fixed typo in warning message (set_data_dir → set_DATA_DIR)
tests/test_cli_ls.py Updated test to use string literals instead of Kind enum
tests/test_cli_data.py Updated tests to use app.meta() instead of direct function calls

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

),
w_meters: Ann[Optional[list[float]], Par(group=mg)] = None,
w_seconds: Ann[Optional[list[float]], Par(group=mg)] = None,
metric: Ann[stats.MetricKind, Par(alias="-m", group=mg)] = stats.MetricKind.sse,
Copy link

Copilot AI Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The metric parameter now uses alias -m but originally had no short alias (only --metric). While this adds convenience, it should be documented as it changes the CLI interface.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants