Skip to content

ci: pin ruff so check-format stops failing on main - #2686

Merged
kevinmessiaen merged 2 commits into
Giskard-AI:mainfrom
BILLKISHORE:fix/ci-format-drift
Aug 3, 2026
Merged

ci: pin ruff so check-format stops failing on main#2686
kevinmessiaen merged 2 commits into
Giskard-AI:mainfrom
BILLKISHORE:fix/ci-format-drift

Conversation

@BILLKISHORE

@BILLKISHORE BILLKISHORE commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

What

make check-format fails on a clean main, so the lint job is red on new PRs. From the run on #2680:

uv run ruff format --check .
unformatted: File would be reformatted
5 files would be reformatted, 291 files already formatted
make: *** [Makefile:101: check-format] Error 1

No commit caused this. install-tools runs uv tool install ruff unpinned, and ruff 0.16 promoted markdown code-block formatting out of preview. CI picked up the new default on its next run, and five docs files that predate it started failing.

Bisecting the tool rather than the tree:

ruff markdown formatting check-format on clean main
0.14.0 preview only pass
0.15.0 preview only pass
0.16.1 on by default fail

Fix

Two commits:

  1. ruff format over the five affected files (README.md, the three package READMEs, libs/giskard-llm/docs/design.md). Formatting only: line wrapping, blank lines between top-level defs, quote normalisation, trailing commas. No prose or code semantics changed.
  2. Pin RUFF_VERSION := 0.16.1 in install-tools, so the next ruff release can't turn make check red on a tree nobody touched. Same reasoning as the existing LICENSECHECK_VERSION pin.

I did consider pinning to 0.15 instead, which also makes main green with a one-line diff since 0.15 ignores markdown entirely. Freezing the toolchain to avoid a formatting change you'll take eventually didn't seem like the right call, especially as the same reformat is already riding along in a number of open PRs.

Testing

make check exits 0 on this branch and fails on main. make test-unit: 37 / 211 / 117 / 757 / 155 passed across core, llm, agents, checks and scan.

Note

The docs reformat already appears as a side effect in several open PRs (#2658 among them, byte-identical to this one). Landing it standalone lets those diffs shrink to just their real changes.

Type of Change

  • 馃 Improvement (non-breaking change which improves an existing feature)

Disclosure: drafted with AI assistance (Claude Code). I reproduced the failure, bisected the ruff versions, and ran the checks myself before submitting.

Recent ruff releases format Python inside markdown code blocks. These files
predate that, so `make check-format` currently fails on an unchanged main.
An unpinned install lets a new ruff release turn `make check` red on an
unchanged tree, same reasoning as the existing licensecheck pin.

@kevinmessiaen kevinmessiaen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing the CI !

@kevinmessiaen
kevinmessiaen enabled auto-merge (squash) August 3, 2026 01:00
@kevinmessiaen
kevinmessiaen merged commit cd06fe7 into Giskard-AI:main Aug 3, 2026
29 of 50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants