Repository navigation
Add RustPython patches for Ruff 0.16.10 - #6
Open
youknowone wants to merge 20 commits into
Open
youknowone wants to merge 20 commits into
youknowone wants to merge 20 commits into
Conversation
Expose RustPython AST metadata fields, apply the RustPython package patch, and make CI usable on fork runners. Assisted-by: Codex:GPT-5 Assisted-by: Grok:grok-4.6 Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Codex:GPT-5
Only the five rustpython-ruff_* crates are published. ruff_python_ast drops its optional ruff_cache, ruff_macros and salsa dependencies along with the cache and salsa features; the serde feature stays but no longer pulls ruff_cache. Workspace crates that requested cache or salsa stop doing so. ruff_annotate_snippets loses its version requirement so cargo drops it from the published parser dev-dependencies. Assisted-by: Claude Code:claude-fable-5-1 Assisted-by: Claude Code:claude-opus-5-5
The "Multiple exception types must be parenthesized when using `as`" error is now added after `as NAME` is parsed and only when the clause continues with `:`. Its range starts at the first exception type and ends at the start of the `:` token, covering `as NAME`. When the name or the `:` is missing, only the errors for those are reported. Assisted-by: Claude Code:claude-opus-5-5
Error messages that RustPython rewrote by matching their text now come out of the parser in their final form: - `Display` of `ParseErrorType`, `LexicalErrorType` and the f-string and t-string error types produces the CPython wording, e.g. `invalid syntax`, `expected ':'`, `positional argument follows keyword argument`, `unterminated f-string literal`, with single quotes instead of backticks in f-string and t-string messages. - The list recovery messages and `Expected an identifier ...` become `invalid syntax`, and the `OtherError` messages for mixed bytes literals, keyword patterns, `not`, trailing commas, `except ... as`, `/` and `*` parameters, and `try` without handlers use the CPython wording. - All other messages, including `UnsupportedSyntaxError` messages, start with a lowercase letter, except `Type parameter list cannot be empty`, `Expected one or more names after 'import'` and `Generator expression must be parenthesized`. Snapshots are updated for the new texts. Assisted-by: Grok:grok-4.6 Assisted-by: Claude Code:claude-opus-5-5
`InvalidStarredExpressionUsage` and `InvalidStarPatternUsage` errors now cover only the first character of the starred expression or star pattern, and a bytes literal with a non-ASCII character is reported over the whole literal instead of the offending character. Assisted-by: Claude Code:claude-opus-5-5
Indentation compares columns with a tab size of 8 and checks the character count only for consistency, so tabs and spaces that agree at tab size 8 are accepted. Inconsistent mixing is reported as `LexicalErrorType::TabError`, and nesting past 99 levels as `LexicalErrorType::TooDeepIndentation`; both are reported at the start of the line. An unmatched dedent is reported at the end of the line, an unexpected indent at the last character of the indentation, and a line continuation error after the backslash. The parser no longer adds an error while the current token is the `Unknown` token of a lexer error. `ParseErrorType::UnexpectedIndentation` now displays as "unexpected indent". `nested_def_blocks_grow_stack` expects the depth error. Assisted-by: Claude Code:claude-opus-5-5
`LexicalErrorType::UnclosedStringError` now carries whether the string is triple-quoted, whether it contains a backslash followed by its quote character, and the line at which the lexer reached the end of the line or the source. The f/t-string `UnterminatedString` and `UnterminatedTripleQuotedString` errors carry the detected line too, and the messages include it. These errors are reported as an empty range at the start of the string, including its prefix, for which `InterpolatedStringContext` now records the start offset. Assisted-by: Claude Code:claude-opus-5-5
Add lexer error kinds for unmatched, mismatched and unclosed brackets and for brackets nested more than 200 deep. An unclosed bracket is reported at the innermost opening bracket instead of as an unexpected EOF, and an unterminated replacement field reports its open bracket. Report unrecognized non-ASCII and control characters with their code points. After parsing, tokenize the source again and move the first tokenizer error in front of the parser's first error when tokenizing the rest of the source would report it. `UnclosedBracket::incomplete` tells whether the parser read to the end of the source with the bracket open. Assisted-by: Claude Code:claude-opus-5-5
Lex number literals with the tokenizer's checks: an underscore or exponent sign without a following digit, a radix prefix without digits, a digit outside an octal or binary literal, leading zeros in a decimal integer, and a number directly followed by a name other than a keyword that can follow it. Report these as `InvalidNumberLiteral`, `InvalidDigit` and `LeadingZerosInDecimalInteger`. Report a string prefix that combines incompatible prefixes, such as `ub''`, as `IncompatibleStringPrefixes` over the prefix. Report an unrecognized character that is not printable by its general category as a non-printable character. Assisted-by: Claude Code:claude-opus-5-5
Add `ParseErrorType::ExpectedIndentedBlock { clause, line }`, where
`line` is the line of the clause's header keyword, and `BlockClause`
for the clause. `except*` is reported apart from `except`. Share the
line counting between this error and tokenizer error prioritization.
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Grok CLI:grok-4.6
Add `ExpressionKind`, the kind of an expression as syntax errors name it, and carry it in the assignment, named assignment, augmented assignment and delete target errors. Report invalid targets after `as` in imports, patterns and `except` clauses with their kind. In an assignment, report the target before `=` as a possible comparison when the expression after `=` is not itself assigned to, and a bare `yield` target with its own error. Target errors come before errors in the expressions after them. Check `for` targets apart from assignment targets: a comparison is not an invalid target there, and the token after it is reported instead. End an unparenthesized tuple at an augmented assignment operator, and report an unparenthesized tuple annotation target at its first element. Assisted-by: Claude Code:claude-opus-5-5
…essages - Parameters: missing default values, `**kwargs` defaults, repeated `*`, bare `*` ranges, parenthesized parameters, missing comma between `/` and `*`, and bounds or constraints on TypeVarTuple and ParamSpec. - Arguments: invalid keyword names, `**x=` and `*x=` assignments, missing keyword values, keyword values followed by `for`, and the positions of ordering errors. - Stars: `*` without an expression at the start of a display, argument or slice, and parenthesized `*x` and `**x`. - Missing commas between expressions inside brackets. - Dictionaries: missing `:` after a later key (new `ExpectedColonAfterDictionaryKey`), missing values and starred values. Assisted-by: Claude Code:claude-opus-5-5
- `=` in a named expression position: "invalid syntax. Maybe you meant '==' or ':=' instead of '='?" or "cannot assign to X here". - Expressions between two string literals. - `pass`/`break`/`continue` before `if` and a statement after `else` in an `if` expression. - Missing `in` after comprehension targets, and unparenthesized tuple comprehension targets in lists and sets. - Invalid `:=` targets at statement level. Assisted-by: Claude Code:claude-opus-5-5
- `elif` after `else`: "'elif' block follows an 'else' block". - `import a from b`: "Did you mean to use 'from ... import ...' instead?". - Mixed `except` and `except*` is reported at the `except` keyword and its star. - `**_` in a mapping pattern is "invalid syntax". Assisted-by: Claude Code:claude-opus-5-5
…ages Report escape sequence errors in string literals as unicode escape decoder errors with their decoder positions, at the closing quote for f-strings and t-strings. Report the f-string and t-string replacement field rules, a quote of the enclosing f-string in a replacement field as a missing `}`, a bracket closing a replacement field brace as unmatched, lambdas whose `:` starts a format spec, and `print`/`exec` statements missing parentheses. Assisted-by: Claude Code:claude-opus-5-5
Add `ParseErrorType::ExpectedIdentifier` for a missing identifier. Treat only missing expressions and identifiers as making an expression incomplete, require a complete prefix of the second expression for the missing comma error, keep string decode errors read before an unknown token, and report them ahead of an unclosed bracket. Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Claude Code:claude-opus-5-5
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Member
Author
|
CI failures come from f2143fa ("Strip unpublished dependencies for the 0.16.10 crates.io release"), not from the parser changes. That commit removes the The five published crates were checked separately at c3c644c:
— commented by Claude Code:claude-opus-5-5 |
Return early from `prioritize_tokenizer_error` when there are no errors instead of re-lexing the source, and track seen string prefixes in a bitmask instead of allocating a `Vec` for every identifier. Assisted-by: Claude Code:claude-opus-5-5
`def f(*args: *b = (1,))` is invalid syntax, not a var-positional parameter with a default value. Assisted-by: Claude Code:claude-opus-5-5
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RustPython patches rebased onto Ruff 0.16.10, plus parser changes that report syntax errors with CPython's messages and ranges so RustPython no longer rewrites parser messages or rescans the source.
invalid_*grammar rules: indented blocks, binding targets, parameters and arguments, missing commas, dictionary items, comprehensions,elif/except/importstatements, f-string and t-string replacement fields,print/execstatementsunicodeescapedecoder errorsThe base branch
upstream-0.16.10is Ruff's0.16.10tag, so the diff shows only the RustPython patches.mainis still on 0.16.5.Validation
cargo test -p rustpython-ruff_python_parsercargo clippy -p rustpython-ruff_python_parser --all-targets -- -D warningsAI assistance
Claude Code:claude-opus-5-5 assisted with porting the patches, implementing the parser changes, running validation, and preparing this pull request.
🤖 Generated with Claude Code