Skip to content

Improve Python 3.15 parser diagnostics and recovery - #7

Draft
youknowdot wants to merge 7 commits into
RustPython:rustpython-0.17.5from
youknowdot:pep798-unpacking-operands-315
Draft

youknowdot wants to merge 7 commits into
RustPython:rustpython-0.17.5from
youknowdot:pep798-unpacking-operands-315

Conversation

@youknowdot

@youknowdot youknowdot commented Oct 19, 2026 •

Copy link
Copy Markdown

Summary

Improve selected CPython 3.15 parser diagnostics and error recovery on the RustPython 0.17.5 parser branch.

  • Unpacking diagnostics distinguish invalid positions in comprehensions, dictionary keys and values, and conditional branches. Recovery preserves valid AST ranges, including else **value.
  • Deferred conditional-unpacking errors preserve encounter order when an otherwise valid call expression is followed by a parse failure. Speculative diagnostic parsing restores parser recovery state.
  • Parameter terminology and repeated-star wording match CPython.
  • Standalone case clauses, assertion assignment/walrus syntax, mapping-rest ordering, and misplaced lazy imports receive specific diagnostics.
  • Unclosed multiline f-string/t-string replacement fields identify the opening brace's line.
  • Missing-comma hints after multiline literal concatenation point to the final literal line without changing original-expression error precedence.

Scope

This PR does not expand accepted PEP 798 operand grammar. General unparenthesized operands in dictionary comprehensions and parenthesized starred generators remain upstream work: astral-sh#30406 and astral-sh#30418. The current upstream PR addresses dictionary comprehensions, not the generator example reported in the issue. This diagnostic patch is not a substitute for those syntax fixes.

Parenthesized operands and already-supported call forms remain covered. Tests for deferred and speculative errors use grammar already accepted by the base parser. Existing upstream fixtures are retained; grammar-expansion-only fixtures introduced on this branch are excluded rather than recording the known upstream rejection as correct behavior.

The diagnostic implementation references CPython 3.15.0 at d220409a13b77de7a6c30abe16ea90db01528edc, including its grammar, replacement-field tests, and syntax tests. No new Cargo feature or diagnostic version gate is introduced. Complete CPython diagnostic parity is not claimed.

This builds on rustpython-0.17.5 at 043bd2bc4898a5348f66332c5cdd19de4fbecbe1. RustPython consumers still need the resulting revision and separate compiler/VM integration. No dependency pin or crates.io release is changed here.

Validation

At cd0390f5cb46db24d4d64d133c2155a05727af54:

  • All-feature parser tests: 257 unit tests, 645 fixture tests, one inline-generation test and ten doctests passed, with snapshot updates disabled for final verification.
  • Parser-only strict all-target/all-feature Clippy passed with --no-deps.
  • Normal changed-file hooks passed.
  • A focused scope check confirms unparenthesized dictionary/generator operand expansion remains absent, parenthesized forms and supported calls remain accepted, and conditional-branch unpacking retains its specific diagnostic and repaired recovery AST.

Dependency-wide strict Clippy remains blocked by nine existing disabled cache/salsa configuration warnings in the unchanged AST dependency. RustPython runtime/compiler integration and the earlier finite CPython comparison were not rerun for this reduced-scope head; earlier results do not establish full compatibility of this revision.

Assistance

Prepared and tested with OpenAI Codex at the maintainer's explicit request for the RustPython-managed fork. No claim of manual maintainer review is implied.

@coderabbitai

coderabbitai Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@youknowdot youknowdot changed the title Complete PEP 798 unpacking operands and diagnostics Complete PEP 798 operands and Python 3.15 diagnostics Oct 21, 2026
@youknowdot
youknowdot force-pushed the pep798-unpacking-operands-315 branch from 90bd525 to 19a5e16 Compare November 13, 2026 05:41
@youknowone
youknowone changed the base branch from rustpython-0.17.1 to rustpython-0.17.5 November 13, 2026 05:42
@youknowdot youknowdot changed the title Complete PEP 798 operands and Python 3.15 diagnostics Improve Python 3.15 parser diagnostics and recovery Nov 14, 2026
youknowdot and others added 7 commits November 14, 2026 13:14
Parse full expression operands in dictionary and starred generator comprehensions. Preserve conditional-unpacking fallback ordering and recover malformed dictionary unpacking without invalid AST ranges. Add parser fixtures and regenerated snapshots.

Assisted-by: OpenAI Codex:model version unavailable
Update parameter terminology and repeated-star wording without changing
parameter parsing or recovered syntax. Regenerate affected snapshots.

Assisted-by: OpenAI Codex:model-version-unavailable
Diagnose standalone case clauses, assertion assignments, mapping rest
ordering and misplaced lazy imports. Preserve ordinary recovered ASTs
and restore speculative recovery state, including lexical-error paths.

Assisted-by: OpenAI Codex:model-version-unavailable
Project multiline literal-concatenation diagnostics onto the final
literal line while retaining original expression ranges for lexical
error precedence.

Assisted-by: OpenAI Codex:model-version-unavailable
Report the opening brace line for multiline f-string and t-string
replacement fields. Track lexical hint boundaries without changing
recovered syntax or existing PEP 798 deferred-error state.

Assisted-by: OpenAI Codex:model-version-unavailable
@youknowdot
youknowdot force-pushed the pep798-unpacking-operands-315 branch from 7e5c901 to cd0390f Compare November 14, 2026 04:16

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants