Commit 80196ed
Implement Phase 2 parser: PEG recursive-descent parser for Python 3.12
Build a complete parser that transforms the lexer's token stream into a
Python 3.12 AST. This establishes the source → tokens → AST pipeline
needed for the interpreter.
AST types (Lython/AST/Types.lean):
- Expr (27 variants), Stmt (25 variants) in mutual inductive blocks
- Supporting types: Comprehension, Arguments, Arg, CallKeyword,
ExceptHandler, FunctionDef, ClassDef, Alias, WithItem, Module
- Dump functions for test assertions via string comparison
Parser infrastructure:
- ParserM monad (StateT over Except) with token array + index
- PEG combinators: attempt/backtrack, many, sepBy, optional
- Token matching helpers for keywords, operators, delimiters, literals
Expression parser (18 precedence levels):
- Atoms through named expressions, full operator precedence
- Primaries: attribute access, calls, subscripts, slicing
- Comprehensions (list/set/dict), implicit string concatenation
- Boolean/comparison chaining, ternary if, lambda, yield, starred
Statement parser:
- Simple: assignment, augmented/annotated assignment, return, pass,
break, continue, raise, assert, global, nonlocal, del, import
- Compound: if/elif/else, while, for, with, try/except/finally,
def, class, async, decorators
- Block parsing via NEWLINE/INDENT/DEDENT tokens
- Function parameter parsing (*args, **kwargs, defaults, annotations)
70+ parser tests covering expressions, statements, and integration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent 632d406 commit 80196ed
13 files changed
Lines changed: 2122 additions & 9 deletions
File tree
- LythonTest
- Lython
- AST
- Parser
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | | - | |
28 | | - | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
29 | 38 | | |
30 | 39 | | |
31 | 40 | | |
32 | 41 | | |
33 | 42 | | |
34 | 43 | | |
| 44 | + | |
35 | 45 | | |
36 | 46 | | |
37 | 47 | | |
38 | 48 | | |
39 | 49 | | |
40 | 50 | | |
41 | 51 | | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
42 | 55 | | |
43 | 56 | | |
44 | 57 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
2 | 2 | | |
3 | | - | |
4 | | - | |
| 3 | + | |
0 commit comments