Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broad grammar improvements #8

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

thomaspurchas
Copy link

There's a few changes in the PR but each of them isn't very substantial, but together do change a few things.

  1. Update tree-sitter to latest, and update tests to handle new ERROR nodes which are more descriptive
  2. Rearrange the repo so follow tree-sitter conventions, and things like tests work as expected
  3. Fix older tests that we're failing
  4. Add support for the else branch of a when generator
  5. Add fields to nodes so that queries are easier to write
  6. Add a stringFragment rule to allow the easy extraction of strings from within string literals
  7. Improve the highlight query for types
  8. Fix some of the tree-sitter cli complaints

The vast majority of the change are just updating the test to include the stringFragment node, which slightly modified almost every test file.

Strongly recommend working through PR on commit-by-commit basis.

thomaspurchas and others added 15 commits May 20, 2024 15:14
The tree-sitter-cli expects all tests to be in a test dir within the
project root.
These tests broke at some point and were never updated. As the new
output doesn't seem incorrect, I'm just updating the expected output to
match what the grammar currently produces.
tree-sitter language repos have a specific layout and package.json which
is defined by the tree-sitter project, and enforced by the
tree-sitter-cli. Most of the changes in package.json are from the
tree-sitter-cli updating it to conformance
Updating tree-sitter has resulting in output changes due to improved
error handling in the tree-sitter parse. The new output is much more
informative when parsing errors occur compared to previous versions.
It useful to be able to query for actual string fragments between string
deliminator. The (stringFragment) now makes it possible to break down
strings into their different fragments and interpolations.
There's no need to use a regex to detect types, we can detect using just
the syntax tree. Using a regex results in unnecessary false-positive
matches.
tree-sitter doesn't like it when you use non-static functions as they
may collide with function names in other grammars. There's no reason not
to make this function static as it not used anywhere else.
Fields make it much easier to write smart queries using the grammar, as
things like detection of default values can be included in the grammar
via field, removing the need for queries to handle those detections
themselves.
Co-authored-by: Kushal Pisavadia <[email protected]>

Fix license metadata
bioball added a commit to bioball/tree-sitter-pkl that referenced this pull request Sep 30, 2024
Ports many of the fixes previously introduced in
apple#8

1. Add support for else body in when generator
2. Remove legacy syntax in for and when generators
3. Add support for shebang comments
4. Add field names to various nodes throughout the grammar
5. Add missing "module" type
@bioball
Copy link
Contributor

bioball commented Sep 30, 2024

Thanks for the PR! Sorry it took so long to address this.

This PR is a little large, but, I'm going to pick of some of these improvements into smaller PRs that can be merged separately. The first is here: #21

bioball added a commit to bioball/tree-sitter-pkl that referenced this pull request Sep 30, 2024
Ports many of the fixes previously introduced in
apple#8

1. Add support for else body in when generator
2. Remove legacy syntax in for and when generators
3. Add support for shebang comments
4. Add field names to various nodes throughout the grammar
5. Add missing "module" type
bioball added a commit to bioball/tree-sitter-pkl that referenced this pull request Sep 30, 2024
Ports many of the fixes previously introduced in
apple#8

1. Add support for else body in when generator
2. Remove legacy syntax in for and when generators
3. Add support for shebang comments
4. Add field names to various nodes throughout the grammar
5. Add missing "module" type
bioball added a commit that referenced this pull request Sep 30, 2024
Ports many of the fixes previously introduced in
#8

1. Add support for else body in when generator
2. Remove legacy syntax in for and when generators
3. Add support for shebang comments
4. Add field names to various nodes throughout the grammar
5. Add missing "module" type
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants