Skip to content

Commit 7ced6d7

Browse files
authored
misc: Add a guide to update grammars (#1079)
1 parent f2643b5 commit 7ced6d7

File tree

3 files changed

+66
-0
lines changed

3 files changed

+66
-0
lines changed

README.md

+5
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,11 @@ cargo insta test --review
8080

8181
Will run the tests, generate the new snapshot references and let you review them.
8282

83+
### Updating grammars
84+
Have a look at
85+
<a href="https://mozilla.github.io/rust-code-analysis/developers/update-grammars.html" target="_blank">Update grammars guide</a>
86+
to learn how to update languages grammars.
87+
8388
# Contributing
8489

8590
If you want to contribute to the development of this software, have a look at the

rust-code-analysis-book/src/SUMMARY.md

+1
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,4 @@
1010
- [Developers Guide](developers/README.md)
1111
- [How-to: Add a new language](developers/new-language.md)
1212
- [How-to: Implement LoC](developers/loc.md)
13+
- [How-to: Update grammars](developers/update-grammars.md)
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# Update grammars
2+
3+
Each programming language needs to be parsed in order to extract its syntax and semantic: the so-called grammar of a language.
4+
In `rust-code-analysis`, we use [tree-sitter](https://github.com/tree-sitter) as parsing library since it provides a set of distinct grammars for each of our
5+
supported programming languages. But a grammar is not a static monolith, it changes over time, and it can also be affected by bugs,
6+
hence it is necessary to update it every now and then.
7+
8+
As now, since we have used `bash` scripts to automate the operations, grammars can be updated natively **only** on `Linux` and `MacOS` systems, but these scripts can also run on `Windows` using `WSL`.
9+
10+
In `rust-code-analysis` we use both **third-party** and **internal** grammars.
11+
The first ones are published on `crates.io` and maintained by external developers,
12+
while the second ones have been thought and defined inside the project to manage variant of some languages
13+
used in `Firefox`.
14+
We are going to explain how to update both of them in the following sections.
15+
16+
## Third-party grammars
17+
18+
Update the grammar version in `Cargo.toml` and `enums/Cargo.toml`. Below an example for the `tree-sitter-java` grammar
19+
20+
```toml
21+
tree-sitter-java = "x.xx.x"
22+
```
23+
where `x` represents a digit.
24+
25+
Run `./recreate-grammars` to recreate and refresh all grammars structures and data
26+
27+
```bash
28+
./recreate-grammars
29+
```
30+
31+
Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems
32+
introduced by changes in the grammars.
33+
34+
Commit your changes and create a new pull request
35+
36+
## Internal grammars
37+
38+
Update dependency `version` field in `Cargo.toml` and `enums/Cargo.toml`. Below an example for the `tree-sitter-ccomment` grammar
39+
40+
```toml
41+
tree-sitter-ccomment = { path = "./tree-sitter-ccomment", version = "=x.xx.x" }
42+
```
43+
where `x` represents a digit.
44+
45+
Open the `Cargo.toml` file of the chosen grammar and:
46+
- Set its version to the **same** value present in the main `Cargo.toml` file
47+
- Increase the `tree-sitter` version to the most recent one
48+
49+
Run `./generate-grammars/generate-grammar.sh` which updates the grammar recreating and refreshing every file and script.
50+
This script requires the name of the grammar as mandatory argument.
51+
Below an example always using the `tree-sitter-ccomment` grammar
52+
53+
```bash
54+
./generate-grammars/generate-grammar.sh tree-sitter-ccomment
55+
```
56+
57+
Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems
58+
introduced by changes in the grammars.
59+
60+
Commit your changes and create a new pull request

0 commit comments

Comments
 (0)