|
| 1 | +# Update grammars |
| 2 | + |
| 3 | +Each programming language needs to be parsed in order to extract its syntax and semantic: the so-called grammar of a language. |
| 4 | +In `rust-code-analysis`, we use [tree-sitter](https://github.com/tree-sitter) as parsing library since it provides a set of distinct grammars for each of our |
| 5 | +supported programming languages. But a grammar is not a static monolith, it changes over time, and it can also be affected by bugs, |
| 6 | +hence it is necessary to update it every now and then. |
| 7 | + |
| 8 | +As now, since we have used `bash` scripts to automate the operations, grammars can be updated natively **only** on `Linux` and `MacOS` systems, but these scripts can also run on `Windows` using `WSL`. |
| 9 | + |
| 10 | +In `rust-code-analysis` we use both **third-party** and **internal** grammars. |
| 11 | +The first ones are published on `crates.io` and maintained by external developers, |
| 12 | +while the second ones have been thought and defined inside the project to manage variant of some languages |
| 13 | +used in `Firefox`. |
| 14 | +We are going to explain how to update both of them in the following sections. |
| 15 | + |
| 16 | +## Third-party grammars |
| 17 | + |
| 18 | +Update the grammar version in `Cargo.toml` and `enums/Cargo.toml`. Below an example for the `tree-sitter-java` grammar |
| 19 | + |
| 20 | +```toml |
| 21 | +tree-sitter-java = "x.xx.x" |
| 22 | +``` |
| 23 | +where `x` represents a digit. |
| 24 | + |
| 25 | +Run `./recreate-grammars` to recreate and refresh all grammars structures and data |
| 26 | + |
| 27 | +```bash |
| 28 | +./recreate-grammars |
| 29 | +``` |
| 30 | + |
| 31 | +Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems |
| 32 | +introduced by changes in the grammars. |
| 33 | + |
| 34 | +Commit your changes and create a new pull request |
| 35 | + |
| 36 | +## Internal grammars |
| 37 | + |
| 38 | +Update dependency `version` field in `Cargo.toml` and `enums/Cargo.toml`. Below an example for the `tree-sitter-ccomment` grammar |
| 39 | + |
| 40 | +```toml |
| 41 | +tree-sitter-ccomment = { path = "./tree-sitter-ccomment", version = "=x.xx.x" } |
| 42 | +``` |
| 43 | +where `x` represents a digit. |
| 44 | + |
| 45 | +Open the `Cargo.toml` file of the chosen grammar and: |
| 46 | + - Set its version to the **same** value present in the main `Cargo.toml` file |
| 47 | + - Increase the `tree-sitter` version to the most recent one |
| 48 | + |
| 49 | +Run `./generate-grammars/generate-grammar.sh` which updates the grammar recreating and refreshing every file and script. |
| 50 | +This script requires the name of the grammar as mandatory argument. |
| 51 | +Below an example always using the `tree-sitter-ccomment` grammar |
| 52 | + |
| 53 | +```bash |
| 54 | +./generate-grammars/generate-grammar.sh tree-sitter-ccomment |
| 55 | +``` |
| 56 | + |
| 57 | +Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems |
| 58 | +introduced by changes in the grammars. |
| 59 | + |
| 60 | +Commit your changes and create a new pull request |
0 commit comments