Vendor tree-sitter parsers

[rien]

We currently use tree-sitter to have parsers for a broad range of programming languages. We provide a list of "supported languages" for which we guarantee that they should work, but we also claim that extra languages should be easy to install.

In practice however, this has some problems: parsers make releases at a different pace, which causes them to have incompatible language versions. Some parsers are not released on npm at all (r-lib/tree-sitter-r#41).

One possible solution is providing our own package @dodona/dolos-parsers where we would vendor a large list of languages, giving us more control and agility. This approach is based on that of https://github.com/Wilfred/difftastic which does something similar for a Rust tool. However, since tree-sitter requires to be built natively for node using node-gyp, we would need to add an extra layer on top of that which will build all the parsers.

[ahan98]

Hi, I'd like to use Dolos for the R language. Since its parser is not on npm, does that mean integration with Dolos is not currently possible? I've installed tree-sitter-r locally by cloning it in my node_modules directory, as instructed in the Dolos documentation, but dolos run -l r *.R still doesn't work.

[rien]

Integration is possible, but you need to jump through some hoops unfortunately.

You can install the parser using the command npm install https://github.com/r-lib/tree-sitter-r which will clone the repository in node_modules, but it will also build the native binaries.

If you've already cloned the repository, you still need to run npm install and npm build inside it, I think. However, removing it and installing the repository directly using npm install will be easier.