From 609ed8bd92bd3974ccd2ff4caa309b768e150466 Mon Sep 17 00:00:00 2001 From: TomJGooding <101601846+TomJGooding@users.noreply.github.com> Date: Thu, 27 Feb 2025 10:15:15 +0000 Subject: [PATCH 1/6] docs(text area): update syntax dependencies Update the syntax highlighting dependencies in the TextArea docs, which now installs separate language packages rather than `tree-sitter-languages`. --- docs/widgets/text_area.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/widgets/text_area.md b/docs/widgets/text_area.md index 928fd14786..19272da941 100644 --- a/docs/widgets/text_area.md +++ b/docs/widgets/text_area.md @@ -36,7 +36,7 @@ To enable syntax highlighting, you'll need to install the `syntax` extra depende poetry add "textual[syntax]" ``` -This will install `tree-sitter` and `tree-sitter-languages`. +This will install `tree-sitter` and various language packages. These packages are distributed as binary wheels, so it may limit your applications ability to run in environments where these wheels are not available. After installing, you can set the [`language`][textual.widgets._text_area.TextArea.language] reactive attribute on the `TextArea` to enable highlighting. From 5492e4faecee63633aa570cf65e13291f79887fe Mon Sep 17 00:00:00 2001 From: TomJGooding <101601846+TomJGooding@users.noreply.github.com> Date: Thu, 27 Feb 2025 11:51:03 +0000 Subject: [PATCH 2/6] docs(text area): update 'see also' link for bundled languages Link to `tree-sitter-language-pack` in the 'see also' section. This is the unoffical replacement for `tree-sitter-languages`, which is now unmaintained and incompatible with the latest tree-sitter. --- docs/widgets/text_area.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/widgets/text_area.md b/docs/widgets/text_area.md index 19272da941..e3ab5c650d 100644 --- a/docs/widgets/text_area.md +++ b/docs/widgets/text_area.md @@ -548,7 +548,7 @@ Styles from the `theme` attribute take priority. - [`EditHistory`][textual.widgets.text_area.EditHistory] - manages the undo stack - The tree-sitter documentation [website](https://tree-sitter.github.io/tree-sitter/). - The tree-sitter Python bindings [repository](https://github.com/tree-sitter/py-tree-sitter). -- `py-tree-sitter-languages` [repository](https://github.com/grantjenks/py-tree-sitter-languages) (provides binary wheels for a large variety of tree-sitter languages). +- `tree-sitter-language-pack` [repository](https://github.com/Goldziher/tree-sitter-language-pack) (provides binary wheels for a large variety of tree-sitter languages). ## Additional notes From f8aefb180ae8fd5a8d707c9e1956993e492a8936 Mon Sep 17 00:00:00 2001 From: TomJGooding <101601846+TomJGooding@users.noreply.github.com> Date: Thu, 27 Feb 2025 15:20:50 +0000 Subject: [PATCH 3/6] docs(text area): update adding custom languages The 'Adding support for custom languages' section for TextArea syntax highlighting needs updating, since `tree-sitter-languages` is no longer compatible with the latest tree-sitter used in Textual. --- docs/widgets/text_area.md | 45 +++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 23 deletions(-) diff --git a/docs/widgets/text_area.md b/docs/widgets/text_area.md index e3ab5c650d..1f9d5c9a23 100644 --- a/docs/widgets/text_area.md +++ b/docs/widgets/text_area.md @@ -440,46 +440,45 @@ To register a language, we require two things: 1. A tree-sitter `Language` object which contains the grammar for the language. 2. A highlight query which is used for [syntax highlighting](#syntax-highlighting). -##### Example - adding Java support +##### Example - adding C++ support -The easiest way to obtain a `Language` object is using the [`py-tree-sitter-languages`](https://github.com/grantjenks/py-tree-sitter-languages) package. Here's how we can use this package to obtain a reference to a `Language` object representing Java: +You can obtain a `Language` object by installing the corresponding tree-sitter package. +Search PyPI for `tree-sitter-` followed by the language name, and install the package using pip: + +``` +pip install tree-sitter-cpp +``` + +With the package installed, you can import the module and use it to create a `Language` object: ```python -from tree_sitter_languages import get_language -java_language = get_language("java") +import tree_sitter_cpp +from tree_sitter import Language + +cpp_language = Language(tree_sitter_cpp.language()) ``` -The exact version of the parser used when you call `get_language` can be checked via -the [`repos.txt` file](https://github.com/grantjenks/py-tree-sitter-languages/blob/a6d4f7c903bf647be1bdcfa504df967d13e40427/repos.txt) in -the version of `py-tree-sitter-languages` you're using. This file contains links to the GitHub -repos and commit hashes of the tree-sitter parsers. In these repos you can often find pre-made highlight queries at `queries/highlights.scm`, +The syntax highlighting also requires a highlight query for the language. +Normally, the repository for the tree-sitter language will include pre-made highlight queries at `queries/highlights.scm`, and a file showing all the available node types which can be used in highlight queries at `src/node-types.json`. -Since we're adding support for Java, lets grab the Java highlight query from the repo by following these steps: - -1. Open [`repos.txt` file](https://github.com/grantjenks/py-tree-sitter-languages/blob/a6d4f7c903bf647be1bdcfa504df967d13e40427/repos.txt) from the `py-tree-sitter-languages` repo. -2. Find the link corresponding to `tree-sitter-java` and go to the repo on GitHub (you may also need to go to the specific commit referenced in `repos.txt`). -3. Go to [`queries/highlights.scm`](https://github.com/tree-sitter/tree-sitter-java/blob/ac14b4b1884102839455d32543ab6d53ae089ab7/queries/highlights.scm) to see the example highlight query for Java. +Since we're adding support for C++, you can find a pre-made highlight query in the `tree-sitter-cpp` repository on GitHub, +in the [`queries/highlights.scm`](https://github.com/tree-sitter/tree-sitter-cpp/blob/v0.23.4/queries/highlights.scm) file. Be sure to check the license in the repo to ensure it can be freely copied. !!! warning - It's important to use a highlight query which is compatible with the parser in use, so - pay attention to the commit hash when visiting the repo via `repos.txt`. + It's important to use a highlight query which is compatible with the parser in use. + Ensure the highlight query file is from the same version of your installed package. -We now have our `Language` and our highlight query, so we can register Java as a language. +We now have our `Language` and our highlight query, so we can register C++ as a language. -```python ---8<-- "docs/examples/widgets/text_area_custom_language.py" -``` + -Running our app, we can see that the Java code is highlighted. +Running our app, we can see that the C++ code is highlighted. We can freely edit the text, and the syntax highlighting will update immediately. -```{.textual path="docs/examples/widgets/text_area_custom_language.py" columns="52" lines="8"} -``` - Recall that we map names (like `@heading`) from the tree-sitter highlight query to Rich style objects inside the `TextAreaTheme.syntax_styles` dictionary. If you notice some highlights are missing after registering a language, the issue may be: From 0325dfec9f405cdbe8f038067f196f111672e49e Mon Sep 17 00:00:00 2001 From: TomJGooding <101601846+TomJGooding@users.noreply.github.com> Date: Thu, 27 Feb 2025 16:45:21 +0000 Subject: [PATCH 4/6] docs(text area): change custom language example to ruby My original update using C++ for the custom language example was unfortunately a poor choice, as most highlights are missing! --- docs/widgets/text_area.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/widgets/text_area.md b/docs/widgets/text_area.md index 1f9d5c9a23..d3a2d629d7 100644 --- a/docs/widgets/text_area.md +++ b/docs/widgets/text_area.md @@ -440,30 +440,30 @@ To register a language, we require two things: 1. A tree-sitter `Language` object which contains the grammar for the language. 2. A highlight query which is used for [syntax highlighting](#syntax-highlighting). -##### Example - adding C++ support +##### Example - adding Ruby support You can obtain a `Language` object by installing the corresponding tree-sitter package. Search PyPI for `tree-sitter-` followed by the language name, and install the package using pip: ``` -pip install tree-sitter-cpp +pip install tree-sitter-ruby ``` With the package installed, you can import the module and use it to create a `Language` object: ```python -import tree_sitter_cpp +import tree_sitter_ruby from tree_sitter import Language -cpp_language = Language(tree_sitter_cpp.language()) +ruby_language = Language(tree_sitter_ruby.language()) ``` The syntax highlighting also requires a highlight query for the language. Normally, the repository for the tree-sitter language will include pre-made highlight queries at `queries/highlights.scm`, and a file showing all the available node types which can be used in highlight queries at `src/node-types.json`. -Since we're adding support for C++, you can find a pre-made highlight query in the `tree-sitter-cpp` repository on GitHub, -in the [`queries/highlights.scm`](https://github.com/tree-sitter/tree-sitter-cpp/blob/v0.23.4/queries/highlights.scm) file. +Since we're adding support for Ruby, you can find a pre-made highlight query in the `tree-sitter-ruby` repository on GitHub, +in the [`queries/highlights.scm`](https://github.com/tree-sitter/tree-sitter-ruby/blob/v0.23.1/queries/highlights.scm) file. Be sure to check the license in the repo to ensure it can be freely copied. @@ -472,11 +472,11 @@ Be sure to check the license in the repo to ensure it can be freely copied. It's important to use a highlight query which is compatible with the parser in use. Ensure the highlight query file is from the same version of your installed package. -We now have our `Language` and our highlight query, so we can register C++ as a language. +We now have our `Language` and our highlight query, so we can register Ruby as a language. -Running our app, we can see that the C++ code is highlighted. +Running our app, we can see that the Ruby code is highlighted. We can freely edit the text, and the syntax highlighting will update immediately. Recall that we map names (like `@heading`) from the tree-sitter highlight query to Rich style objects inside the `TextAreaTheme.syntax_styles` dictionary. From 7d06c59b55cbd0bfdabb36a7e2e2d7e61fc4f6a2 Mon Sep 17 00:00:00 2001 From: TomJGooding <101601846+TomJGooding@users.noreply.github.com> Date: Thu, 27 Feb 2025 17:08:04 +0000 Subject: [PATCH 5/6] docs(text area): update custom language code example --- .../widgets/text_area_custom_language.py | 37 +++++++++++-------- docs/widgets/text_area.md | 4 +- 2 files changed, 24 insertions(+), 17 deletions(-) diff --git a/docs/examples/widgets/text_area_custom_language.py b/docs/examples/widgets/text_area_custom_language.py index f96dbc72a3..1bfdc69bf6 100644 --- a/docs/examples/widgets/text_area_custom_language.py +++ b/docs/examples/widgets/text_area_custom_language.py @@ -1,34 +1,39 @@ from pathlib import Path -from tree_sitter_languages import get_language +import tree_sitter_ruby +from tree_sitter import Language from textual.app import App, ComposeResult from textual.widgets import TextArea -java_language = get_language("java") -java_highlight_query = (Path(__file__).parent / "java_highlights.scm").read_text() -java_code = """\ -class HelloWorld { - public static void main(String[] args) { - System.out.println("Hello, World!"); - } -} +ruby_language = Language(tree_sitter_ruby.language()) +ruby_highlight_query = (Path(__file__).parent / "ruby_highlights.scm").read_text() + +ruby_code = """\ +class Greeter + def initialize(name) + @name = name + end + + def say_hello + puts "Hello, #{@name}!" + end +end """ class TextAreaCustomLanguage(App): def compose(self) -> ComposeResult: - text_area = TextArea.code_editor(text=java_code) - text_area.cursor_blink = False + text_area = TextArea.code_editor(text=ruby_code) - # Register the Java language and highlight query - text_area.register_language("java", java_language, java_highlight_query) + # Register the Ruby language and highlight query + text_area.register_language("ruby", ruby_language, ruby_highlight_query) - # Switch to Java - text_area.language = "java" + # Switch to Ruby + text_area.language = "ruby" yield text_area -app = TextAreaCustomLanguage() if __name__ == "__main__": + app = TextAreaCustomLanguage() app.run() diff --git a/docs/widgets/text_area.md b/docs/widgets/text_area.md index d3a2d629d7..b3bd005e9c 100644 --- a/docs/widgets/text_area.md +++ b/docs/widgets/text_area.md @@ -474,7 +474,9 @@ Be sure to check the license in the repo to ensure it can be freely copied. We now have our `Language` and our highlight query, so we can register Ruby as a language. - +```python +--8<-- "docs/examples/widgets/text_area_custom_language.py" +``` Running our app, we can see that the Ruby code is highlighted. We can freely edit the text, and the syntax highlighting will update immediately. From b1096449cd7e134754bde143e4ea62ac57391668 Mon Sep 17 00:00:00 2001 From: TomJGooding <101601846+TomJGooding@users.noreply.github.com> Date: Thu, 27 Feb 2025 17:26:38 +0000 Subject: [PATCH 6/6] docs: remove unused java_highlights.scm file Remove the unused highlight query file. This was probably once required to screenshot the custom language example in the TextArea docs, but this widget and its docs have since been updated. --- docs/examples/widgets/java_highlights.scm | 140 ---------------------- 1 file changed, 140 deletions(-) delete mode 100644 docs/examples/widgets/java_highlights.scm diff --git a/docs/examples/widgets/java_highlights.scm b/docs/examples/widgets/java_highlights.scm deleted file mode 100644 index b6259be125..0000000000 --- a/docs/examples/widgets/java_highlights.scm +++ /dev/null @@ -1,140 +0,0 @@ -; Methods - -(method_declaration - name: (identifier) @function.method) -(method_invocation - name: (identifier) @function.method) -(super) @function.builtin - -; Annotations - -(annotation - name: (identifier) @attribute) -(marker_annotation - name: (identifier) @attribute) - -"@" @operator - -; Types - -(type_identifier) @type - -(interface_declaration - name: (identifier) @type) -(class_declaration - name: (identifier) @type) -(enum_declaration - name: (identifier) @type) - -((field_access - object: (identifier) @type) - (#match? @type "^[A-Z]")) -((scoped_identifier - scope: (identifier) @type) - (#match? @type "^[A-Z]")) -((method_invocation - object: (identifier) @type) - (#match? @type "^[A-Z]")) -((method_reference - . (identifier) @type) - (#match? @type "^[A-Z]")) - -(constructor_declaration - name: (identifier) @type) - -[ - (boolean_type) - (integral_type) - (floating_point_type) - (floating_point_type) - (void_type) -] @type.builtin - -; Variables - -((identifier) @constant - (#match? @constant "^_*[A-Z][A-Z\\d_]+$")) - -(identifier) @variable - -(this) @variable.builtin - -; Literals - -[ - (hex_integer_literal) - (decimal_integer_literal) - (octal_integer_literal) - (decimal_floating_point_literal) - (hex_floating_point_literal) -] @number - -[ - (character_literal) - (string_literal) -] @string - -[ - (true) - (false) - (null_literal) -] @constant.builtin - -[ - (line_comment) - (block_comment) -] @comment - -; Keywords - -[ - "abstract" - "assert" - "break" - "case" - "catch" - "class" - "continue" - "default" - "do" - "else" - "enum" - "exports" - "extends" - "final" - "finally" - "for" - "if" - "implements" - "import" - "instanceof" - "interface" - "module" - "native" - "new" - "non-sealed" - "open" - "opens" - "package" - "private" - "protected" - "provides" - "public" - "requires" - "return" - "sealed" - "static" - "strictfp" - "switch" - "synchronized" - "throw" - "throws" - "to" - "transient" - "transitive" - "try" - "uses" - "volatile" - "while" - "with" -] @keyword