Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history 
commit abaef02eaaec44234b331dec0395ee2b63740d05
Author: Moosems <[email protected]>
Date:   Sat Jun 1 13:59:59 2024 +0300

    Final tests for 0.3.0

commit 114270dac0f18c2efaac73cc020950b6b5e17454
Author: Moosems <[email protected]>
Date:   Sat Jun 1 13:04:19 2024 +0300

    Fix highlighting matching

commit 01025ad000f03ae5e2c60a20c0aaf640c919fb14
Author: Moosems <[email protected]>
Date:   Fri May 31 23:27:18 2024 +0300

    Update docs and example

commit 1ebc628f2849be64fee46e064b19e3cef42991eb
Author: Moosems <[email protected]>
Date:   Fri May 31 23:24:25 2024 +0300

    Update tests

commit d24db4f3f245230c768f500a9101d93d06c9ef0e
Author: Moosems <[email protected]>
Date:   Fri May 31 23:01:48 2024 +0300

    Reformat, upgrade version, add example

commit 767cd367344b13df6c48fc3a0f6f0002eed8e557
Author: Moosems <[email protected]>
Date:   Fri May 31 22:55:02 2024 +0300

    Convert to tmp_files

commit daf49770a7e2c9471bbd510d5cea03a8cebd0f6c
Author: Moosems <[email protected]>
Date:   Fri May 31 21:00:24 2024 +0300

    Improve highlight and add examples

commit 139814aa2bb789c0ace1094fbe30a4b32323421f
Author: Moosems <[email protected]>
Date:   Fri May 31 20:35:30 2024 +0300

    Remove reasoning

commit 5986e6da85c735d1f4043055a694d022c76d5941
Author: Moosems <[email protected]>
Date:   Fri May 31 20:34:20 2024 +0300

    Update readme with intriguing findings

    On my system the first usage takes up to 3x longer for the same input while it waits for the server process to initialize (due to the need to start up the python interpreter)

commit 4a7bb51ebe0e00b2bc92fa95f36922eea29ae2bc
Author: Moosems <[email protected]>
Date:   Fri May 31 20:20:46 2024 +0300

    Have responses written to tmp files

commit ada8506328f078bde448b2c4d7f642d4a739cb69
Author: Moosems <[email protected]>
Date:   Fri May 31 17:53:29 2024 +0300

    Return with temp_file

commit b2e88075dda42d78e28271de6a57977884be5ac4
Author: Moosems <[email protected]>
Date:   Fri May 31 17:49:41 2024 +0300

    Set better example

    Not killing the IPC when using it for a short amount of time causes issues (when the python interpreter closes before the server sends its last output)

commit 4ac82c9d3c63b394de33fb7a164992e455cb8a8f
Author: Moosems <[email protected]>
Date:   Fri May 31 17:27:02 2024 +0300

    Add temp file to mesage type

    Required adding temp files to messages (even if not used yet)

commit 279a60ee6784f180729bd6d06aafae824f4c0e16
Author: Moosems <[email protected]>
Date:   Fri May 31 17:21:40 2024 +0300

    Use all_ids in ipc.py

commit 2870c57ee8e81eefaf17a0507a498b30ca3e866c
Author: Moosems <[email protected]>
Date:   Fri May 31 17:19:15 2024 +0300

    Add tempfile to ids in ipc, server confirm id

commit 2f88b05153348c21fc48385bc1f9c20b5abfeab8
Author: Moosems <[email protected]>
Date:   Fri May 31 16:38:06 2024 +0300

    Create failing example

commit 46a6271e4ab402676835fae703f022d49adc4f6a
Author: Moosems <[email protected]>
Date:   Fri May 31 16:37:45 2024 +0300

    Squashed commit of the following:

    commit f9b3c7b
    Author: Moosems <[email protected]>
    Date:   Fri May 31 16:36:36 2024 +0300

        Reformat

    commit 4c18c0d
    Author: Moosems <[email protected]>
    Date:   Fri May 31 16:34:04 2024 +0300

        Squashed commit of the following:

        commit 9fb8a906d6a17da7996406ab58c4d2ca3a4e9f21
        Author: Moosems <[email protected]>
        Date:   Fri May 31 16:33:54 2024 +0300

            Update docs

        commit cec86a408c7b30194b23aa4388359da81c638ac0
        Author: Moosems <[email protected]>
        Date:   Fri May 31 16:30:02 2024 +0300

            Use tuples

commit 6218eb1dce635f5cb40d5723f7c5c2455eb195de
Author: Moosems <[email protected]>
Date:   Fri May 31 16:34:21 2024 +0300

    Squashed commit of the following:

    commit 4c18c0d
    Author: Moosems <[email protected]>
    Date:   Fri May 31 16:34:04 2024 +0300

        Squashed commit of the following:

        commit 9fb8a906d6a17da7996406ab58c4d2ca3a4e9f21
        Author: Moosems <[email protected]>
        Date:   Fri May 31 16:33:54 2024 +0300

            Update docs

        commit cec86a408c7b30194b23aa4388359da81c638ac0
        Author: Moosems <[email protected]>
        Date:   Fri May 31 16:30:02 2024 +0300

            Use tuples
  • Loading branch information
@Moosems
Moosems committed Jun 1, 2024
1 parent f9b3c7b commit 4084835
Show file tree
Hide file tree
Showing 15 changed files with 327 additions and 168 deletions.
90 changes: 37 additions & 53 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@ In the Command Line, paste the following: `pip install salve_ipc`

## Description

Salve is an IPC library that can be used by code editors to get autocompletions, replacements, and syntax highlighting.
Salve is an IPC library that can be used by code editors to easily get autocompletions, replacements, and syntax highlighting.

> **Note**
> The first time that the system is loaded or a new server needs to be started it will take a fair bit longer than if it is simply kept alive by pinging regularly.
## Documentation

Expand All @@ -33,7 +36,7 @@ The `Token` dataclass gives easy type checking for tokens returned from the high

The `hidden_chars` (`dict[str, str]`) dictionary holds a bunch of hidden (zero width) characters as keys and then names for them as values. `Token`'s of type "Hidden_Char" give the index to hidden characters and allow the user to display hidden characters to them that they may not see. These characters appear in code posted on forums or blogs by those hoping to prevent others from simply copy-pasting their code along with many other places.

### `Request` and `Response` TypedDict classes
### `Response` TypedDict classes

The `Request` and `Response` TypedDict classes allow for type checking when handling output from salve_ipc.

Expand All @@ -47,61 +50,42 @@ The `tokens_from_result()` function takes the results from a `highlight` command

### `IPC` Class

| Method | Description | Arguments |
| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `.ping()` | Pings the server. After five seconds the server closes if not pinged so it is better for performance to keep it alive but it will be reopened either way | None |
| `.get_response()` | Gets a response of the requested command | `command`: str |
| `.request()` | Makes a request to the server | `command`: str, `file`: str, `expected_keywords`: list[str] ("autocomple" or "replacements"), `current_word`: str ("autocomple" or "replacements"), `language`: str ("highlight") |
| `.cancel_request()` | Cancels request of command type and removes reponse if it was recieved. Must be called before `.get_response()` to work | `command`: str |
| `.update_file()` | Updates files stored on the server that are used to get responses | `filename`: str, `current_state`: str (just the text of the file) |
| `.remove_file()` | Removes a file of the name given if any exists. Note that a file should only be removed when sure that all other requests using the file are completed. If you delete a file right after a request you run the risk of it removing the file before the task could be run and causing a server crash. | `filename`: str |
| `.kill_IPC()` | This kills the IPC process and acts as a precaution against wasted CPU when the main thread no longer needs the IPC | None |
| Method | Description | Arguments |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `.ping()` | Pings the server. After five seconds the server closes if not pinged so it is better for performance to keep it alive but it will be reopened either way | None |
| `.get_response()` | Gets a response of the requested command | `command`: str |
| `.request()` | Makes a request to the server | `command`: str, `file`: str, `expected_keywords`: list[str] ("autocomple" or "replacements"), `current_word`: str ("autocomple" or "replacements"), `language`: str ("highlight") |
| `.cancel_request()` | Cancels request of command type and removes reponse if it was recieved. Must be called before `.get_response()` to work | `command`: str |
| `.update_file()` | Updates files stored on the server that are used to get responses | `filename`: str, `current_state`: str (just the text of the file) |
| `.remove_file()` | Removes a file of the name given if any exists. Note that a file should only be removed when sure that all other requests using the file are completed. If you delete a file right after a request you run the risk of it removing the file before the task could be run and causing a server crash (`Request`'s go after `Notification`'s and `Ping`'s). | `filename`: str |
| `.kill_IPC()` | This kills the IPC process and acts as a precaution against wasted CPU when the main thread no longer needs the IPC | None |

### Basic Usage:

```python
from os import set_blocking
from selectors import EVENT_READ, DefaultSelector
from sys import stdin, stdout

from salve_ipc import IPC, Response

autocompleter = IPC()

set_blocking(stdin.fileno(), False)
set_blocking(stdin.fileno(), False)
selector = DefaultSelector()
selector.register(stdin, EVENT_READ)

stdout.write("Code: \n")
stdout.flush()

while True:
# Keep IPC alive
autocompleter.ping()

# Add file
autocompleter.add_file("test", "")

# Check input
events = selector.select(0.025)
if events:
# Make requests
for line in stdin:
autocompleter.update_file("test", line)
autocompleter.request(
"autocomplete",
expected_keywords=[],
full_text=line,
current_word=line[-2],
)

# Check output
output: Response | None = autocompleter.get_response()
if not output:
continue
stdout.write(str(output) + "\n")
stdout.flush()
from time import sleep

from salve_ipc import IPC, Response, tokens_from_result

context = IPC()

context.update_file(
"test",
open(__file__, "r+").read(),
)

context.request(
"autocomplete",
file="test",
expected_keywords=[],
current_word="t",
)

sleep(1)

output: Response = context.get_response("autocomplete") # type: ignore
print(tokens_from_result(output["result"])) # type: ignore
context.kill_IPC()
```

## How to run and contribute
Expand Down
2 changes: 2 additions & 0 deletions examples/example_usage.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,5 @@
# Write response
stdout.write(str(output) + "\n")
stdout.flush()

context.kill_IPC()
1 change: 1 addition & 0 deletions examples/gui_client.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,4 @@ def ping_loop() -> None:

root.after_idle(ping_loop)
root.mainloop()
context.kill_IPC()
23 changes: 23 additions & 0 deletions examples/simple_autocomplete_example.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
from time import sleep

from salve_ipc import IPC, Response, tokens_from_result

context = IPC()

context.update_file(
"test",
open(__file__, "r+").read(),
)

context.request(
"autocomplete",
file="test",
expected_keywords=[],
current_word="t",
)

sleep(1)

output: Response = context.get_response("autocomplete") # type: ignore
print(tokens_from_result(output["result"])) # type: ignore
context.kill_IPC()
22 changes: 0 additions & 22 deletions examples/simple_example.py
View file

This file was deleted.

17 changes: 17 additions & 0 deletions examples/simple_highlight_example.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
from time import sleep

from salve_ipc import IPC, Response, tokens_from_result

context = IPC()

context.update_file(
"test",
open(__file__, "r+").read(),
)

context.request("highlight", file="test", language="python")

sleep(1)
output: Response | None = context.get_response("highlight")
print(tokens_from_result(output["result"])) # type: ignore
context.kill_IPC()
19 changes: 19 additions & 0 deletions examples/simple_replacements_example.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
from time import sleep

from salve_ipc import IPC, Response

context = IPC()

context.update_file(
"test",
open(__file__, "r+").read(),
)

context.request(
"replacements", file="test", expected_keywords=[], current_word="contest"
)

sleep(1)
output: Response | None = context.get_response("replacements")
print(output["result"]) # type: ignore
context.kill_IPC()
30 changes: 15 additions & 15 deletions salve_ipc/highlight/highlight.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
from pygments.token import _TokenType

default_tokens: list[str] = [
"Token.Text" "Token.Text.Whitespace",
"Token.Text.Whitespace",
"Token.Text",
"Token.Error",
"Token.Keyword",
"Token.Name",
Expand All @@ -20,8 +21,8 @@
"Token.Generic",
]
generic_tokens: list[str] = [
"Text",
"Whitespace",
"Text",
"Error",
"Keyword",
"Name",
Expand Down Expand Up @@ -67,7 +68,7 @@ def get_new_token_type(old_token: str) -> str:
"""Turns pygments token types into a generic predefined Token"""
new_type: str = generic_tokens[0]
for index, token in enumerate(default_tokens):
if token.startswith(old_token):
if old_token.startswith(token):
new_type = generic_tokens[index]
break
return new_type
Expand Down Expand Up @@ -162,25 +163,24 @@ def find_hidden_chars(lines: list[str]) -> list[Token]:
def get_highlights(full_text: str, language: str = "text") -> list[Token]:
"""Gets pygments tokens from text provided in language proved and converts them to Token's"""
lexer: Lexer = get_lexer_by_name(language)
split_text: list[str] = full_text.splitlines()
new_tokens: list[Token] = []
og_tokens: list[tuple[_TokenType, str]] = list(lex(full_text, lexer))
start_index: tuple[int, int] = (1, 0)

for token in og_tokens:
new_type: str = get_new_token_type(str(token[0]))
token_str: str = token[1]
token_len: int = len(token_str)
new_token = Token(start_index, token_len, new_type)
new_tokens.append(new_token)
for line in split_text:
og_tokens: list[tuple[_TokenType, str]] = list(lex(line, lexer))
for token in og_tokens:
new_type: str = get_new_token_type(str(token[0]))
token_str: str = token[1]

if token_str == "\n":
start_index = (start_index[0] + 1, 0)
continue
token_len: int = len(token_str)
new_token = Token(start_index, token_len, new_type)
new_tokens.append(new_token)

start_index = (start_index[0], start_index[1] + token_len)
start_index = (start_index[0], start_index[1] + token_len)
start_index = (start_index[0] + 1, 0)

# Add extra token types
split_text: list[str] = full_text.splitlines()
new_tokens += get_urls(split_text)
new_tokens += find_hidden_chars(split_text)

Expand Down
Loading

0 comments on commit 4084835

Please sign in to comment.