Skip to content

Commit

Permalink
chore: upgrade to latest ypy_websocket and introduce anyio
Browse files Browse the repository at this point in the history
  • Loading branch information
echarles committed Jun 29, 2023
1 parent 054a767 commit 1d95e40
Show file tree
Hide file tree
Showing 37 changed files with 200 additions and 718 deletions.
6 changes: 3 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ CONDA_ACTIVATE=source $$(conda info --base)/etc/profile.d/conda.sh ; conda activ
CONDA_DEACTIVATE=source $$(conda info --base)/etc/profile.d/conda.sh ; conda deactivate
CONDA_REMOVE=source $$(conda info --base)/etc/profile.d/conda.sh ; conda remove -y --all -n

ENV_NAME=datalayer
ENV_NAME=jupyter-rtc-test

.DEFAULT_GOAL := default

Expand All @@ -23,9 +23,9 @@ default: help ## default target is help

env: warning ## env
($(CONDA); \
SLUGIFY_USES_TEXT_UNIDECODE=yes conda env create -n ${ENV_NAME] -f ${DATALAYER_HOME}/src/environment.yml )
SLUGIFY_USES_TEXT_UNIDECODE=yes conda env create -n ${ENV_NAME} -f ${DATALAYER_HOME}/src/environment.yml )
@exec echo "-------------------------------------------------------"
@exec echo "conda activate ${ENV_NAME]"
@exec echo "conda activate ${ENV_NAME}"
@exec echo "-------------------------------------------------------"

env-rm: warning ## env-rm
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ You can run the stress tests from the CLI (Command Line Interface) or from a UI
<img alt="Jupyter RTC Test" src="https://datalayer-jupyter-examples.s3.amazonaws.com/jupyter-rtc-test.gif" />
</div>

## Distributed
## Distributed Actors

The tests aims to replicate real life cases, meaning that most of them are running in a distrbuted way.

Expand Down Expand Up @@ -44,6 +44,7 @@ The Jupyter RTC Stack is spread across various repositories
<div align="center" style="text-align: center">
<img alt="Jupyter RTC Stack" src="https://raw.githubusercontent.com/datalayer/jupyter-rtc-test/main/style/svg/jupyter-rtc-stack.image.svg" />
</div>

## Community

The RTC stress test are being discussed [in an issue on the official JupyterLab repository](https://github.com/jupyterlab/jupyterlab/issues/14532).
Expand All @@ -65,7 +66,7 @@ You can read more about the Conflict free Replicated Data Types (CRDT).
- [CRDT.tech](https://crdt.tech)
- [Jupyter Real Time Collaboration | JupyterCon 2020](https://www.youtube.com/watch?v=G5CVtJIBE5I)
- [Wikipedia](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type)
- [Y.js](https://docs.yjs.dev), the library used as CRDT implementation.
- [Y.js](https://docs.yjs.dev) used as CRDT implementation.

## ⚖️ License

Expand Down
48 changes: 23 additions & 25 deletions conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,14 @@
import subprocess

import pytest
import pytest_asyncio

from pathlib import Path
from websockets import serve
from websockets import serve # type: ignore

from y_py import encode_state_as_update, encode_state_vector, YDoc
from ypy_websocket import WebsocketServer

from ypy_websocket import WebsocketServer

pytest_plugins = [
# "jupyter_server.pytest_plugin",
# "jupyter_rtc_test.tests.jupyter_server_fixtures",
]

HERE = Path(__file__).parent

Expand All @@ -36,11 +31,6 @@ def update_json_file(path: Path, d: dict):
# update_json_file(HERE / "node_modules/@jupyterlab/nbformat/package.json", d)


@pytest.fixture
def jp_server_config(jp_server_config):
return {"ServerApp": {"jpserver_extensions": {"jupyter_rtc_test": True}}}


class TestYDoc:
def __init__(self):
self.ydoc = YDoc()
Expand All @@ -58,24 +48,32 @@ def update(self):


@pytest.fixture
def test_ydoc():
return TestYDoc()
async def y_websocket_server(request):
try:
kwargs = request.param
except Exception:
kwargs = {}
websocket_server = WebsocketServer(**kwargs)
try:
async with websocket_server, serve(websocket_server.serve, "127.0.0.1", 1234):
yield websocket_server
except Exception:
pass


@pytest_asyncio.fixture
async def y_websocket_client(request):
@pytest.fixture
def y_websocket_client(request):
client_id = request.param
p = subprocess.Popen(["node", f"{HERE / 'src/__tests__/clients/client-'}{client_id}.mjs"])
yield p
p.kill()


@pytest_asyncio.fixture
async def y_websocket_server(request):
try:
kwargs = request.param
except Exception:
kwargs = {}
websocket_server = WebsocketServer(**kwargs)
async with serve(websocket_server.serve, "127.0.0.1", 1234):
yield websocket_server
@pytest.fixture
def test_ydoc():
return TestYDoc()


@pytest.fixture
def anyio_backend():
return "asyncio"
2 changes: 1 addition & 1 deletion docs/environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ git clone https://github.com/datalayer/jupyter-rtc-test.git && \
cd jupyter-rtc-test
```

If you already have created a conda environment, please remove it.
If you already have created a conda environment with name `jupyter-rtc-test`, please remove it.

```bash
conda deactivate && \
Expand Down
8 changes: 4 additions & 4 deletions docs/issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ This page contains encountered issues while running the tests. [Potential soluti

## Insert and Delete random characters

[Insert and delete random characters](https://github.com/datalayer/jupyter-rtc-test/blob/main/src/components/stress-cli/tabs/scenarii/scenarii.json#L3) scenario: Node.js often do not converge, Python and Browser sometimes do not converge.
[Insert and delete random characters](https://github.com/datalayer/jupyter-rtc-test/blob/main/src/components/cli/tabs/scenarii/scenarii.json#L3) scenario: Node.js often do not converge, Python and Browser sometimes do not converge.

## YNotebook Update Source

[test_jupyter_ydoc_update_source](https://github.com/datalayer/jupyter-rtc-test/blob/main/jupyter_rtc_test/tests/l6_jupyter_ydoc/stress-cli/test_jupyter_ydoc_update_source.py) - Fails to append instead of replacing the text content.
[test_jupyter_ydoc_update_source](https://github.com/datalayer/jupyter-rtc-test/blob/main/jupyter_rtc_test/tests/l6_jupyter_ydoc/cli/test_jupyter_ydoc_update_source.py) - Fails to append instead of replacing the text content.

## Caught error while handling a Yjs update Error

Expand All @@ -37,11 +37,11 @@ Caught error while handling a Yjs update Error: Unexpected case

## Set the content of a YNotebook cell

[Set the content of a YNotebook cell](https://github.com/datalayer/jupyter-rtc-test/blob/main/src/components/stress-cli/tabs/scenarii/scenarii.json#L24) scenario: the source sometimes is appended instead of being set.
[Set the content of a YNotebook cell](https://github.com/datalayer/jupyter-rtc-test/blob/main/src/components/cli/tabs/scenarii/scenarii.json#L24) scenario: the source sometimes is appended instead of being set.

## KeyError: nbformat

[Set the content of a YNotebook cell](https://github.com/datalayer/jupyter-rtc-test/blob/main/src/components/stress-cli/tabs/scenarii/scenarii.json#L24) scenario: python exception after some time KeyError: 'nbformat'
[Set the content of a YNotebook cell](https://github.com/datalayer/jupyter-rtc-test/blob/main/src/components/cli/tabs/scenarii/scenarii.json#L24) scenario: python exception after some time KeyError: 'nbformat'

```
KeyError: 'nbformat'
Expand Down
6 changes: 3 additions & 3 deletions docs/tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,16 @@ yarn kill && \

The following subsections narrow down the tests, starting from the lower technical layers to the higher ones.

## 0. Y.js
## 0. Yjs

`Y.js` delivers the JavaScript shared data types for building collaborative software and is developed in https://github.com/yjs/yjs
`Yjs` delivers the JavaScript shared data types for building collaborative software and is developed in https://github.com/yjs/yjs

```bash
yarn test:js:0
yarn test:py:0
```

**♪ Note** `Y.js` should be replaced by `Ywasm`.
**♪ Note** `Yjs` should be replaced by `Ywasm`.

## 1. Y Rust - Yrs

Expand Down
4 changes: 2 additions & 2 deletions jupyter_rtc_test/stresser/handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def custom_hook(args):

def run_nodejs_client(id, script, textLength, warmupPeriodSeconds, room_name):
time.sleep(random.randint(0, int(warmupPeriodSeconds)))
nodejs_process = subprocess.Popen(["node", f"{HERE}/../../src/__tests__/clients/stress-ui/" + script, str(id), textLength, warmupPeriodSeconds, room_name])
nodejs_process = subprocess.Popen(["node", f"{HERE}/../../src/__tests__/clients/ui/" + script, str(id), textLength, warmupPeriodSeconds, room_name])
return nodejs_process

def run_python_client(id, script, textLength, warmupPeriodSeconds, room_name):
Expand All @@ -44,7 +44,7 @@ def run_python_client(id, script, textLength, warmupPeriodSeconds, room_name):

def run_browser_client(id, script, textLength, warmupPeriodSeconds, room_name):
time.sleep(random.randint(0, int(warmupPeriodSeconds)))
browser_process = subprocess.Popen(["node", f"{HERE}/../../src/__tests__/clients/stress-ui/" + script, str(id), textLength, warmupPeriodSeconds, room_name])
browser_process = subprocess.Popen(["node", f"{HERE}/../../src/__tests__/clients/ui/" + script, str(id), textLength, warmupPeriodSeconds, room_name])
return browser_process


Expand Down
Loading

0 comments on commit 1d95e40

Please sign in to comment.