Investigate "asyncifying" the core interpreter loop

[ethanhs]

Emscripten has an asyncify pass which can allow synchronous C code yield to the JS event loop and even block on an async JS call.

I expect asyncifying the core interpreter loop would probably be difficult, if not impossible, and I'm not sure if it would cause problems, but it could provide a way to have synchronous Python I/O in the async browser environment.

[ethanhs]

Oh woops, link to asyncify: https://emscripten.org/docs/porting/asyncify.html

[pmp-p]

Hi, asyncify ( old and new versions of it ) have an enormous impact on C recursive calls in the vm loop because it has to record alls stacks. It's quite ok to use it at the top level but i really don't think it's possible to convert cpython to full stackless ( there's a partial one named stackless but it was not upstreamed and - still - rely a lot on the C stack).
Maybe @joemarshall ( https://github.com/joemarshall/pyodide-async ) could elaborate on that and has other ideas to avoid the huge performance drop.

[tiran]

It's quite ok to use it at the top level but i really don't think it's possible to convert cpython to full stackless ( there's a partial one named stackless but it was not upstreamed and - still - rely a lot on the C stack).

I got good news for you. Mark, Pablo and others have removed stack usage for Python to Python calls in 3.11-dev. They are working on more C stack removals:

• https://bugs.python.org/issue45256
• https://bugs.python.org/issue45829

[ethanhs]

That sounds promising!

Though honestly in the near and probably medium term, it might be good to just run WASM Python in a webworker and use sharedarraybuffer to communicate with atomics. The only downside is its only available on newer Firefox/Chrome, and atomics aren't implemented at all in Safari...

[katharosada]

Been looking into browser support for SharedArrayBuffer + Atomics. The caniuse/MDN data is a bit out of date and the overall support is better than it looks.

• Latest Safari (15.2) now supports SAB and Atomics on both MacOS and iOS - will take a while for people to do an OS update though.
• Chrome for Android does support it - not sure how long that's been the case
• Firefox for Android will support it soon, beta version (97) has it now

In the meantime I've been working on a WebWorker + Atomics demo of a REPL for Pyodide so I would be happy to add one here. https://github.com/katharosada/wasm-terminal

[pmp-p]

WebWorker + Atomic

i guess it's #47

@katharosada can that way python/pyodide thread still access UI events / sound or GUI thread/WebGL via emscripten C/C++ support libraries : how would you integrate the two glue files from https://emscripten.org/docs/api_reference/module.html#Module.onCustomMessage?

/EDIT i have my answer webgl is ok, UI events can be handled though doc is scarse, but sound is not implemented at all