Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify needed shared libs in wasm kernel spec #221

Merged
merged 6 commits into from
Jan 16, 2025

Conversation

anutosh491
Copy link
Collaborator

@anutosh491 anutosh491 commented Jan 15, 2025

Fixes #219

Built on top of #220 and should be rebased once that goes in

@anutosh491 anutosh491 marked this pull request as draft January 15, 2025 07:21
@anutosh491
Copy link
Collaborator Author

Quite some things won't be required for the wasm kernel as mentioned here (#185)

I shall make the changes in wasm_kernel.json.in once I confirm the requirement for the variables. Converting to draft till then.

@codecov-commenter
Copy link

codecov-commenter commented Jan 15, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.03%. Comparing base (a643ea4) to head (b9db408).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #221      +/-   ##
==========================================
+ Coverage   80.72%   81.03%   +0.30%     
==========================================
  Files          19       19              
  Lines         970      970              
  Branches       93       93              
==========================================
+ Hits          783      786       +3     
+ Misses        187      184       -3     

see 1 file with indirect coverage changes

see 1 file with indirect coverage changes

@anutosh491
Copy link
Collaborator Author

As compared to the native kernel, the wasm kernel spec won't be having

  1. The env field
  "env": {
      "PATH":"@XEUS_CPP_PATH@",
      "LD_LIBRARY_PATH":"@XEUS_CPP_LD_LIBRARY_PATH@"
  }
  

This is because even if we want to access an environment variables i.e. our wasm module needs something, it needs to end up in our js file (xcpp.js) before the wasm module is initialized and that is done through a pre.js file and not this way. See the following to see how xeus-r injects env variables to make use of it (jupyter-xeus/xeus-r@6344e4d)

  1. The resource dir
"-resource-dir", "@XEUS_CPP_RESOURCE_DIR@"

This is because, we can't use resource dir this way through a flag. For accessing the resource dir, first we want CppInterOp to provide a resource dir that needs to be preloaded along side the sysroot using --preload-file.

  1. The include dir
"-I", "@XEUS_CPP_INCLUDE_DIR@",

Again we can't access a local path like this /Users/anutosh491/micromamba/envs/xeus-cpp-wasm-host through a flag. xeus-cpp-wasm-host is basically the environment we are interested in. And accessing stuff out of this environment would anyways be done by Jupyterlite-xeus leveraging empack

After running jupyter lite build, we can see the _output directory would fetch all the required kernel packages (the tar.gz files) and create an empack_env_meta.json (looks like this for me locally)

{
    "prefix": "/",
    "packages": [
        {
            "name": "emscripten-abi",
            "version": "3.1.45",
            "build": "h267e887_32",
            "filename_stem": "emscripten-abi-3.1.45-h267e887_32",
            "filename": "emscripten-abi-3.1.45-h267e887_32.tar.gz"
        },
        {
            "name": "xeus-lite",
            "version": "3.0.0",
            "build": "h5ef0a50_1",
            "filename_stem": "xeus-lite-3.0.0-h5ef0a50_1",
            "filename": "xeus-lite-3.0.0-h5ef0a50_1.tar.gz"
        },
       .........
       ........
        {
            "name": "cppinterop",
            "version": "1.5.0",
            "build": "h18da88b_2",
            "filename_stem": "cppinterop-1.5.0-h18da88b_2",
            "filename": "cppinterop-1.5.0-h18da88b_2.tar.gz"
        },
        {
            "name": "nlohmann_json",
            "version": "3.11.3",
            "build": "he2bab0f_1",
            "filename_stem": "nlohmann_json-3.11.3-he2bab0f_1",
            "filename": "nlohmann_json-3.11.3-he2bab0f_1.tar.gz"
        }
    ]
}

This basically leads to unpacking contents from these packages in the browser before we create the interpreter.

image

@anutosh491 anutosh491 marked this pull request as ready for review January 15, 2025 08:58
@vgvassilev vgvassilev requested a review from mcbarton January 15, 2025 10:38
@anutosh491 anutosh491 mentioned this pull request Jan 16, 2025
4 tasks
Copy link

@martinRenou martinRenou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just a small question

"metadata": {
"debugger": false,
"shared": {
"libclangCppInterOp.so": "lib/libclangCppInterOp.so"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌🏽

@@ -71,12 +71,14 @@ jobs:
- name: Jupyter Lite integration
shell: bash -l {0}
run: |
micromamba create -n xeus-lite-host jupyterlite-core
micromamba create -n xeus-lite-host jupyterlite-core jupyter_server notebook

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is notebook really required?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really but I think was added earlier by a contributor as an alternative to lab, hence the change is present there. Keeping it for now and can be removed subsequently if not adding value.

Copy link

@martinRenou martinRenou Jan 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as an alternative to lab

Installing notebook will have no impact on this concerning JupyterLite.

It's the jupyter lite build --apps notebook --apps lab which impacts which kinds of UIs are installed.

JupyterLite already defaults to providing the notebook UI actually, you'll only need to point to a different URL. e.g. for xeus-r you can go on https://jupyter-xeus.github.io/xeus-r/tree instead of https://jupyter-xeus.github.io/xeus-r/lab

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh okay wow, I didn't know we could frame the url like this. Getting rid of it !

Comment on lines -279 to -280
cp $PREFIX/bin/xcpp.data _output/extensions/@jupyterlite/xeus/static
cp $PREFIX/lib/libclangCppInterOp.so _output/extensions/@jupyterlite/xeus/static

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to see this being removed :D

@anutosh491
Copy link
Collaborator Author

Thanks for the reviews. Merging !

@anutosh491 anutosh491 merged commit 1904838 into compiler-research:main Jan 16, 2025
13 checks passed
@anutosh491 anutosh491 deleted the wasm_kernel branch January 16, 2025 09:07
@anutosh491 anutosh491 mentioned this pull request Jan 16, 2025
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade to jupyterlite-xeus 3
3 participants