Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy load infrequently used modules? #7

Open
ethanhs opened this issue Nov 27, 2021 · 6 comments
Open

Lazy load infrequently used modules? #7

ethanhs opened this issue Nov 27, 2021 · 6 comments

Comments

@ethanhs
Copy link
Owner

ethanhs commented Nov 27, 2021

The build scripts now load all of the modules that aren't already removed from a zip file, but one thing we could do is lazy load things instead, only putting the most frequently used modules in the browser cache (which we are close to maxing out, I think they are often 20-50MB, and the data assets are just over 20MB).

@ethanhs
Copy link
Owner Author

ethanhs commented Dec 2, 2021

We've cut things down to under 15MB so I think we don't need to do this. If we add more modules and the binary size creeps back up I think we could consider doing this.

@brettcannon
Copy link
Collaborator

Do we have an easy way to keep an eye on the build size to help make sure we don't blow it out? Maybe something as part of CI if we get near/hit 20 MB?

@ethanhs
Copy link
Owner Author

ethanhs commented Dec 3, 2021

Yeah we should do certain checks that what we are building is reasonable (e.g. that it includes zlib, that we are under 20MB, etc).

@brettcannon
Copy link
Collaborator

Is this worth keeping open? Or should we be working towards a solution?

@ethanhs
Copy link
Owner Author

ethanhs commented Feb 11, 2022

I've been thinking about this a bit recently.

I think a lot of people will want to only include certain modules (whether Python or C) and add custom libraries, since binary size is important in a lot of Emscripten compiled scenarios.

See also #60 and pyodide/pyodide#646

Loading minimal CPython then lazy loading the rest seems like a reasonable thing to do, but I'm not sure if that part belongs in CPython :)

@brettcannon
Copy link
Collaborator

I think we should allow for the possibility, but I don't think we should be the ones implementing the functionality if it requires JS code (else we are making design decisions that may not mesh with what people want/need).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants