SPEC 14: Interactive Documentation

[agriyakhetarpal]

This new SPEC, titled "Interactive Documentation", aims to describe the process of implementing interactive documentation deployments for Scientific Python projects' websites through WebAssembly-powered in-browser computation. The process is detailed under various sections in the SPEC that build up incrementally, and various resources for Core Projects' members have been shared to look at while reading through it.

This topic has been in the scope of discussion across the 2024 Summit and is in contention for the upcoming 2025 Summit as well; I hope that the existence of this document will help drive further discussions in this area and its subsequent endorsement and adoption by projects.

Closes scientific-python/summit-2024#19
See scientific-python/cookie#157
Closes Quansight-Labs/czi-scientific-python-mgmt#17
See also scientific-python/summit-2025#5
See also scientific-python/summit-2025#25

Discourse thread: https://discuss.scientific-python.org/t/spec-14-interactive-documentation/1910
Read it live: https://deploy-preview-388--scientific-python-specs.netlify.app/specs/spec-0014/

[betatim]

What do you think of linking to some documentation pages (Numpy, scikit-learn) that already have this setup so that people can "witness the miracle" with just a few clicks? Basically, showing them how cool interactive docs are by linking them to a few good examples. A bit like itneractive docs let users of that library witness the miracle of the library :D

We could then even go a step further and in the section(s) on "how to do this for your project" link to how things are implemented for the examples linked to here.

[agriyakhetarpal]

Good idea! I had added an end-to-end example at the end at https://jupyterlite.github.io/sphinx-demo/ and linked a few PRs on the implementation, but it does make sense to add a few examples from NumPy and scikit-learn up at the top as well.

We could then even go a step further and in the section(s) on "how to do this for your project" link to how things are implemented for the examples linked to here.

Makes sense to me!

[betatim]

Super cool to see this!

One feedback on the organisation: I started reading the SPEC and only after covering a lot of "stuff" did we get to a section that explains to the reader how to do this for their project. Can we somehow change the order? I think most readers will be looking for information on "is this interesting/cool to do?" (we can answer this with screenshots and linking them to existing examples) and then if they want to do this the next question is "How do i do this for my project?". I think only after that will people worry about things related to CI, etc Or put differently: I think we should try and make it as easy as possible to get people started (and then hooked on :D) having interactive docs. Even at the cost of then, once they are hooked, "disappointing" them with "well, actually, it isn't all as easy as we made it look at the start". What do you think?

Completely different topic: how popular is mybinder.org for interactive docs that don't work in jupyterlite? I think for most use-cases it isn't needed, but maybe someone who is a bit more plugged in to this has an idea. We still use it for scikit-learn, but even I mostly use the jupyterlite badge these days

[agriyakhetarpal]

Thanks for the nice initial feedback, @betatim!

One feedback on the organisation: I started reading the SPEC and only after covering a lot of "stuff" did we get to a section that explains to the reader how to do this for their project. Can we somehow change the order? I think most readers will be looking for information on "is this interesting/cool to do?" (we can answer this with screenshots and linking them to existing examples) and then if they want to do this the next question is "How do i do this for my project?". I think only after that will people worry about things related to CI, etc

Yes, this is something we've considered, and it raises the question of the audience/readers for these SPECs. I might be wrong here: while these have been meant for maintainers and developers of core projects who would like to know the messy aspects upfront before deploying anything, I've seen recent shifts in the marketing of these SPECs as they are also being written for projects outside of the core ecosystem, where the assumption that their package is already in Pyodide/emscripten-forge (like it is for most core projects) may not hold and they have to go through these steps anyway.

I agree with your comment, though. I think an "Overview" section with a few live examples sounds best here (and we can move the NumPy example screenshots within that section). It should also be possible to include a notebook from the scikit-learn documentation within the page in an iframe if that helps :)

Or put differently: I think we should try and make it as easy as possible to get people started (and then hooked on :D) having interactive docs. Even at the cost of then, once they are hooked, "disappointing" them with "well, actually, it isn't all as easy as we made it look at the start". What do you think?

💯!

Completely different topic: how popular is mybinder.org for interactive docs that don't work in jupyterlite? I think for most use-cases it isn't needed, but maybe someone who is a bit more plugged in to this has an idea. We still use it for scikit-learn, but even I mostly use the jupyterlite badge these days

I've never actually used Binder either 😄 However, I imagine that server-side computation for interactive documentation, as opposed to its client-side counterpart, also has a fair bit of users¹. Besides direct usage, Sphinx-Gallery offers the Binder badge in the sidebar that you've mentioned of course, and there are a few projects that that were built with similar goals in mind: https://github.com/executablebooks/sphinx-thebe and https://github.com/jupyter-book/thebe (the latter also has a thebe-lite package that can work with Pyodide, but I haven't used it). So while I don't have a clear answer to this as I've never run any metrics to find out (and I don't think Plausible Analytics tracks the /lite/lab/ pages for the JupyterLite case, does it? ²), I think we should refrain from mentioning anything more about it besides the one sentence we have added in the "Runtime considerations" section, and our stance is going to be limited to Sphinx (see also: scientific-python/summit-2025#25 (comment)). We could also consider renaming the SPEC to "WebAssembly-powered Interactive Documentation" if that serves the topic better, but I can guess that the consensus would be that no one would like hosting a server for this and utiling WebAssembly for this would be the norm in the coming time as tooling matures. :)

Footnotes

1. Maybe someone from the mybinder.org or the https://2i2c.org/ teams could help get us some analytics here :D ↩

2. Note to self: check this and see if we can include this functionality in JupyterLite itself, also xref: https://github.com/jupyterlite/jupyterlite/issues/1637 ↩

[stefanv]

Thanks for the time you are investing in the ecosystem, @agriyakhetarpal!

I haven't had a chance to look at the PR, but just wanted to drop a link to the SPEC Purpose & Process document, which is important background reading for all SPEC authors.

[agriyakhetarpal]

Thank you, @stefanv ! I can confirm that I've read through it – I did so before I opened the PR. 😃

I was also wanting to open a post on the Discourse forum as per https://scientific-python.org/specs/purpose-and-process/#new-spec-proposals, but it turned out that I don't have permissions to do so. Who may I contact to receive them?

[agriyakhetarpal]

This is me: https://discuss.scientific-python.org/u/agriyakhetarpal/

Alternatively, if you or someone else could post it on my behalf, that would also be fine with me. :)

[stefanv]

I was also wanting to open a post on the Discourse forum as per https://scientific-python.org/specs/purpose-and-process/#new-spec-proposals, but it turned out that I don't have permissions to do so. Who may I contact to receive them?

As long as you are logged in, you should be able to post there:

Can you take another look, and let me know?

[agriyakhetarpal]

Thanks! Don't mind me: I was looking at https://discuss.scientific-python.org/c/specs/6, which is restricted, but https://discuss.scientific-python.org/c/specs/ideas/9 is open for me. On it!

[betatim]

As a project maintainer I enjoy SPECs or guides that directly show me what I need to do to implement them. Not sure how typical I am but I find often that I read a SPEC or a guide once I am already convinced that something is a good idea and the open question is: how much work is this going to be and how easy is it to maintain? To answer this I look at examples like the jupyter lite one, but then also "real world" examples like "can I work out how much stuff Numpy had to do to get this?". This is why I'd vote for putting such information early.

Maybe I should have added it in my first comment: I helped build and run mybinder.org for a few years, so I am biased about it :D - things like thebe were made with interactive docs in mind. But after thinking about all this, I think the SPEC is already quite long and useful(!) so maybe it is smarter to not make it longer. Instead there could be a new SPEC about things like thebe and binder, if there is enough demand.

[rgommers]

One feedback on the organisation: I started reading the SPEC and only after covering a lot of "stuff" did we get to a section that explains to the reader how to do this for their project.

Yes, this is something we've considered, and it raises the question of the audience/readers for these SPECs. I might be wrong here: while these have been meant for maintainers and developers of core projects who would like to know the messy aspects upfront before deploying anything

I think @betatim is right. High-level picture and happy-path-first matter. Too much "messy aspect" stuff first is almost never the right thing to do, it will be confusing to the reader new to the topic - try to defer as much as possible to later in the document, to an appendix, or to an external link.

[Carreau]

Suggested change

	- "Matthias Bussionnier <[email protected]>"
	- "Matthias Bussonnier <[email protected]>"

[willingc]

Overall, I like this SPEC. I would go a step further to include reference to Binder.

[willingc]

I wonder if it makes sense to mention Binder here since it serves a similar purpose for the reader for projects where pyodide does not support the library.

[willingc]

Perhaps consider a more specific name than Interactive Documentation. This SPEC focuses on WebAssembly enabled interactive documentation. Binder offers a way to serve interactive documentation when the pyodide/emscriptem approach is unable to support a scientific library or an example with dependencies not supported by wasm. If leaving the title as Interactive Documentation, I think it would be helpful to include a reference to Binder as an alternative when wasm is unsuitable for an example.

[willingc]

@agriyakhetarpal As an FYI, here are some brief Binder resources:

• https://jupyter.org/binder
• https://blog.jupyter.org/mybinder-org-serves-two-million-launches-7543ae498a2a
• https://doi.org/10.25080/Majora-4af1f417-011