Proof of concept: Rendering docs with mdBook

[neongreen]

Just checking that mdbook works.

Includes a version selector.

How to run:

brew install mdbook    # or: cargo install mdbook (untested)
mdbook serve

Most of the theme/ files can be removed, except for index.hbs. We might want to make this a plugin or smth so that we don't have to track templates from the official mdbook. Or we can inject the element dynamically and then we don't need to vendor any theme files at all, just set up output.html.additional-js = ....

a few tasks

• Do contributors actually have a problem with Python? Ie. do we have people who a) would've contributed to docs if not for Python, or b) limit themselves to minor edits b/c of Python?
• Does mkdocs have any nice perks we would lose?

[neongreen]

this is https://lit.dev, a web components framework from Google -- we can vendor it if necessary, but idk if there's a good reason for it.

React, Preact, etc would also work, but I didn't want an extra transpilation step for .jsx. With Lit, I don't need any JS tools on my machine at all.

[neongreen]

This file is currently generated by mike. We'd have to find another way to update it. Luckily the format is super simple:

[
  {
    "version": "prerelease",
    "title": "prerelease",
    "aliases": []
  },
  {
    "version": "v0.23.0",
    "title": "v0.23.0",
    "aliases": [
      "latest"
    ]
  },
  {
    "version": "v0.22.0",
    "title": "v0.22.0",
    "aliases": []
  },
  {
    "version": "v0.21.0",
    "title": "v0.21.0",
    "aliases": []
  },
  ...
]

[ilyagr]

I think we'd need a replacement for mike, yes.

This is my main worry about this PR: for a moment, I thought that mkdocs finally implemented a version switcher, but it seems that we'd have to take care of its guts ourselves if we go with mdbook. Perhaps they would accept such tools as part of their project?

[neongreen]

Just for testing; this will be More Proper in the prod version

(I should look at mkdocs's implementation perhaps)

[ilyagr]

If you go to prerelease/bookmarks/, and switch to a pre-bookmarks jj version, will this show an ugly 404?

A lot of the complexity in the mkdocs-material version switcher functionality comes from trying to avoid this. See e.g. squidfunk/mkdocs-material#7338 (which has tests and thus might be slightly easier to follow than what ended up merged).

It is possible that fixing it is not worth the complexity, but then we'd need to make our 404 page much nicer.

(Other than that, it's refreshing to seem how short and nice this can be)

[neongreen]

This is probably far-fetched, but it would be super cool if the examples in codeblocks actually run when generating, so the command output is always up-to-date or is at least tested and fails CI :)

@necauqua btw this is probably possible with mdbook (see https://docs.rs/mdbook-cmdrun/latest/mdbook_cmdrun/), but could be risky for contributors. Idk how to make it non-risky.

[ilyagr]

Thank you for looking into this!

To summarize my view on the state of mkdocs and potential improvements: mkdocs+mkdocs-material is not great; there are three serious problems with it:

1. It requires people to install Python.
2. It uses a really old Markdown parser that is inconsistent and confusing. See docs: minor fix to Markdown formatting #4378 for an example and a further rant.
3. mkdocs-material often breaks, especially its version switcher. Fixing these bugs can take time, and I'm not sure how well-supported it's going to be in the future. When I contributed a fix to one of the version switcher bugs, the maintainer refused to take the tests I wrote for it, so I rue the day something breaks there again. (OTOH, since the development seems to have gotten slower, that might take a while). See various PRs linked from Fixes to "staying on the same page functionality" (version with tests removed) squidfunk/mkdocs-material#7559.

I think 1 is fixable relatively easily, by switching to https://github.com/astral-sh/uv, which can install Python itself and has had all the features we need for a few months now. So, the instructions for contributors would be:

• Obtain uv (install, or download a binary, or compile with cargo -- though it's not on crates.io, so I'd need to find the correct command to put in the docs)
• Do uv run -- mkdocs serve in the root of the jj repo. uv will install Python if needed together with other dependencies, no need to worry about it.

I have a draft for the uv switch, I just hadn't had a good reason to finish up. I forget what state that branch is in, but you can play with uv and the pyproject.toml/uv.lock from https://github.com/ilyagr/jj/tree/uv. (I haven't rewritten the docs there to be about uv rather than rye, which uv replaces; one reason for this is that I think how the docs are written is one of the most important parts so I left it for later). An annoying aspect of finishing that up (or this PR for that matter) is changing the GitHub actions and double-checking they still work, the release action especially (I end up making releases on my fork to test).

---

This, however, would not fix 2 or 3, which feel more serious. For those, I was considering taking a look at https://docusaurus.io, which seems to use a modern Markdown parse and has a version switcher. I've seen some new projects using it. One example: https://ki-editor.github.io/ki-editor/docs/introduction (it uses some interactive features we wouldn't need, at least at first, but it has a nice simple theme. Docusaurus docs themselves are an example with a version switcher)

I am hoping that Docusaurus has some replacement for mike, but I haven't looked deeply enough into how functional it is.

Docusaurus in the Node ecosystem, which could make 1 a problem again, but maybe we could have it run in Deno or Bun. Those are as easy to install as uv. Update: There is no official support for Deno or Bun :(. Deno has some bugs for supporting Docusaurus that are marked fixed. Bun has oven-sh/bun#3426. So, we'd need to test further, and even if it works now, it might not in the future.

Update: Or maybe there is some uv-like tool for Node.

Update 2: On Discord, @chriskrycho mentioned the Vue.js docs as a nice example. They seem to use https://github.com/vuejs/vitepress , which looks very similar to Docusaurus from a distance, but might have other pros/cons. On second look, they don't seem to support versioning though.

Aside on versioning: wanting a publishing system with versioning restricts us a lot. If this is really a problem, we could switch to just having two docs websites: on for prerelease version and one for latest stable version. Still, I'd try Docusaurus first. Also, that'd break a lot of our URLs.

---

I have not myself seriously considered adding a version switcher to MkBook, that (together with making a replacement for mike) seemed like a lot of work, but it's worth discussing also, and might be less work than I imagined.

[chriskrycho]

In Node, as long as you have Node on your system, you can use npx, which is sort of the (much slower and less awesome) Node inspiration for uv run (and, I believe, for pipx which preceded it). It still has the issue of requiring you to have Node on your system to start, though, whereas you can brew install uv and not have to think about managing Python versions at all.

[ilyagr]

It still has the issue of requiring you to have Node on your system to start

Yes, I wonder if they have a solution to it. Now that uv exists, perhaps somebody will get inspired.

Actually, node can be installed by uv via https://pypi.org/project/nodeenv as I discovered from https://squidfunk.github.io/mkdocs-material/customization/#environment-setup. This is a bit awkward, but could be good enough in practice.

[neongreen]

Docusaurus does versioning by copying past versions into the repo, see .../versioned_docs/ in their repo (eg. https://github.com/facebook/docusaurus/pull/10000/files)

Aside on versioning: wanting a publishing system with versioning restricts us a lot.

I feel like it's the version switcher specifically that limits us -- otherwise you're free to pick any static site generator and serve the list of folders. It covers the usecase of "I'm stuck on an old version and I need docs for that" (Docusaurus does this with older 2.x versions). I suppose we'd also need a bit of logic to inject "this version is old" banners into past docs.

[neongreen]

I just remembered that Proto exists: https://moonrepo.dev/docs/proto

[neongreen]

Yeah, Proto turns out to be neat.

1. It can get node, pnpm, and make sure deps are built before starting docusaurus
2. It is also a generic task-runner with deps, so we can express things like "docs depend on cli/*.rs, so if those change, the clap snapshot task has to happen first"

[neongreen]

I'm going to close this PR because a) a combination of #4822 and #4829 seems good enough, and b) I couldn't find almost any mdbook customization in the wild, which isn't a good sign