Repository navigation
[INFRA] Move CI to GitHub Actions, modernize the docs build, and proofread the handbook - #33
Conversation
Travis (node 10, remark 5) and CircleCI (python 3.7.5, pipenv, mkdocs 1.0.4) both pinned toolchains that no longer run, and the CircleCI changelog bot never produced a commit -- it referenced a src/pregh-changes.md that does not exist. Replace both with two GitHub Actions workflows: - ci.yml: lint Markdown with remark, build the site with `mkdocs build --strict`, upload it as an artifact, and check that no internal link is broken. - linkcheck-external.yml: the external link check, on a weekly schedule and on demand, so that a third-party outage does not block a PR. Dependencies: - package.json/package-lock.json replace npm-requirements.txt and pin remark-cli 12 with the current lint presets. - requirements.txt pins mkdocs 1.6.x and mkdocs-material 9.x (below MkDocs 2.0, which drops the plugin and theming systems the handbook relies on). Pipfile/Pipfile.lock, which only CircleCI used, are gone. - readthedocs.yml becomes .readthedocs.yaml on ubuntu-24.04/python 3.12, and only requests the htmlzip format, which is the only extra format produced for MkDocs projects. Site configuration: - The theme_customizations/ footer override was written against Material 4 and its markup (md-footer-nav, md-flex) no longer exists in Material 9, so it rendered unstyled. Drop it and use the `copyright` setting plus the `navigation.footer` feature instead. - Add site_url/repo_url/edit_uri and a light/dark palette toggle. - Wire computing/discovery-jupyter.md, computing/kerberos.md, computing/troubleshooting.md, index.md and CODE_OF_CONDUCT.md into the nav; they existed but were unreachable. - Drop the unused jQuery bundle and the neurodocker script that built the patched linkchecker image for the retired CircleCI job. .remarkrc: `list-item-indent` was set to "tab-size", which is not a valid value for the current rule and demanded a 4-space indent the handbook has never used; set it to "one". Turn off no-file-name-irregular-characters, which only flagged CODE_OF_CONDUCT.md and additional_resources.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
Typos and wording: "cluser" -> "cluster", "gift the the" -> "gift to the", "cary" -> "carry", "anyting" -> "anything", "succesful", "resonanse", "Some times", "reran", "verison", "get fetch upstream", "`-a-`", "Old fashion way", "may be useful" and similar; agreement and tense fixes throughout; gendered "himself"/"him" for a study subject replaced with neutral wording. Links: the README pointed at a DBIC_logo/ directory that never existed in the repository and at Travis/CircleCI badges -- point it at src/images/logo.png and the new workflow. The CC0 row in the licenses appendix showed an Open Data Commons URL as the text of a Creative Commons link. Release_Guideline.md linked bids-validator to a non-existent dbic/ fork and to dbic.readthedocs.io rather than dbic-handbook.readthedocs.io; Release_Protocol.md pointed at the wrong Read the Docs project slug; a .bashrc comment in the Discovery page pointed at an anchor that does not exist. reproin.md carried a duplicate ///dbic/QA definition and an unresolved empty link. Rendering: several blocks indented by 2 spaces (the heudiconv meta-study example, the ssh config in kerberos.md) rendered as prose rather than as code; indented code blocks are now fenced and tagged with a language. `**Step N: ...**` paragraphs in discovery.md and the bulleted "steps" in discovery-jupyter.md become real headings, and heading levels no longer skip a rank. data-paper.md and several section pages gained the H1 they were missing. CONTRIBUTING.md now describes the GitHub Actions workflow and the `npm install` / `npm run lint` / `npm run fix` cycle instead of Travis and `cat npm-requirements.txt`; the PR template points at the `site` artifact instead of the CircleCI preview. `npm run lint` reported 219 warnings before this pass and reports none after it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
Content fidelity: - reproin.md: restore the `_ses-+` / `_ses-=` session markers. The earlier pass read "or just say to maintain, create (starts with 1)" as garbled filler and deleted it, but it documents two implemented features of the heuristic (`infotoids`: `+` allocates the next session numbered from 001, `=` reuses the previous one). Deleting them turned a three-way choice into a false either/or on a page operators follow at the console. Also restore upstream's "session" wording, correct "known to BIDS" to "known to ReproIn" (mrs/BEP022 is not finalized), and stop asserting the heuristic was "developed within" HeuDiConv. - datalad.md, discovery.md: both new cross-references pointed at the ACL background section, which contains no git configuration. The text they replaced meant "Installing data" (steps 1-3) -- commit 8149457 added it under a heading literally named "Discovery filesystem". - datalad.md: the ReproNim/containers anchor is `#a-typical-yoda-workflow`; the previous edit wrapped the pre-existing broken `#a-typical-workflow` in confident link text while containers.md used the correct one. - discovery.md: `> 0.19.3` had been widened to "at least 0.19.3". - index.md, troubleshooting.md: restore "unfortunate", "smited", "1 crazy tip" and "The Great and Evil ACL". The linter never asked for those. - datalad.md: note that Discovery needs `-J1`, contradicting the `-J4` example now that this page links there directly. Site configuration: - site_url lacked Read the Docs' `/en/latest/` prefix, so every published page advertised a canonical URL that 404s, and sitemap.xml and 404.html were wrong the same way. Now taken from READTHEDOCS_CANONICAL_URL so it stays correct per version. - MkDocs 1.6 returns the existing `File.page` when a file is reused in nav, discarding the later title, so all eleven placeholder entries rendered as "PyBIDS" and "Passwordless SSH" never appeared. Give each placeholder its own stub under src/todo/ and list kerberos.md once. - The `copyright` string replacing the old footer override had lost the GitHub hyperlink, leaving an instruction with nothing to click. - Enable pymdownx.highlight: the newly fenced code blocks emitted Pygments spans with no `.highlight` wrapper, which Material never styles. - Enable the privacy plugin so mermaid and the web fonts are self-hosted rather than fetched from unpkg.com on a floating major tag at page load. - Invert the CONTRIBUTING/CODE_OF_CONDUCT symlinks so the edit-this-page links resolve to real files instead of corrupting a symlink. - Add a validation block: --strict alone does not fail on broken anchors or pages dropped from nav, the two classes of bug fixed by hand here. - Drop the %23 from a shields.io badge URL; self-hosting turned it into a literal fragment and broke the image. CI: - linkcheck-external could never pass: LinkChecker exits non-zero on warnings alone, and the site produces ~30 redirect warnings. Add --no-warnings and skip the canonical URLs, which do not exist until a version is published. - Replace the `site/*.html site/*/*.html` and `src/*.md src/*/*.md` globs, which silently checked fewer files as soon as a page went three levels deep, with recursive traversal. - Pin linkchecker, add if-no-files-found: error, add timeouts, run CI on rel/* pushes as Release_Protocol.md assumes, and stop cancelling in-progress runs on the default branch. - Wire up remark-preset-lint-recommended, which was declared but unused. - Drop references to files this branch removed from tools/dbicotomize_bids. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
The Mailman page at dbic.dartmouth.edu 404s; the list now lives at mrusers@groups.dartmouth.edu. That subdomain has an MX record pointing at Microsoft 365 but no A record, so it is mail-only -- there is no listinfo or archive page to link to. Use a mailto: instead, and spell the address out in CONTRIBUTING.md so it is readable without hovering the link. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
… moves
Discovery runs Slurm, not PBS/Moab. src/computing/discovery-jupyter.md still
documented `#PBS` directives, `mksub`, `myjobs` and `qdel`, none of which
exist on the cluster any more, so the procedure could not be followed at all.
Replace them with `sbatch`, `squeue --me` and `scancel`, and translate the
submission script: `-l walltime=` becomes `--time=`, `-l nodes=1:ppn=1`
becomes `--nodes=1 --ntasks-per-node=1`, `-l feature=bigmem` becomes an
explicit `--mem=8G`, and the `<job_name>.[oe]<job_ID>` pair becomes a single
`--output=jupyter-notebook-%j.out`. The script is syntax-checked with
`bash -n`. A short note records the old command names so that anyone
following older instructions can tell what changed.
Research Computing also now runs Open OnDemand, which submits the job and
proxies the notebook without a password file or an ssh tunnel, so lead with
that and keep the manual route for cases the OOD form does not cover. Both
routes need the Dartmouth network or VPN, which the page did not say.
rc.dartmouth.edu reorganized from /index.php/... to /hpc/...; update the
three dead links to the pages that carry the same content, verified 200 with
matching titles:
/index.php/discovery-overview/ -> /hpc/discovery-overview/
/discovery-overview/accessing-the-cluster -> /hpc/intro-to-hpc/logging-into-the-cluster/
/index.php/using-discovery/scheduling-jobs -> /hpc/intro-to-hpc/submitting-a-batch-job/
The 2019 Intro_to_Cluster.pdf has no replacement; point that TODO at
/hpc/intro-to-hpc/ instead.
ood.dartmouth.edu resolves only inside Dartmouth's network, so the external
link check can never reach it from a hosted runner; add it to the ignore list.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
The ~/.ssh/config in this section only does anything if the ssh binary was built against GSS-API; otherwise the GSSAPI* options are ignored and ssh falls back to a password prompt, which is indistinguishable from Kerberos being misconfigured. Say so, give `ssh -Q kex | grep gss` as the check, and point Debian users at openssh-client-gssapi. Debian introduced openssh-client-gssapi in openssh 1:9.8p1-5. It is still an empty package depending on openssh-client in Debian 13 (trixie), and its own description says future releases will remove GSS-API support from openssh-client, so installing it now is what future-proofs the setup. Nested lists were rendering flat, which this change also fixes. MkDocs uses Python-Markdown, which needs 4-space indentation for the content of a nested list item; the earlier style pass reformatted these to CommonMark's 2 spaces, so every nested list on the site collapsed into its parent. kerberos.md and data-paper.md regressed in that pass; reproin.md and stimuli.md were already flat before it. Re-indent all four to 4 spaces and turn off the two remark rules that mandate the CommonMark spacing, since the renderer, not the linter, decides what actually reaches readers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
Revert the CONTRIBUTING/CODE_OF_CONDUCT symlink inversion. GitHub does not render symlinked markdown -- it serves the link target as an 18-byte file -- so putting the symlinks at the repo root would have turned the two documents the README, the pull request template and GitHub's own community-health checks point at into stubs. Real files go back at the root. That inversion existed to keep the edit-this-page links off symlinks, so drop content.action.edit and edit_uri instead; the header repository link still gets readers to GitHub. Finish the nested-content indentation fix. Python-Markdown needs four spaces for continuation paragraphs too, not just nested lists, so the issue-label list in CONTRIBUTING.md was still rendering as three one-item lists with the explanations dedented out of their bullets. Record the four-space rule in CONTRIBUTING.md, since neither remark nor mkdocs --strict can catch it. Drop `npm run fix`. The invocation was wrong (`remark --output src/` waits on stdin forever), and corrected it rewrites every bullet to `*` and re-flattens the nested lists, which then fails `npm run lint`. Explain what to do instead. Replace `ssh -Q kex | grep gss` with `ssh -G <host> | grep -i gssapiauthentication`. GSS-API key exchange is a Debian/Fedora patch that upstream OpenSSH does not carry, so the old check reports a missing feature on Apple's and Homebrew's ssh, where GSSAPIAuthentication works fine -- and that page tells macOS users to `brew install krb5` two steps earlier. Also stop implying bookworm needs openssh-client-gssapi: it still ships GSS-API in openssh-client. Use discovery.dartmouth.edu, matching Research Computing's current docs and the ~/.ssh/config in the Kerberos section, which the jupyter page contradicted with discovery7. Export XDG_RUNTIME_DIR so it reaches the server, drop a `sleep` that could never run after a blocking server, simplify the port arithmetic, and stop calling s01 a scheduling node -- it is a compute node in the standard partition, as the page's own squeue example shows. Keep img.shields.io and i.imgur.com out of the privacy plugin's fetches. A failed fetch is a warning, and --strict and fail_on_warning promote that to a failed build, so decoration on one page should not gate CI or a release; cache .cache so the fetches that remain are not repeated on every run. Remove the README image: src/images/logo.png is byte-identical to favicon.png, 32x32, and was being displayed at 600px. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
|
The red check on this PR needs a maintainer action outside the repository, so flagging it rather than working around it.
That is this PR's doing: it deletes The alternative, if you would rather not touch CircleCI yet, is to keep a stub Everything the PR does control is green on
Also worth noting, since the PR description lists it as a gap: Read the Docs pull-request builds are already enabled, so the rendered preview CircleCI's artifact used to provide is not lost after all — this PR renders at https://dbic-handbook--33.org.readthedocs.build/en/33/ . That build also confirms the Generated by Claude Code |
DBIC is a Dartmouth facility rather than a standalone open-source community, so a project code of conduct is the wrong instrument here. The file was also inherited verbatim from bids-specification and still routed harassment reports to a Stanford address, which is worse than having nothing. Remove CODE_OF_CONDUCT.md and its nav entry, and have README.md and CONTRIBUTING.md cite Dartmouth's Nondiscrimination and Anti-Harassment Policy instead, with the channels that policy names: the Office of Equal Opportunity, Accessibility, and Title IX (EOATIX), the Title IX reporting form, and the anonymous Integrity Helpline. Office name, phone number and location are quoted from the policy page itself; all three URLs return 200. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
The CircleCI configuration ended with four jobs that generated a changelog,
ran remark over it, and pushed the result back to master. They were dropped
along with the rest of the file; port them instead.
Two defects kept that chain from ever producing a commit, and both are fixed
here rather than carried over:
- --header-label Changelog emitted the title as a paragraph rather than a
heading, so it is now --header-label '# Changelog'.
- "remark --output" followed by "remark --frail" could not converge: the
writer emits '*' bullets and our style guide requires '-', so the check
always rejected the file the format step had just written. The bullet
style is now passed explicitly.
The --base src/pregh-changes.md the generator was pointed at has never
existed in this repository, so it is gone rather than ported, and the job
runs on the built-in GITHUB_TOKEN instead of a stored token with push access
to master.
src/CHANGES.md is regenerated here so the page carries the merged pull
requests rather than an empty stub.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
- Grant issues:read and pull-requests:read. A permissions block sets every unlisted scope to none, and the generator fetches /issues and /pulls even with --no-issues, which only filters what it already fetched. The job would have died with a 403 on its first run. - Restore the REL: half of the CircleCI guard. Release_Protocol.md asks the maintainer to curate src/CHANGES.md by hand for a release; regenerating on the release commit would revert that immediately. - Do not run from a fork, and do not run off master. workflow_dispatch can be started from any branch, and the push names master explicitly, so a dispatch from a feature branch could have fast-forwarded master onto it. - Rebase and push in step with each other, under set -euo pipefail. The old loop rebased a third time with no push left to make, and relied on an unstated shell flag: without errexit a conflicted rebase leaves the commit behind and the push reports "Everything up-to-date" and succeeds. A conflicting rebase is now aborted and reported. - Pass --no-stdout to the lint check so it stops printing the whole document. Release_Protocol.md gains a note that the generator owns the file between releases, since curation only survives until the next merge into master.
reproin.md: the modality list was reworded in this branch from "Known to
BIDS modalities are" to "The modalities known to ReproIn are", which turned
a vague statement into a checkable one that is false. Upstream
heudiconv/heuristics/reproin.py has
KNOWN_DATATYPES = {"anat", "func", "dwi", "behav", "fmap"}
and no occurrence of "mrs" at all; anything outside that set is warned about
and dropped by parse_series_spec(). So add the missing behav, and say
plainly that upstream does not convert mrs, which the MRS samples further
down the page otherwise imply that it does.
discovery-jupyter.md: the comment added in this branch claimed "module load
python" is what provides jupyter and cited two Research Computing pages as
evidence. Neither documents a python module -- they demonstrate
"module load R/4.1.2" and conda environments -- and our own discovery.md
uses a specific name, python/3.7-Anaconda-datalad. Point at
"module avail python" instead of asserting a module that may not exist.
Also fix a contradiction on the same page: step 1 ran "jupyter notebook
password" on the login node with nothing loaded, while step 2 loaded an
environment before starting the server. Either jupyter is on PATH and the
job script's load is not what provides it, or it is not and step 1 fails
before the reader reaches step 2. Load the environment in step 1 too, and
say it has to be the same one, since the password is written into that
environment's config.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
Three corrections found in review. The claim that src/pregh-changes.md "has never existed in this repository" is wrong. It was added in ee03765 and deleted one commit later in 1c0797d, both ancestors of master, when this repository was split out of bids-specification in October 2019. The earlier check used git log --all --diff-filter=A -- '*pregh-changes*' which returns nothing: git's default history simplification hides it, and --full-history is needed to see it. Relatedly, a missing --base file was never capable of breaking the changelog chain. github_changelog_generator 1.18.0 guards both uses of it -- generator.rb:62 and generator_tags.rb:105 -- with File.file?, so a path that is not there is silently skipped. Only the remark bullet-style conflict actually failed. Release docs: the handbook link in Release_Guideline.md pointed at /en/stable/, which 404s. The Read the Docs project has exactly one active version, latest, tracking master, so /en/stable/ and the versioned /en/vX.Y.Z/ URLs the protocol tells maintainers to use do not resolve and there is no stable or tag build to trigger. Point the guideline at /en/latest/ and say in the protocol that the version has to be activated on Read the Docs by hand. BEP022 has been merged: MRS is in the BIDS specification as of 1.11.1, so reproin.md no longer calls it unfinalized, and links the specification section instead of only the proposal document. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB
Travis and CircleCI both pinned toolchains that no longer run, so CI is replaced with GitHub Actions and the docs build is moved onto current MkDocs 1.6 / Material 9. Along the way: a typo and wording pass over the handbook, several broken or moved links fixed, and the Jupyter-on-Discovery page rewritten for Slurm because it still documented PBS.
Locally verified:
npm run lint,mkdocs build --clean --strict, and the internal link check all pass. Rendered preview: https://dbic-handbook.readthedocs.io/en/latest/ (per-PR build: https://dbic-handbook--33.org.readthedocs.build/en/33/)CI and build
.github/workflows/ci.ymllints Markdown with remark, builds withmkdocs --strict, uploads the site as an artifact, and checks internal links.linkcheck-external.ymlchecks external links weekly and on demand, so a third-party outage does not block a PR.github-changelog-generator→remark→Changelog-bot) is ported to.github/workflows/changelog.ymlrather than dropped: a push tomasterregeneratessrc/CHANGES.mdand commits it back. It never once produced a commit under CircleCI, because theremark --output/remark --frailpair could not converge — remark's writer emits*bullets while the style guide requires-, so the check always rejected the file the format step had just written. The port fixes that with an explicit--setting "bullet: '-'", and runs on the built-inGITHUB_TOKENinstead of a stored token with push access tomaster. Two smaller things are also corrected:--header-labelnow carries the#that left the title as a paragraph rather than a heading (cosmetic — it linted clean), and the--base src/pregh-changes.mdis not ported, since that file was deleted in October 2019 when this repository was split out of bids-specification and the generator silently ignores a--basepath that is not there.package.json+ lockfile replacenpm-requirements.txt;requirements.txtpins mkdocs>=1.6,<2and mkdocs-material>=9.5,<10.Pipfile(CircleCI-only) is gone.readthedocs.ymlbecomes.readthedocs.yaml.theme_customizations/partials/footer.htmlwas written for Material 4 and its classes (md-footer-nav,md-flex) have no CSS in Material 9, so it rendered unstyled. Replaced with thecopyrightsetting plusnavigation.footer. The unused jQuery bundle is removed.site_urlnow readsREADTHEDOCS_CANONICAL_URL.masterset nosite_urlat all, so no canonical URL was published before this PR; the point of the environment variable is that Read the Docs serves under/en/<version>/, so a hard-coded root URL would make every page of this PR's preview claim to be the production page.discovery-jupyter.md,kerberos.md,troubleshooting.md,index.md) are wired into the nav, and the eleven placeholder entries get their own stubs — sharing oneTODO.mdmade MkDocs 1.6 render all of them as "PyBIDS".Content
bids-validatorlink pointing at a non-existentdbic/fork, and a CC0 row whose link text and href disagreed.rc.dartmouth.edureorganized from/index.php/...to/hpc/...; the dead links are updated to the pages carrying the same content.mrusers@groups.dartmouth.edu, a mail-only domain with no web page, so the old Mailman URL becomes amailto:.discovery-jupyter.mddocumented#PBS,mksub,myjobsandqdel, none of which exist on the cluster now. Rewritten forsbatch/squeue/scancel, and it now leads with Open OnDemand, which submits the job and proxies the notebook without a password file or ssh tunnel.sshitself must be built with GSS-API support, and points Debian users atopenssh-client-gssapi.CODE_OF_CONDUCT.mdis removed. DBIC is a Dartmouth facility rather than a standalone open-source community, and the file was inherited verbatim from bids-specification — it still routed harassment reports to a Stanford address. README and CONTRIBUTING now cite Dartmouth's Nondiscrimination and Anti-Harassment Policy and the channels it names (EOATIX, the Title IX reporting form, the anonymous Integrity Helpline).reproin.md,stimuli.mdandCONTRIBUTING.md: MkDocs uses Python-Markdown, which needs four-space indentation for nested list content and continuation paragraphs where CommonMark accepts two. The style pass early in this branch regressedkerberos.mdanddata-paper.mdthe same way; both are fixed. Written down in CONTRIBUTING.md since no tool in the pipeline catches it.latest.Release_Guideline.mdlinked/en/stable/, which 404s, andRelease_Protocol.mdtold maintainers to triggerstableand tag builds that do not exist; both now say what actually works.Known gaps
discovery-jupyter.md's Slurm directives follow Research Computing's documented examples, but nobody has submitted the job on Discovery yet — worth one real run before anyone relies on it. In particular, the page does not name a specific Python module, because neither RC page documents one; it tells the reader to pick viamodule avail pythonor a conda environment.jupyter notebook password, hash injupyter_notebook_config.json) only works with classic Notebook ≤ 6; Notebook 7 dropped that subcommand and readsjupyter_server_config.json. Left as-is because which version Discovery ships is unknown.KNOWN_DATATYPESdoes not includemrs, so the MRS examples onreproin.mdneed a heuristic that adds it. The page now says so rather than listingmrsas a type ReproIn knows.Intro_to_Cluster.pdfhas no replacement on the new RC site; that TODO now points at/hpc/intro-to-hpc/..github/workflows/from CONTRIBUTING.md 404s until this branch merges, which is expected.Release_Protocol.mdasks for survives the release, but the next ordinary merge intomasterrebuilds the whole file from the pull request titles.Release_Protocol.mdnow says so. Durable curation would need a--basefile, which this repository has not had since 2019.master, the changelog workflow's push will be rejected —GITHUB_TOKENdoes not bypass protection without an explicit Actions bypass in the ruleset.masteris unprotected today.🤖 Generated with Claude Code
https://claude.ai/code/session_01VRhvoqJnGm6JELocnjyPyB