Problem
The generated pkgdown site in docs/ is a committed, tracked build
artifact, but the review workflow has no defined point at which it is
rebuilt. This creates two recurring problems during a review:
-
The committed site goes stale. Per-issue PRs change documentation
(rename a dataset, move a vignette, add a Download section), but the
committed docs/ HTML is not regenerated, so the published site
contradicts the package until someone rebuilds it manually.
-
Rebuilding per issue causes toolchain churn. If a reviewer does
rebuild docs/ inside a per-issue PR, the local pkgdown/bootstrap
version usually differs from whatever built the committed site, so the
PR picks up a whole new docs/deps/bootstrap-x.y.z/ tree and rewrites
every HTML asset reference. That churn is unrelated to the issue's
content and pollutes the diff.
There is no guidance telling per-issue reviews to leave docs/ alone, and
/review-complete has no step that rebuilds the site once at the end.
Its PR body has a ## Final Checks line "Documentation and website build
successfully", but nothing in the skill actually builds the site.
Evidence from a live review
During a four-area review of a package (metadata, data, docs), this
happened concretely:
- The data-area PR renamed the
codebook dataset to dictionary. The
correct source changes were made, but rebuilding docs/ would have added
a bootstrap version bump and other toolchain churn, so the reviewer
reverted docs/ and flagged a follow-up.
- Two PRs later, the committed
docs/reference/ still contained
codebook.html and codebook.md (pages for a dataset that no longer
exists), and had no dictionary pages. The site was stale for the rest
of the review.
- The docs-area PR (which moved a vignette to
vignettes/articles/, added a
Download section, completed _pkgdown.yml) hit the same wall: rebuilding
mixed necessary content with a pkgdown 2.2.0 / bootstrap 5.3.8 bump
against the committed 5.3.1 site. Reverted again, flagged again.
So the reviewer had to flag "rebuild the site" as a follow-up in every area,
and the stale pages persisted across the whole review.
Proposed change
-
Per-issue reviews must not rebuild or commit docs/. Add a line to
the /review-issue skill (and/or the docs checklist) stating that the
generated site is rebuilt once at the end, not per issue, and that
per-issue PRs should revert any incidental docs/ changes and note the
pending rebuild. This keeps per-issue diffs to source files and avoids
stale-vs-churn whipsaw.
-
/review-complete gains an explicit single site-rebuild step, run
from the final dev state after all four areas are merged, before the
dev-to-main PR. Concretely: run pkgdown::build_site(), commit the
regenerated docs/ (removing any orphaned pages such as a renamed
dataset's old reference page), and confirm it built without errors.
Because it runs once from the finished state, the site is internally
consistent and there is no per-issue toolchain churn.
-
Enabling GitHub Pages should be called out as a maintainer action in
/review-complete (or the docs checklist), since the "website published
on GitHub Pages" required item cannot be satisfied by rebuilding docs/
alone and an agent cannot change repo settings.
Note on the rebuild toolchain
A local pkgdown::build_site() in /review-complete still inherits
whatever pkgdown/bootstrap version the reviewer has installed, so the
committed docs/deps/ version can drift between reviews. A follow-up worth
considering is building/deploying the site from a pinned pkgdown GitHub
Actions workflow (and possibly not committing docs/ at all), which would
remove the toolchain-drift problem entirely. Out of scope for the immediate
fix, but related.
Problem
The generated pkgdown site in
docs/is a committed, tracked buildartifact, but the review workflow has no defined point at which it is
rebuilt. This creates two recurring problems during a review:
The committed site goes stale. Per-issue PRs change documentation
(rename a dataset, move a vignette, add a Download section), but the
committed
docs/HTML is not regenerated, so the published sitecontradicts the package until someone rebuilds it manually.
Rebuilding per issue causes toolchain churn. If a reviewer does
rebuild
docs/inside a per-issue PR, the local pkgdown/bootstrapversion usually differs from whatever built the committed site, so the
PR picks up a whole new
docs/deps/bootstrap-x.y.z/tree and rewritesevery HTML asset reference. That churn is unrelated to the issue's
content and pollutes the diff.
There is no guidance telling per-issue reviews to leave
docs/alone, and/review-completehas no step that rebuilds the site once at the end.Its PR body has a
## Final Checksline "Documentation and website buildsuccessfully", but nothing in the skill actually builds the site.
Evidence from a live review
During a four-area review of a package (metadata, data, docs), this
happened concretely:
codebookdataset todictionary. Thecorrect source changes were made, but rebuilding
docs/would have addeda bootstrap version bump and other toolchain churn, so the reviewer
reverted
docs/and flagged a follow-up.docs/reference/still containedcodebook.htmlandcodebook.md(pages for a dataset that no longerexists), and had no
dictionarypages. The site was stale for the restof the review.
vignettes/articles/, added aDownload section, completed
_pkgdown.yml) hit the same wall: rebuildingmixed necessary content with a pkgdown 2.2.0 / bootstrap 5.3.8 bump
against the committed 5.3.1 site. Reverted again, flagged again.
So the reviewer had to flag "rebuild the site" as a follow-up in every area,
and the stale pages persisted across the whole review.
Proposed change
Per-issue reviews must not rebuild or commit
docs/. Add a line tothe
/review-issueskill (and/or the docs checklist) stating that thegenerated site is rebuilt once at the end, not per issue, and that
per-issue PRs should revert any incidental
docs/changes and note thepending rebuild. This keeps per-issue diffs to source files and avoids
stale-vs-churn whipsaw.
/review-completegains an explicit single site-rebuild step, runfrom the final
devstate after all four areas are merged, before thedev-to-main PR. Concretely: run
pkgdown::build_site(), commit theregenerated
docs/(removing any orphaned pages such as a renameddataset's old reference page), and confirm it built without errors.
Because it runs once from the finished state, the site is internally
consistent and there is no per-issue toolchain churn.
Enabling GitHub Pages should be called out as a maintainer action in
/review-complete(or the docs checklist), since the "website publishedon GitHub Pages" required item cannot be satisfied by rebuilding
docs/alone and an agent cannot change repo settings.
Note on the rebuild toolchain
A local
pkgdown::build_site()in/review-completestill inheritswhatever pkgdown/bootstrap version the reviewer has installed, so the
committed
docs/deps/version can drift between reviews. A follow-up worthconsidering is building/deploying the site from a pinned pkgdown GitHub
Actions workflow (and possibly not committing
docs/at all), which wouldremove the toolchain-drift problem entirely. Out of scope for the immediate
fix, but related.