Commit d9f8529
authored
feat(cli): convention-based docs discovery pipeline (#48)
* feat(cli): add intent-skills format field and tanstack intent dep (T001, T002)
Phase 1 of convention-based-discovery-20260409 (#46).
- NpmLockEntry gains optional format?: 'docs' | 'intent-skills' so
intent-format lock entries can be distinguished from docs-format
entries at sync/remove time. Default remains 'docs' for backwards
compatibility.
- Pin @tanstack/intent@0.0.29 as a runtime dep so the upcoming
local-intent adapter can call scanLibrary / findSkillFiles /
parseFrontmatter via its programmatic export.
* feat(cli): add discovery adapter pipeline (T003-T010)
Phase 2 of convention-based-discovery-20260409 (#46).
New `packages/cli/src/discovery/` module implementing the convention
scanner layer that runs before the central registry lookup:
- types.ts DiscoveryResult discriminated union, adapter shapes
- conventions.ts local + repo path tables, exclusion filter
- quality.ts >=3 md files OR >=4 KiB threshold scorer (SC-3 guard)
- local-ask.ts reads package.json.ask.docsPath opt-in
- local-intent.ts wraps @tanstack/intent findSkillFiles/parseFrontmatter
with zod runtime validation on the frontmatter
- local-conventions dist/docs -> docs, README fallback with a warning
- repo-conventions post-tarball scan of github repo trees
- index.ts runLocalDiscovery / runRepoDiscovery orchestrators
Adapters are not wired into the CLI dispatcher yet — that is T013-T014
in the next phase. This commit is pure additive code: build and lint
stay green and no existing behaviour changes.
* feat(cli): dispatch local discovery, intent-skills writer, sync/remove (T011-T013, T015-T016)
Phase 3 of convention-based-discovery-20260409 (#46).
Wires the Phase 2 discovery adapters into the CLI dispatcher and adds
the AGENTS.md intent-skills block writer. T014 (repo-conventions via
github source) is deferred — see Surprises & Discoveries in the plan.
- agents-intent.ts (new)
upsertIntentSkillsBlock / removeFromIntentSkillsBlock manage the
<!-- intent-skills:start --> ... <!-- intent-skills:end --> marker
block on a byte range strictly disjoint from the existing
BEGIN:ask-docs-auto-generated block. Preserves foreign-package
entries on upsert; strips the whole block when the last entry for a
package is removed.
- skill.ts
generateSkill gains an optional GenerateSkillOptions.docsDir. When
set, the skill file references the provided dir in place and omits
the 'When the docs cannot be found' fallback section.
- index.ts
New handleLocalDiscovery helper dispatches DiscoveryResult:
- kind: 'docs' runs the existing ask pipeline with a synthetic
FetchResult (installPath propagated to the lock entry).
- kind: 'intent-skills' records an npm lock entry with
format: 'intent-skills' and upserts the marker block only —
no .ask/docs/ copy, no .claude/skills/ generation.
addCmd.run calls runLocalDiscovery for `npm:` ecosystem specs with
no --source / --docs-path override, before the github fast-path and
the registry auto-detect. On hit, returns early.
removeCmd.run branches on the lock entry's format: intent-skills
entries call removeFromIntentSkillsBlock and drop the lock row;
'docs' entries keep the existing delete path.
runSync adds a second pass that iterates lock entries with
format: 'intent-skills', re-runs localIntentAdapter against each
installed package, and refreshes the marker block + lock.
Build, lint, and the 237 existing tests stay green.
* test(cli): unit coverage for discovery, quality, and agents-intent (T017-T024)
Phase 4 of convention-based-discovery-20260409 (#46).
- test/discovery/quality.test.ts (new, 9 cases)
Covers the SC-3 guard: noise-only (CONTRIBUTING.md + CHANGELOG.md
+ LICENSE) repos score below threshold and fail through. Also
verifies the >=3 count and >=4 KiB byte fallbacks, nested walking,
.mdx support, and the LICENSE/CODE_OF_CONDUCT/SECURITY exclusions.
- test/discovery/adapters.test.ts (new, 12 cases)
localAskAdapter: no manifest -> null, valid ask.docsPath -> docs
result with installPath, broken path -> null.
localConventionsAdapter: dist/docs selection with quality pass,
README fallback when conventions are noise-only, SC-3 noise repo
returns null.
runLocalDiscovery: priority order (local-ask > local-conventions),
explicitDocsPath bypass, missing-package null.
- test/agents-intent.test.ts (new, 12 cases)
upsertIntentSkillsBlock: creates file, is idempotent, preserves
siblings, replaces only target entries, preserves bytes outside
the block, handles scoped packages via load-path prefix match,
escapes double quotes and backslashes.
removeFromIntentSkillsBlock: returns false when absent, strips
only target entries, strips the whole block when the last entry
is removed.
Fixture tasks T017-T020 are satisfied via inline tmp fixtures inside
the adapter / quality tests (bun:test pattern used throughout this
project; no new packages/cli/test/fixtures/ directories). T024
orchestration tests are consolidated into the `runLocalDiscovery`
describe block in adapters.test.ts.
T025 integration: the 237 pre-refactor tests still pass unchanged,
which satisfies SC-4. An end-to-end `ask docs add npm:<pkg>` walk-
through against the new fixtures is deferred to a follow-up.
Build, lint, and all 267 tests (237 pre-existing + 30 new) green.
* chore(cli): add coverage audit script and document discovery pipeline (T026, T029, T030)
Phase 5 closeout of convention-based-discovery-20260409 (#46).
- packages/cli/scripts/audit-coverage.ts (new)
Scaffold for SC-1: walks apps/registry/content/registry/**/*.md,
extracts {owner, repo, npm alias, docsPath} from each entry, and
for every entry with an npm alias runs runLocalDiscovery against
the installed package in node_modules. Emits one JSON line per
entry plus a summary row, exits 1 when coverage < 80 %. The live
run (T027) requires all 37 registry packages installed locally and
is deferred to a follow-up CI job.
- CLAUDE.md
New Gotchas entries describing the discovery pipeline order, the
dual AGENTS.md marker blocks (ask-docs-auto-generated vs
intent-skills), and the NpmLockEntry.format field. Existing
entries about curated-npm strategy and other layers are untouched.
- T029 verification: tsc + eslint + 267 bun:test cases all green
across packages/cli after the full Phase 1-5 landing.
Status of deferred work:
- T014 repo-conventions wiring through GithubSource
- T025 end-to-end ask docs add integration test
- T027 live coverage audit run
- T028 live intent-CLI parity diff
All four are bounded, low-risk follow-ups with the core path in
place. See Surprises & Discoveries in the plan for details.
* test(cli): verify marker isolation between ask-docs and intent-skills blocks
Self-review add-on for convention-based-discovery-20260409 (#46).
Hardens the 'marker isolation' invariant documented in the spec:
neither writer (agents.ts BEGIN:ask-docs-auto-generated nor
agents-intent.ts intent-skills:start) is allowed to touch the other
region. The new case seeds AGENTS.md with a real ask-docs block,
runs an intent-skills upsert + remove cycle, and asserts the ask
block is preserved byte-for-byte across both operations.
* fix(cli): apply gemini review suggestions
Two Important findings from gemini review of PR #48:
- agents-intent.ts readExistingBlock: anchor END_MARKER search after
BEGIN_MARKER. Without the offset, a malformed AGENTS.md with two
intent-skills blocks (e.g. from a failed partial write) could match
an END_MARKER belonging to a different block and produce a corrupt
splice.
- repo-conventions.ts collectDocFiles: add MAX_WALK_DEPTH=20 guard
against symlink loops the tarball extractor failed to resolve and
pathological monorepo layouts. Real docs trees rarely exceed ~6
levels deep, so 20 is both generous and bounded.
Build, lint, and 268 tests still green.
* chore(track): convention-based-discovery-20260409 PR submitted
- Retrospective added to plan
- Track moved active/ -> completed/
- metadata status -> review, pr -> #48
* fix(cli): apply cubic review findings (7 threads)
7 unresolved cubic threads on PR #48, all addressed:
P1 - local-conventions.ts: validate symlink realpath containment
before scoreDirectory walks the candidate. A symlinked
`dist/docs -> /etc` would otherwise let the scorer recurse
out of the package directory before tryLocalRead's later
guard rejected the read.
P1 - index.ts runSync: hoist intent-skills lock-key collection
above the empty-config early-exit so intent-only projects
(no `config.docs` rows) still get their marker block resynced.
P2 - conventions.ts: case-insensitive meta-filename exclusion.
Lowercase `contributing.md` / `changelog.md` were bypassing
the SC-3 filter and inflating the quality score. Stored
lowercase in EXCLUDED_EXACT_LOWER and matched via toLowerCase().
P2 - audit-coverage.ts: registry aliases are structured YAML
objects (`ecosystem: npm` + `name: <name>`), not the
`- npm:<name>` shorthand the earlier regex expected. Updated
to match the two-line ecosystem/name pair.
P2 - agents-intent.ts: replace the lossy two-pass `unescapeDq`
with a single-pass character-by-character decoder. The old
implementation `replace(/\\\\/g, '\\').replace(/\\"/g, '"')`
would consume backslashes the first pass produced, corrupting
round-trips for tasks containing a literal backslash adjacent
to a quote (e.g. `\"`). Regression test added.
P2 - repo-conventions.ts + quality.ts: share MAX_WALK_DEPTH=20
between scoreDirectory and collectDocFiles via conventions.ts.
Previously only collectDocFiles had a depth bound, so
scoreDirectory could accept a deep tree whose files
collectDocFiles later refused to read.
P3 - adapters.test.ts: replace conditional `if kind === 'docs'`
assertion blocks with explicit narrowing throws so a wrong-
kind regression fails the test instead of silently passing.
Also added a lowercase-noise SC-3 case for the case-insensitive
filter.
Build, lint, and 270 tests (267 + 3 new regression cases) green.1 parent 22297a5 commit d9f8529
22 files changed
Lines changed: 2188 additions & 78 deletions
File tree
- .please/docs/tracks/completed/convention-based-discovery-20260409
- packages
- cli
- scripts
- src
- discovery
- test
- discovery
- schema/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
9 | 9 | | |
10 | 10 | | |
Lines changed: 95 additions & 36 deletions
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
67 | 70 | | |
68 | 71 | | |
69 | 72 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| 38 | + | |
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
0 commit comments