Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .changeset/search-guidance-tier.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
'@astryxdesign/cli': patch
---

[fix] `search` indexes a component's usage guidance, one tier below its description.

A component's best practices are where the reader's vocabulary lives. `Banner` describes itself as "a persistent message"; only its guidance says "caution", "problems", "form errors". None of those words found it, because guidance was never read — 97 core components ship guidance, and all of it was invisible to search.

Measured on the real registry, before and after: `caution`, `problems`, `sources` and `attention` each now return the component whose guidance defines them, and each returned nothing relevant before.

Guidance scores 45, below description's 50, so a component that IS the answer still outranks one whose advice merely mentions the term — the ordering that put `Toast` behind `Card`, `Dialog` and `Item` on "notification".

It sits deliberately BELOW `MIN_TOKEN_SCORE`, so it never counts as a matched concept in a multi-word query. That is not a detail: letting it count was measured moving `nested menu` from SideNav to List, and `explain why a field is required` from Field to TextInput — a component whose guidance happens to mention the other word displacing the one that is the answer. Breadth is not relevance, the same reason `weakKeywords` are capped. With the floor left at 50, a 28-query sweep shows zero top-result changes and zero regressions, while the single-word gains above are kept.

@josephfarina
5 changes: 4 additions & 1 deletion packages/cli/api/search/search.doc.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@ export const doc = {
description:
'The single "I\'m looking for X" entry point across every content domain. ' +
'Ranking is keyword + fuzzy (not embeddings); name and keyword signals outrank ' +
'incidental prose mentions, so an exact match always sorts first.',
'incidental prose mentions, so an exact match always sorts first. A component\'s ' +
'usage guidance (its best practices) is indexed one tier below its description, ' +
'so the words a reader types still find it — but a component that IS the answer ' +
'always outranks one whose advice merely mentions the term.',
importPath: '@astryxdesign/cli/api',
signature:
'search(query: string, options?: SearchOptions): Promise<SearchResponse>',
Expand Down
74 changes: 72 additions & 2 deletions packages/cli/api/search/search.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
* 60 name substring (>=4 chars, >=50% coverage)
* 60 exact weak-keyword match
* 50 description / prose mentions the term
* 45 usage guidance mentions the term
* 40 name Levenshtein distance 2
* 40 weak-keyword substring
* 30 keyword Levenshtein distance 2
Expand All @@ -31,6 +32,14 @@
* Name + keyword signals always outweigh description/prose, so an exact match
* sorts above an incidental mention.
*
* Description and guidance are separate tiers on purpose. A component's own
* one-line description saying "notification" is a claim about what it IS; the
* same word inside another component's best-practice advice is a passing
* mention. Scored equally, `Toast` — "a brief, non-blocking notification" —
* ties with `Card`, `Dialog` and `Item`, which merely mention notifications in
* their guidance, and ties break alphabetically, so Toast falls off the end of
* its own best query.
*
* `keywords` carry AUTHORED intent — a block's `componentsUsed`, a page's
* `category` words. `weakKeywords` are DERIVED: the components a page template
* happens to render, scraped out of its JSX. Derived signal is deliberately
Expand Down Expand Up @@ -67,6 +76,7 @@ import {ERROR_CODES} from '../../foundation/response/error-codes.mjs';
* @property {string[]} [weakKeywords]
* @property {string} [description]
* @property {string[]} [prose]
* @property {string[]} [guidance]
* @property {string} [_import]
* @property {string} [_title]
* @property {string} [_displayName]
Expand Down Expand Up @@ -189,6 +199,14 @@ export function tokenizeQuery(term) {
* Minimum per-token score (in the multi-word pass) to count as a real match.
* 50 = a genuine name/keyword/description hit; below that is loose Levenshtein
* fuzz that would otherwise turn gibberish queries into noise.
*
* Guidance (45) is deliberately BELOW this floor, so it never counts as one of
* the matched concepts in a multi-word query. Measured: letting it count moved
* `nested menu` from SideNav to List and `explain why a field is required` from
* Field to TextInput — in both cases a component whose guidance happens to
* mention the other word displaced the one that IS the answer. Breadth is not
* relevance, the same reason `weakKeywords` are capped. Guidance still decides
* single-word queries and still breaks ties, which is where it earns its place.
*/
const MIN_TOKEN_SCORE = 50;

Expand Down Expand Up @@ -287,11 +305,12 @@ export function scoreQuery(term, tokens, candidate) {
* @param {string[]} [candidate.weakKeywords] - Derived signal (components a page renders).
* @param {string} [candidate.description]
* @param {string[]} [candidate.prose] - Extra free-text blobs (doc section text, best practices).
* @param {string[]} [candidate.guidance] - Usage guidance (features, best practices) — scored a tier below description.
* @returns {{score: number, reason: string} | null}
*/
export function scoreCandidate(
term,
{name, keywords = [], weakKeywords = [], description = '', prose = []},
{name, keywords = [], weakKeywords = [], description = '', prose = [], guidance = []},
) {
let best = 0;
let reason = '';
Expand Down Expand Up @@ -359,7 +378,7 @@ export function scoreCandidate(
}
}

// ── Prose / description signals (stem-tolerant whole word) ──────
// ── Prose / description / guidance signals (stem-tolerant whole word) ──
// Match the term's stem as a whole word, tolerating plural/gerund suffixes
// so "chart" matches "charts" and "filter" matches "filtering".
if (term.length >= 3) {
Expand All @@ -369,12 +388,25 @@ export function scoreCandidate(
if (description && re.test(description.toLowerCase())) {
consider(50, `description mentions "${term}"`);
} else {
let matchedProse = false;
for (const blob of prose) {
if (blob && re.test(String(blob).toLowerCase())) {
consider(50, `docs mention "${term}"`);
matchedProse = true;
break;
}
}
// A tier below prose: guidance is what a component says about USING it,
// so the term appearing there is weaker evidence than the component's own
// summary. Only consulted when nothing stronger matched.
if (!matchedProse) {
for (const blob of guidance) {
if (blob && re.test(String(blob).toLowerCase())) {
consider(45, `guidance mentions "${term}"`);
break;
}
}
}
}
}

Expand All @@ -396,6 +428,39 @@ async function loadModuleDoc(docPath, exportName = 'docs') {
}
}

/**
* Build component candidates from core's own tree: name + keywords +
* usage/description from the component's .doc.mjs.
* @param {string} coreDir
* @returns {Promise<Candidate[]>}
*/
/**
* Usage guidance from a component doc: its feature list and its best-practice
* advice, flattened to plain strings.
*
* This is where a reader's vocabulary usually lives. `Banner` calls itself "a
* persistent message" and only its guidance names "form errors, system
* updates, maintenance notices" — so a search for the words people actually
* type finds nothing without it.
*
* @param {any} doc
* @returns {string[]}
*/
function guidanceFrom(doc) {
if (!doc) return [];
const features = Array.isArray(doc.features) ? doc.features : [];
const practices = Array.isArray(doc.usage?.bestPractices) ? doc.usage.bestPractices : [];
return [...features, ...practices]
.map(entry =>
typeof entry === 'string'
? entry
: [entry?.title, entry?.text, entry?.description, entry?.do, entry?.dont]
.filter(Boolean)
.join(' '),
)
.filter(Boolean);
}

/**
* Build component candidates from core's own tree: name + keywords +
* usage/description from the component's .doc.mjs.
Expand All @@ -412,18 +477,22 @@ async function gatherCoreComponents(coreDir) {
/** @type {string[]} */
let keywords = [];
let description = '';
/** @type {string[]} */
let guidance = [];
if (readme && readme.endsWith('.doc.mjs')) {
const doc = await loadModuleDoc(readme);
if (doc) {
keywords = Array.isArray(doc.keywords) ? doc.keywords : [];
description = doc.usage?.description || doc.description || '';
guidance = guidanceFrom(doc);
}
}
candidates.push({
domain: 'component',
name: comp,
keywords,
description,
guidance,
_import: resolveImportPath(coreDir, comp),
});
}
Expand Down Expand Up @@ -452,6 +521,7 @@ async function gatherIntegrationComponents(cwd) {
name: rec.name,
keywords: doc && Array.isArray(doc.keywords) ? doc.keywords : [],
description: doc ? doc.usage?.description || doc.description || '' : '',
guidance: guidanceFrom(doc),
_import: rec.package,
});
}
Expand Down
71 changes: 71 additions & 0 deletions packages/cli/api/search/search.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -241,3 +241,74 @@ describe('search scoring — derived keywords rank below authored ones', () => {
expect(derived?.reason).toMatch(/renders Dialog/);
});
});

describe('search — usage guidance is indexed, a tier below description', () => {
// The vocabulary a reader types usually lives in a component's guidance, not
// in its one-line description. Banner calls itself "a persistent message";
// only its best practices name "caution", "problems", "form errors". Before
// this, none of those words found it.
const banner = {
name: 'Banner',
keywords: ['alert', 'notification'],
description: 'A persistent message shown above content.',
guidance: [
'Pick a status that matches the message: info for updates, warning for caution.',
'Use error for problems the reader must resolve before continuing.',
],
};

it('finds a term that appears ONLY in guidance', () => {
// Red before this change: guidance was never read, so this scored null.
const hit = scoreCandidate('caution', banner);
expect(hit).not.toBeNull();
expect(hit?.reason).toMatch(/guidance mentions "caution"/);
});

it('scores guidance BELOW description, so a component about X outranks one that merely mentions X', () => {
const own = scoreCandidate('persistent', banner);
const mention = scoreCandidate('caution', banner);
expect(own?.score).toBe(50);
expect(mention?.score).toBe(45);
expect(mention.score).toBeLessThan(own.score);
});

it('never lets guidance outrank a name or keyword hit', () => {
expect(scoreCandidate('banner', banner)?.score).toBe(100);
expect(scoreCandidate('notification', banner)?.score).toBe(90);
});

it('prefers the stronger signal when a term is in both description and guidance', () => {
const both = scoreCandidate('message', banner);
expect(both?.score).toBe(50);
expect(both?.reason).toMatch(/description mentions/);
});

it('stays below MIN_TOKEN_SCORE, so guidance never counts as a matched CONCEPT', () => {
// Measured regression this prevents: counting guidance as a matched term
// moved `nested menu` from SideNav to List, and `explain why a field is
// required` from Field to TextInput — in both cases a component whose
// guidance happens to mention the other word displaced the real answer.
// Guidance decides single-word queries; it must not win multi-word ones on
// breadth. Same reason weakKeywords are capped.
expect(scoreCandidate('caution', banner)?.score).toBeLessThan(50);
// Both words are in this candidate's guidance and nowhere else, so if
// guidance counted as a concept this would come back as a 2/2 match.
const multi = scoreQuery('caution problems', tokenizeQuery('caution problems'), banner);
expect(multi).toBeNull();
});

it('lets a real description hit still win the multi-word pass', () => {
// The floor must exclude guidance without muting the tiers above it.
const hit = scoreQuery('persistent message', tokenizeQuery('persistent message'), banner);
expect(hit?.reason).toMatch(/matches 2\/2 terms/);
});

it('tolerates the object-shaped bestPractices entries core actually ships', () => {
// Core writes `{guidance: true, description: '...'}`, not plain strings.
const hit = scoreCandidate('resolve', {
name: 'X',
guidance: ['Use error for problems the reader must resolve.'],
});
expect(hit?.score).toBe(45);
});
});
Loading