Repository navigation
wix-app: one SDK lookup call per feature, with the call shape - #1684
kobiburnley wants to merge 19 commits into
Conversation
scripts/sdk-lookup.cjs resolves every vertical entity in one call from the installed SDK packages: the namespace to use and why (search method, fqdn version, preview/deprecated, curated rule), the import, the methods for the intent, a `shape` line (search method with free text → filter object with $or → builder only for its listed fields: generated builders chain AND and have no or()), required fields such as revision, and the entity's declaration file:line. It reads sdk-index.json when a package ships one and derives the same record from the .d.ts files otherwise. Step 3, its checklist, SDK_MODULE_MAP.md, DATA_SOURCES.md and STORES_VERSIONING.md now give one order: lookup → curated note → docs for what it does not print → fields in the declaration. Also: members.getCurrentMember is the SDK name of Get My Member (SKILL.md, BACKEND_API.md and the filtered-read judge said getMyMember). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lists The arm C eval (contacts-search-2026-10-08T11-57-34-056Z, run 1) printed searchContacts(search: CursorSearch, options: …) — the REST request shape from index.typings.d.ts. The public call takes ContactSearch with options optional, so the run wrote contactsV5.CursorSearch, hit TS2769 and spent 107 s recovering. Parameters now come from `interface <Name>Signature` in index.d.ts when it exists. Filter lists now come from the package too: <Entity>SearchSpec and <Entity>QuerySpec declare the free-text fields and, per operator group, the fields and sort (16 of 16 search methods, 80 of 86 query methods in a six-vertical install). A shipped sdk-index.json list is merged field by field, so the docs' list (which omits name.full for contacts) cannot hide a field the declarations have. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… to --intent write, and drops filter lines without operators Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dk-index.json The packages already ship the public signatures, the search and query specs, permissions and required fields. The index added only the scope ID and the docs link, and its source line sent one eval run to search the disk for the file. Method scopes are read from @permissionScopeId when a package declares it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… argument and response The lookup printed searchContacts(search: ContactSearch), and the first ContactSearch a grep finds is an empty @hidden placeholder. In contacts-search-2026-10-08T16-26-55-730Z/1 the run spent 15 calls (115 s) looking for the argument's shape. The lookup now prints arg (cursorPaging, filter, sort, search) and returns (pagingMetadata.cursors.next), read from the real type and two levels deep: 80 of 102 such methods have an object argument, 101 of 102 a response shape. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lookup now gives the method list and the namespace-to-directory map, so the method grep and the ls recipe go; the E= line moves to the field check that uses it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
12 cases over @wix/crm 1.0.1648 (contacts V4 and V5) and refunds: the namespace choice, the shape line, public params, arg and returns, the filter list, builder operators, revision, scopes, the install command for a missing wrapper, and the exit codes. Runs in the test job whenever wix-app or the test changes; skill scripts get no eval tag of their own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kobiburnley
changed the title
|
… filter list on its own entity - A query builder that also takes one query object (the @hidden typedQueryX, 73 of 192 packages) prints it as `or`, with its arg and response keys. When the builder has no filter or sort (extendedBookings.queryExtendedBookings) the shape line sends the call there, not to "startsWith on ONE field". - A spec declared with no field (`wql: []`) prints "filter fields: not in the package (the method docs page lists them)". - The <Entity>QuerySpec / SearchSpec goes only to a query or search that returns that entity: services.queryPolicies and 7 others no longer show the services filter list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…@wix/sdk-runtime Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # skills/wix-app/references/STORES_VERSIONING.md
* wix-app: the SDK lookup says how to apply a search term
Without a free-text search method, the search shape line now names the
fields a term can go in: those that take $startsWith and are declared
string on the entity (ids and numbers in a string left out), as one
$or filter. When the package lists the operators and no field fits, it
says the API cannot match a term, instead of a generic $or hint.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* wix-app: lookup search term — options free text, '*' operators, $contains
Checked against the docs for members, eCommerce orders and coupons:
- A query method can take free text in its options with no search method
(queryMembers' options.search). The shape line now prints that call and
says that search.fields defaults to one field (profile.nickname).
- A field under operators '*' is not a sure $startsWith: orders' number is
a string in the package but takes no $startsWith. Such fields are now
textFieldsAny, printed with a pointer to the docs page and $eq.
- "No filter list has $contains" read as "no API has it"; coupons' code
and name take it. The line now sends the agent to the docs page.
The members eval scenario now judges options.search with named fields
and the FULL fieldset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* wix-app: lookup sends an unlisted filter straight to the docs page
coupons-search-2026-10-13T10-27-21-561Z run 0 spent 14 requests (78 s)
after the lookup said the package does not list queryCoupons' filter
fields: it read index.typings.d.ts four times and grepped index.js
before it searched the docs, which had the answer ($contains on code
and name).
When the package has no filter list, the shape line now says that
nothing more is in the package, names the docs search to run
(SearchWixSDKDocumentation "queryCoupons filter", which returns the
Filter and sort article), and says to prefer $contains where the docs
list it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* wix-app: lookup states a typed but unsearchable search.expression
form-submissions-search-2026-10-13T10-44-09-306Z run 0 spent 22
requests (113 s) after the lookup said searchSubmissionsByNamespace has
"no free-text search" while its arg line showed search: { mode,
expression, fuzzy }: it re-read the submissions typings seven times and
then searched the docs. Both were right in part: the argument types the
generic search object, and FormSubmissionSearchSpec declares no
searchable field.
The line now says exactly that, sends the agent to the docs search
next (nothing more is in the package), and, when the docs have no text
filter either, says what a search must then do: read every page, re-read
on each search, no silent cap. The run's code capped the scan at 2000
and cached it for the page's lifetime.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* wix-app: lookup trusts operators '*' and warns of an ignored expression
Read calls on a real site (2026-10-13) showed that the package was right
and the docs were not: orders' number, under operators '*', takes
$startsWith ({ $startsWith: "1003" } returns orders 10030-10034),
although the method's docs capability list gives it none. e09ddeb had
sent such fields to the docs page and to $eq; they are now ordinary text
fields again.
They also showed that searchOrders, a search method without free text,
returns every order for a search.expression, with no error. The shape
line now says so and puts the term in filter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
| "source": "SKILL.md (Identity and Elevation Requirement)" | ||
| }, | ||
| { | ||
| "entities": ["member", "members"], |
There was a problem hiding this comment.
The package already says this: getCurrentMember's JSDoc notes that it "requires visitor or member authentication". I suggest printing the methods' JSDoc notes in the lookup and dropping this rule. @applicableIdentity can't replace the notes: getCurrentMember is tagged APP only, VISITOR never appears, and 30% of methods have no tag.
There was a problem hiding this comment.
Done in 6cee451. The lookup now prints the > **Note:** text of each method as a note line (for example, getCurrentMember: "This method requires visitor or member authentication."). The members rule keeps only the PRIVATE-members fact, which the package does not say. A test on the members fixture checks that the identity tag is APP and that the note is printed.
| "source": "references/dashboard-page/DATA_SOURCES.md, references/backend-event/COMMON-EVENTS.md" | ||
| }, | ||
| { | ||
| "entities": ["location", "locations"], |
There was a problem hiding this comment.
These are worth keeping: neither the package nor the docs say it, and the docs suggest the opposite. But SKILL.md:319 repeats them. I suggest keeping the per-method facts here, and having SKILL.md keep only the general rule plus a link to this file.
There was a problem hiding this comment.
Done in 6cee451. The locations and cart facts stay in sdk-curated.json only. SKILL.md keeps the general rule (the scope name never settles it) and links to sdk-curated.json, which the lookup prints as note.
| with: | ||
| node-version: '22' | ||
|
|
||
| - name: wix-app — sdk-lookup tests |
There was a problem hiding this comment.
derive-tags.ts:21 ignores scripts/**, so changes to sdk-lookup.cjs or sdk-curated.json run no evals (as with #1745), yet the skill tells the agent to trust the script over the typings. I suggest mapping these files to the sdk-module-map / dashboard-page tags.
There was a problem hiding this comment.
Done in 6cee451. The skill gate has a new tag-globs input (glob: tag[, tag] lines), checked before ignore-globs. evalforge-wix-app-gate.yml maps scripts/sdk-lookup.cjs and scripts/sdk-curated.json to sdk-module-map and dashboard-page. deriveTags, the config and the README are updated, with tests. The gate runs the action from main, so the mapping takes effect after this PR merges.
|
|
||
| - the namespace to use and why (a `search*` method for a search; the highest API version; a curated rule), and the other namespaces; | ||
| - the import line; | ||
| - `shape`: the one call to write for the intent, and why — a `search*` method with the term in `search.search.expression`; else a query whose options take free text (`queryMembers`' `options.search` — name its `fields`, which default to one); else one filter object with `$or` of `$startsWith` on the text fields it names (a prefix match; a field with operators `'*'` takes every operator; a `search*` method without free text ignores `search.expression` with no error), or `no text match in the package` when no field takes it; a query builder only for filters on the fields its operators list, because builders chain clauses with AND and have no `or()` — most builder methods also take one query object (`queryX({ filter, sort, … })`, printed as `or`), which is the call when the builder has no filter at all (`extendedBookings`); |
There was a problem hiding this comment.
This bullet describes shapeFor in prose, a second copy that #1745 already had to extend. The output gives the reason on every shape line. I suggest keeping only when to run the lookup, what to take from it, and what it doesn't cover.
There was a problem hiding this comment.
Done in 6cee451. The shape bullet is now one line: the one call for the intent, with its reason and, for a search, where the term goes. The details stay only in the script output.
| - the entity type as `file:line` — the declaration to verify fields against ([DATA_SOURCES.md](references/dashboard-page/DATA_SOURCES.md#confirming-a-field--the-part-that-ships-bugs)); | ||
| - the curated rules for the entity (`note`), such as the Stores V1/V3 check. | ||
|
|
||
| Take the namespace, the method and the call shape from its output; do not re-derive them from the typings. **Never run one query per field and merge the results in the client** — when the `shape` line says one call cannot match several fields, use the one field it names and tell the user. When it says `no text match in the package` and the method docs page lists no text filter either, tell the user that the API cannot match a search term. A search then reads every page and matches the values itself, re-read on each search and with no silent cap — never match only one fetched page, or a capped or cached set, and present it as a search. When the lookup prints `site`, the choice depends on the site (Stores: call `getCatalogVersion()` and support both). `not installed` prints the install command: install, then run the lookup again. A name it does not know exits 1 with near matches. |
There was a problem hiding this comment.
The "never merge per-field queries" and "no text match" rules are also printed by the script. I suggest keeping only the script's copy.
There was a problem hiding this comment.
Done in 6cee451. I removed both rules from SKILL.md, because the script prints them under the shape line. The paragraph now says to follow the lines under shape.
| const depDir = resolveDir(dep, dir); | ||
| if (!depDir) continue; | ||
| const entry = autos.get(dep) ?? { dir: depDir, exportedAs: [] }; | ||
| const m = src.match(new RegExp(`import \\* as (\\w+) from '${dep}';\\s*export \\{ \\1 as (\\w+) \\}`)); |
There was a problem hiding this comment.
Today the packages this doesn't match are service plugins the wrapper doesn't export, so the label is right. But an equivalent change to the export file, e.g. export * as ns from '…', would also be labeled "not exported" (line 676), and the agent would trust it. I suggest failing loudly when the export file mentions the package but this regex doesn't match.
There was a problem hiding this comment.
Done in 6cee451. When the export file has an export … from or an import * as for the package that this regex does not read, the lookup stops with exit code 4 and names the wrapper and the file. A bare import '…' (for example, @wix/app-management imports custom-charges this way) exports nothing, so it is still labeled "not exported". A test with an export * as ns from '…' fixture checks the exit code.
| return null; | ||
| } | ||
|
|
||
| function deriveRecord(name, dir) { |
There was a problem hiding this comment.
Once wix-private/public-sdk#2153 is released, is the plan to delete this copy and call @wix/sdk-runtime/sdk-lookup?
There was a problem hiding this comment.
Yes. When @wix/sdk-runtime/sdk-lookup is released, the script calls it, and this copy is deleted in a follow-up PR.
| - [stores/CATEGORIES.md](stores/CATEGORIES.md) — collections (V1) ↔ categories (V3) | ||
|
|
||
| For methods not listed here, use `SearchWixSDKDocumentation` then `ReadFullDocsMethodSchema` (use `ReadFullDocsArticle` only for prose). Always return the required permission scopes to the user. | ||
| For methods not listed here, run the [SDK lookup](../SKILL.md#sdk-lookup-one-call-for-every-vertical-entity) for `products` (it prints both namespaces and their methods), then read the method's docs page for its scope and filter list. Always return the required permission scopes to the user. |
There was a problem hiding this comment.
The lookup already prints the filter list when the package declares one, and this line no longer names the tool. I suggest: "…then ReadFullDocsMethodSchema for its permission scope (and the filter list if the lookup printed none)."
- The lookup prints each method's JSDoc notes ("requires visitor or member
authentication"), so the curated members rule drops getCurrentMember.
- It exits 4 when a wrapper exports a package in a form it does not read,
instead of labeling the package "not exported".
- SKILL.md keeps the general rules and links to sdk-curated.json; the shape
details and the search rules live only in the script output.
- The skill gate gets a tag-globs input, and the wix-app gate maps
sdk-lookup.cjs and sdk-curated.json to sdk-module-map and dashboard-page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| preferRule && c.ns.namespace === preferRule.choice.prefer ? 1 : 0, | ||
| c.ns.namespace ? 1 : 0, | ||
| intent === 'search' && c.record.methods.some((m) => m.kind === 'search') ? 1 : 0, | ||
| versionOf(c.record.fqdn), |
There was a problem hiding this comment.
When different packages export the same namespace, the ranking picks one by API version, which says nothing about the feature. With @wix/ecom and @wix/pricing-plans installed, orders resolves to pricing-plans (v2 over v1). With ecom and @wix/events, it resolves to whichever package is read first. 13 namespace names are shared between packages. I suggest using the version only within one package (contacts vs contactsV5). Across packages, default to the module map's package and print the entity's one-line description from the package next to each alternative ("completed purchase from a buyer" vs "purchase of a Pricing Plan"), so the agent can switch when the feature matches another package.
There was a problem hiding this comment.
Done in 1688e7c.
- The API version and a search method now rank only the namespaces of one package (
contactsvscontactsV5). - Across packages, the module map package wins, then the package name. The order never depends on which package is read first.
- When several packages export the name, each
useandalsoline prints the first sentence of the entity description from the package, then this line: "several packages export this name: if the feature matches another package's description, use that one".
With @wix/ecom, @wix/pricing-plans and @wix/events installed, orders now picks @wix/ecom for both read and search. Pricing Plans shows "An order object includes all of the details related to the purchase of a Pricing Plan.", and Events shows "no description in the package", because its Order has no JSDoc. In that install, forms (events and forms) is the only other shared name, and it picks @wix/forms. A test covers the ecom and pricing-plans case.
…t the API version Several packages export the same namespace (orders: ecom, pricing-plans, events). The API version and a search method now rank only the namespaces of one package. Across packages the module map's package wins, then the package name, and each one prints its entity description from the package. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RotemBotavia
left a comment
RotemBotavia
left a comment
There was a problem hiding this comment.
categories has no module map row, so it falls back to alphabetical order and picks @wix/blog, which has no description. The other seven alternatives do show descriptions, including the generic @wix/categories (“A category for organizing items into a hierarchical structure.”), so the agent can still switch.
forms picked @wix/events over the map’s @wix/forms, but only because tool-share has an old @wix/forms (1.0.93) whose methods were all marked preview. The latest version (1.0.146) is no longer preview, so a fresh install picks @wix/forms as the author said.
… after the namespace categories has no module map row, so the package name order picked @wix/blog, which has no entity description. Across packages, the package named after the namespace (@wix/categories) now comes next, then a package whose entity has a description, then the package name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@RotemBotavia Thanks for the review. About your two observations:
|
|
#skipreview |
|
#forcemerge |
…re the methods A contacts-search eval run piped the write-intent output through head -60. The contactsV5 method list filled the 60 lines, so the type line was cut, and the run used four round trips to find the UpdateContact declaration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
This PR moved to https://github.com/wix-private/ax-skills/pull/17, because wix/skills is now a read-only mirror (#1766). The branch is the same. It also has 6500d22, which prints the entity |
This branch has not been deployed
2 participants
Why
To find an SDK call,
wix-appgave the agent four rules that conflict ("MCP only for gaps", "the installed package first", "SDK docs first", hand-writtengreprecipes overauto_sdk_*typings). In eval runs the agent spent minutes grepping hashed.d.tsfiles, and some runs chose the wrong namespace (contactsinstead ofcontactsV5) or the REST request type instead of the public one.What changes
scripts/sdk-lookup.cjs: one call resolves every vertical entity from the installed SDK packages. Per entity it prints:search*method for a search, thefqdnversion, preview/deprecated, a curated rule), plus the other namespaces;shapeline: the one call to write. Asearch*method with free text comes first, then one filter object with$or, then a builder only for the fields its operators list. Generated builders chain AND and have noor()(0 of 71 in a six-vertical install);revision);argandreturns, the keys of the argument (cursorPaging,filter,sort,search) and of the response (pagingMetadata.cursors.next), two levels deep;file:line, for the field check.Every record comes from the installed packages'
.d.tsfiles: the JSDoc (@fqn,@permissionId,@permissionScopeId,@documentationMaturity,@deprecated), the public<Name>Signature, the<Entity>SearchSpec/QuerySpecspecs and the builder members. No SDK change is needed.scripts/sdk-curated.json: 6 rules no package can make (Stores V1/V3, inventory, refunds → orders, locations, members, cart), each copied from existing skill text.One discovery order in
SKILL.mdStep 3 and its checklist,SDK_MODULE_MAP.md,DATA_SOURCES.mdandSTORES_VERSIONING.md:This resolves the "declaration first" vs "MCP for gaps" conflict, and the article-tool vs schema-tool conflict at
STORES_VERSIONING.md:133.DATA_SOURCES.md: the method grep and thelsrecipe go (the lookup gives both), which keeps the file under the 10,000-char fetch limit.Fix:
members.getCurrentMemberis the SDK name of Get My Member.SKILL.md,BACKEND_API.mdand thefiltered-read-not-elevatedjudge saidgetMyMember.SKILL.mdSDK types: how to key a map by an SDK enum.Partial<Record<…WithLiterals, V>>cannot be indexed by a status (TS7053, checked withtscon@wix/bookings); use a fullRecord, orPartial<Record<`${ns.Enum}`, V>>. A measuredupcoming-bookingsrun lost atscpass to it.Tests
scripts/test-sdk-lookup.cjs(new, in the CItestjob when wix-app changes): 12 cases over trimmed copies of real@wix/crm1.0.1648 and refunds packages: the namespace choice, the shape line, public params,arg/returns, the filter list, builder operators,revision, scopes, the install command for a missing wrapper, and the exit codes.patterns-skill-eval-feature-vibe(contacts-search,contacts-table,refunds,product-inventory) found three defects in earlier commits, all fixed here: the REST parameter type (CursorSearch) instead of the public one, a missingname.fullin the free-text list, and an agent search for the argument's shape past the emptyContactSearchplaceholder (15 calls in one run).Evals
dashboard-page/sdk-lookup-one-call.yml(contacts + orders in one lookup call;contactsV5.searchContactsfor the search box).refunds-dashboard-sdk-not-cms.yml: the judge accepts the lookup as a source and does not count the skill's own scripts asnode_modulesexploration.session-methods-called-directly.yml: taggedstores-versioning, because it reads Stores products.Follow-ups
@wix/sdk-runtime/sdk-lookup, tested there and, in a later metro PR, against metro's generated output. Its records are identical to this script's on all 171 packages. When it is released, this script calls it and keeps only the ranking, the curated rules and the output.@permissionScopeId), so the agent gets them from the docs page. The lookup prints them if a package declares them.SKILL.mdstill names MCP tools (pre-existing).🤖 Generated with Claude Code