Skip to content

perf: denormalize latestVersionSummary into skillSearchDigest#776

Open
sethconvex wants to merge 2 commits intoopenclaw:mainfrom
sethconvex:perf/denormalize-latestVersionSummary-into-digest
Open

perf: denormalize latestVersionSummary into skillSearchDigest#776
sethconvex wants to merge 2 commits intoopenclaw:mainfrom
sethconvex:perf/denormalize-latestVersionSummary-into-digest

Conversation

@sethconvex
Copy link
Contributor

Summary

  • Adds latestVersionSummary to skillSearchDigest schema and syncs it via the existing trigger (SHARED_KEYS)
  • buildPublicSkillEntries now reads summary directly from HydratableSkill instead of type-narrowing with 'latestVersionSummary' in skill
  • Eliminates ~9MB of skillVersions reads per listPublicPageV2 call; old rows without the field fall back to fetching the full version doc

Test plan

  • npx vitest run — all 714 tests pass
  • npx convex dev --once — typecheck + push to dev
  • Deploy to prod, run backfillLatestVersionSummary to populate existing digest rows
  • Monitor insights — skillVersions should disappear from listPublicPageV2

🤖 Generated with Claude Code

Eliminates ~9MB of skillVersions reads per listPublicPageV2 call by
copying latestVersionSummary from skills into the digest via the
existing trigger. Old rows without the field fall back to fetching
the full version doc.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@vercel
Copy link
Contributor

vercel bot commented Mar 12, 2026

Someone is attempting to deploy a commit to the Amantus Machina Team on Vercel.

A member of the Team first needs to authorize it.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 12, 2026

Greptile Summary

This PR denormalizes latestVersionSummary into the skillSearchDigest table so that listPublicPageV2 can serve version metadata directly from the lightweight digest rows, avoiding a separate ~6KB skillVersions doc read per skill. The schema is extended consistently across both tables, the SHARED_KEYS array and HydratableSkill type are updated to include the new field, and the buildPublicSkillEntries function is simplified to read the field directly with a clean fallback to ctx.db.get for old rows that haven't been backfilled yet.

  • latestVersionSummary added to skillSearchDigest schema with a validator that mirrors the skills table exactly
  • SHARED_KEYS in skillSearchDigest.ts extended so the field is synced by the existing trigger
  • HydratableSkill type updated so the TypeScript compiler enforces parity between the two shapes
  • buildPublicSkillEntries simplified from an 'in' skill type-narrowing pattern to a direct field access; rows without the field still fall back to fetching the full version doc
  • Test helpers updated; the fallback path for old (un-backfilled) digest rows is no longer covered by the listPublicPageV2 tests — worth adding a case before the backfill runs in production

Confidence Score: 4/5

  • Safe to merge; the fallback logic for un-backfilled rows is correct and the schema changes are consistent.
  • The implementation is correct: schema validators match between skills and skillSearchDigest, the SHARED_KEYS constraint ensures compile-time parity, and the undefined fallback to ctx.db.get is preserved for old rows. Score is 4 rather than 5 solely because the listPublicPageV2 test suite no longer exercises the fallback code path, which will be the live path in production until backfillLatestVersionSummary completes.
  • convex/skills.listPublicPageV2.test.ts — missing test coverage for the fallback path when latestVersionSummary is absent
Prompt To Fix All With AI
This is a comment left during a code review.
Path: convex/skills.listPublicPageV2.test.ts
Line: 336-341

Comment:
**Fallback path no longer covered by tests**

`makeSkill` now unconditionally includes `latestVersionSummary`, so every test in this file exercises the "use summary" fast path. The regression path — where `latestVersionSummary` is `undefined` and `buildPublicSkillEntries` must fall back to `ctx.db.get(skill.latestVersionId)` — is no longer tested. This matters because the PR description explicitly calls out that backfilling existing digest rows is a post-deploy step, meaning old rows without the field will hit this code path in production.

Consider adding a test with `latestVersionSummary: undefined` (or omitting it) that verifies `db.get` is called with the version ID and the result is surfaced correctly.

How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: 576588b

Comment on lines +336 to +341
latestVersionSummary: {
version: '1.0.0',
createdAt: 1,
changelog: '',
changelogSource: 'user' as const,
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fallback path no longer covered by tests

makeSkill now unconditionally includes latestVersionSummary, so every test in this file exercises the "use summary" fast path. The regression path — where latestVersionSummary is undefined and buildPublicSkillEntries must fall back to ctx.db.get(skill.latestVersionId) — is no longer tested. This matters because the PR description explicitly calls out that backfilling existing digest rows is a post-deploy step, meaning old rows without the field will hit this code path in production.

Consider adding a test with latestVersionSummary: undefined (or omitting it) that verifies db.get is called with the version ID and the result is surfaced correctly.

Prompt To Fix With AI
This is a comment left during a code review.
Path: convex/skills.listPublicPageV2.test.ts
Line: 336-341

Comment:
**Fallback path no longer covered by tests**

`makeSkill` now unconditionally includes `latestVersionSummary`, so every test in this file exercises the "use summary" fast path. The regression path — where `latestVersionSummary` is `undefined` and `buildPublicSkillEntries` must fall back to `ctx.db.get(skill.latestVersionId)` — is no longer tested. This matters because the PR description explicitly calls out that backfilling existing digest rows is a post-deploy step, meaning old rows without the field will hit this code path in production.

Consider adding a test with `latestVersionSummary: undefined` (or omitting it) that verifies `db.get` is called with the version ID and the result is surfaced correctly.

How can I resolve this? If you propose a fix, please make it concise.

Verifies that old digest rows without latestVersionSummary correctly
fall back to ctx.db.get(latestVersionId) for version data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@sethconvex
Copy link
Contributor Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b32a499774

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

'latestVersionSummary' in skill
? (skill as Doc<'skills'>).latestVersionSummary
: undefined
const summary = skill.latestVersionSummary

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add a migration to populate digest latestVersionSummary

Switching listPublicPageV2 to trust skill.latestVersionSummary here means the optimization only works if old skillSearchDigest rows are populated, but this commit never backfills that table: the sync path only runs when a skills row mutates (convex/functions.ts trigger), and backfillLatestVersionSummaryInternal skips already-synced skill rows (convex/maintenance.ts), so most existing digests will keep the field absent and continue hitting ctx.db.get(skill.latestVersionId) on every page. In production this prevents the intended read reduction for historical data; add a one-time digest backfill (or force-touch skills) as part of this change.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant