Skip to content

feat(ai): pilot structured outputs for bullet reframing behind a flag - #104

Open
NickCrew wants to merge 1 commit into
mainfrom
103-structured-outputs-bulletreframe-pilot
Open

feat(ai): pilot structured outputs for bullet reframing behind a flag#104
NickCrew wants to merge 1 commit into
mainfrom
103-structured-outputs-bulletreframe-pilot

Conversation

@NickCrew

Copy link
Copy Markdown
Collaborator

Summary

Pilots Anthropic native structured outputs on reframeBulletForVector: when
VITE_FACET_STRUCTURED_OUTPUTS is on, the call sends
output_config.format = { type: 'json_schema', schema } and parses the
guaranteed-valid JSON directly, skipping the <result>/fence/brace extraction
heuristics. Default off; a per-call override makes the path deterministically
testable.

Why

First step of #103 — replacing the hand-rolled JSON-extraction scaffolding
(<result> sentinels, unwrapFencedJson, brace-matching, JsonExtractionError)
with grammar-constrained JSON, which deletes the "model returned unparseable
JSON" failure class. bulletReframing is the deliberate first target: its output
contract is a trivial { reframed, reasoning }, so it proves the mechanism
end-to-end (proxy output_config passthrough — already wired; the direct-parse
path; the flag) with near-zero schema risk before the larger generators migrate.

Alternatives considered

Verification

$ pnpm run typecheck
> tsc --noEmit -p tsconfig.app.json
(no errors)

$ pnpm exec vitest run src/test/bulletReframing.test.ts
 Test Files  1 passed (1)
      Tests  13 passed (13)

$ pnpm run test
 Test Files  210 passed (210)
      Tests  3445 passed (3445)

$ pnpm run lint
> eslint .
(no errors)

Two new tests cover the structured path (asserts output_config.format is sent
with the schema and that a pure-JSON text block parses directly) and the default
path (asserts no output_config is sent).

Risk & blast radius

Low and reversible. The new behavior is gated by a default-off flag
(VITE_FACET_STRUCTURED_OUTPUTS) plus a per-call override, so production behavior
is unchanged until the flag is flipped; rollback is a config change, not a revert.
The existing semantic shape-check is retained as a content backstop (structured
outputs guarantee shape, not content). No proxy change is required — output_config
already flows through normalizeOutputConfig untouched.

Protected paths: touches .env.example (one default-off flag line) and the
env-flag wiring in src/utils/facetEnv.ts. No changes to proxy, supabase,
identity, persistence, engine, store, or migrations. No data migration.

Open questions / follow-ups (all tracked on #103)

  • thinking + format is untested — neither pilot call passes a thinking
    budget, so the structured-outputs × adaptive-thinking interaction isn't
    exercised here. Test it on the first thinking-enabled call that migrates.
  • identity.extract needs the metrics / constraints schema-shape decision
    before it can migrate.
  • Rollout should standardize on Zod for schema derivation + runtime validation.

Part of #103


  • Commits are atomic and bisectable (this repo rebase-merges — they land as-is)
  • CI green locally: typecheck, lint, test (CodeQL/build run in CI)
  • If a protected path is touched, it's called out under Risk above (.env.example)

@supabase

supabase Bot commented Jun 29, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project zxcptjtlcvbtvzxybqio because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@vercel

vercel Bot commented Jun 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
facet-app Ready Ready Preview, Comment Jun 29, 2026 2:03am

Route reframeBulletForVector through Anthropic native structured outputs
(output_config.format) when VITE_FACET_STRUCTURED_OUTPUTS is enabled, parsing
the guaranteed-valid JSON directly and skipping the <result>/fence/brace
extraction heuristics. Default off, with a per-call override for tests so the
path is deterministically exercisable. The output schema is hand-written to stay
within the constrained-decoding subset and dependency-free.

First step of #103; proves the mechanism on the smallest output contract before
migrating the larger generators.
@NickCrew
NickCrew force-pushed the 103-structured-outputs-bulletreframe-pilot branch from b399623 to a9e3c74 Compare June 29, 2026 02:02
@github-actions

Copy link
Copy Markdown

Bundle Size Report

Category Size
Total assets 37.72MB
JavaScript (91 chunks) 6.58MB
CSS 370.55KB
WASM 27.99MB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

1 participant