feat(ai): pilot structured outputs for bullet reframing behind a flag - #104
Open
NickCrew wants to merge 1 commit into
Open
feat(ai): pilot structured outputs for bullet reframing behind a flag#104NickCrew wants to merge 1 commit into
NickCrew wants to merge 1 commit into
Conversation
|
This pull request has been ignored for the connected project Preview Branches by Supabase. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Route reframeBulletForVector through Anthropic native structured outputs (output_config.format) when VITE_FACET_STRUCTURED_OUTPUTS is enabled, parsing the guaranteed-valid JSON directly and skipping the <result>/fence/brace extraction heuristics. Default off, with a per-call override for tests so the path is deterministically exercisable. The output schema is hand-written to stay within the constrained-decoding subset and dependency-free. First step of #103; proves the mechanism on the smallest output contract before migrating the larger generators.
NickCrew
force-pushed
the
103-structured-outputs-bulletreframe-pilot
branch
from
June 29, 2026 02:02
b399623 to
a9e3c74
Compare
Bundle Size Report
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pilots Anthropic native structured outputs on
reframeBulletForVector: whenVITE_FACET_STRUCTURED_OUTPUTSis on, the call sendsoutput_config.format = { type: 'json_schema', schema }and parses theguaranteed-valid JSON directly, skipping the
<result>/fence/brace extractionheuristics. Default off; a per-call override makes the path deterministically
testable.
Why
First step of #103 — replacing the hand-rolled JSON-extraction scaffolding
(
<result>sentinels,unwrapFencedJson, brace-matching,JsonExtractionError)with grammar-constrained JSON, which deletes the "model returned unparseable
JSON" failure class.
bulletReframingis the deliberate first target: its outputcontract is a trivial
{ reframed, reasoning }, so it proves the mechanismend-to-end (proxy
output_configpassthrough — already wired; the direct-parsepath; the flag) with near-zero schema risk before the larger generators migrate.
Alternatives considered
identity.extractinstead (the handoff's first suggestion):rejected for now — its output contract contains open-keyed maps (
metrics: {},preferences.constraints: {}) that constrained decoding cannot represent(
additionalPropertiesmust befalse). It needs a schema-shape decisionfirst; tracked in Structured outputs: replace hand-rolled JSON extraction with native grammar-constrained JSON #103.
zod-to-json-schema/z.toJSONSchema()):rejected for the pilot to keep it dependency-free and the schema provably
inside the constrained-decoding subset. Recorded in Structured outputs: replace hand-rolled JSON extraction with native grammar-constrained JSON #103 as the recommended
approach for the broader rollout (it adds a runtime validation backstop), which
is a deliberate dependency/bundle decision rather than a pilot side-effect.
not pursued — at current sizes only 2 of ~20 system prompts clear the model
cache minimums (Sonnet 4.6 = 2048, Opus/Haiku = 4096 tokens), and the large
stable context (identity) is re-serialized per call in the user message and
mutates across the bulk-deepen loop, so it would not cache. (Out of scope for
this PR; noted on Structured outputs: replace hand-rolled JSON extraction with native grammar-constrained JSON #103.)
Verification
Two new tests cover the structured path (asserts
output_config.formatis sentwith the schema and that a pure-JSON text block parses directly) and the default
path (asserts no
output_configis sent).Risk & blast radius
Low and reversible. The new behavior is gated by a default-off flag
(
VITE_FACET_STRUCTURED_OUTPUTS) plus a per-call override, so production behavioris unchanged until the flag is flipped; rollback is a config change, not a revert.
The existing semantic shape-check is retained as a content backstop (structured
outputs guarantee shape, not content). No proxy change is required —
output_configalready flows through
normalizeOutputConfiguntouched.Protected paths: touches
.env.example(one default-off flag line) and theenv-flag wiring in
src/utils/facetEnv.ts. No changes to proxy, supabase,identity, persistence, engine, store, or migrations. No data migration.
Open questions / follow-ups (all tracked on #103)
thinking+formatis untested — neither pilot call passes a thinkingbudget, so the structured-outputs × adaptive-thinking interaction isn't
exercised here. Test it on the first thinking-enabled call that migrates.
identity.extractneeds themetrics/constraintsschema-shape decisionbefore it can migrate.
Part of #103
.env.example)