Skip to content

Commit 8f4368d

Browse files
authored
Delegate built-in reasoning to pi-ai (#4622)
Delegate built-in reasoning configuration to pi-ai instead of rewriting provider request payloads in the model catalog. This preserves existing OpenAI/Codex/Anthropic reasoning defaults while letting pi-ai own provider-specific payload shapes like OpenAI reasoning summaries and Anthropic adaptive thinking. ## Root Cause The built-in model catalog was using `onPayload` to manually mutate raw provider request bodies for reasoning models. That duplicated provider-specific behavior that pi-ai already implements, and made fixes like Anthropic adaptive thinking and OpenAI reasoning summaries easy to drift or conflict. ## Approach - Add first-class `reasoning` and `thinkingBudgets` fields to `AgentConfig`. - Pass those fields through `context-factory` into `createPiAgentAdapter`. - Forward configured reasoning options into pi-ai stream calls. - Remove built-in OpenAI/Codex/Anthropic reasoning payload rewriting from `model-catalog.ts`. - Preserve existing user-facing defaults: - OpenAI auto → `minimal` - OpenAI Codex auto/minimal → `low` - Anthropic auto → `minimal` - Anthropic thinking budgets keep the previous tuned values, including `high: 24576` - Keep `onPayload` available as an advanced escape hatch, but stop using it for built-in reasoning defaults. ## Key Invariants - Built-in reasoning-capable OpenAI, OpenAI Codex, and Anthropic models still enable reasoning by default. - DeepSeek/Kimi behavior is unchanged. - OpenAI Codex still avoids unsupported `minimal` by mapping auto/minimal to `low`. - Anthropic older budget-based models retain the existing token budget tuning. - Anthropic adaptive-thinking models are delegated to pi-ai so they receive the correct adaptive payload shape. ## Non-goals - Does not remove the public/runtime `onPayload` hook. - Does not change model selection or provider availability detection. - Does not attempt to add built-in reasoning defaults for every provider marked as reasoning-capable. - Does not add real-provider integration tests in this PR. ## Trade-offs The main alternative was to keep expanding catalog-level `onPayload` rewrites for each provider-specific reasoning behavior. Delegating to pi-ai is cleaner because pi-ai already tracks provider API details such as OpenAI reasoning summaries and Anthropic adaptive thinking. The trade-off is that Electric Agents now depends more directly on pi-ai’s `reasoning` option behavior, which is already the abstraction we use for provider streaming. ## Verification ```bash pnpm --filter @electric-ax/agents-runtime exec vitest run test/pi-adapter.test.ts pnpm --filter @electric-ax/agents exec vitest run test/model-catalog.test.ts pnpm --filter @electric-ax/agents-runtime exec tsc --noEmit --pretty false pnpm --filter @electric-ax/agents exec tsc --noEmit --pretty false GITHUB_BASE_REF=main node scripts/check-changeset.mjs ``` All passed. Recommended smoke coverage before merge, if provider credentials are available: - OpenAI reasoning model, e.g. `gpt-5` - OpenAI Codex model - Anthropic older budget-based thinking model - Anthropic adaptive-thinking model such as `claude-sonnet-4-6` or Opus 4.7 ## Files changed - `.changeset/delegate-reasoning-pi-ai.md` - Adds patch changeset coverage for `@electric-ax/agents` and `@electric-ax/agents-runtime`. - `packages/agents-runtime/src/types.ts` - Adds `reasoning` and `thinkingBudgets` to `AgentConfig`. - `packages/agents-runtime/src/context-factory.ts` - Passes configured reasoning options into the pi adapter. - `packages/agents-runtime/src/pi-adapter.ts` - Forwards `reasoning` and `thinkingBudgets` into pi-ai stream calls. - `packages/agents-runtime/test/pi-adapter.test.ts` - Adds coverage that configured reasoning options are passed to stream calls. - `packages/agents/src/model-catalog.ts` - Removes built-in provider payload rewriting via `onPayload`. - Resolves built-in reasoning defaults into `AgentConfig.reasoning`. - Preserves Anthropic thinking budgets for pi-ai. - `packages/agents/test/model-catalog.test.ts` - Updates expectations from raw payload mutation to delegated reasoning config.
1 parent 6dbb0bb commit 8f4368d

7 files changed

Lines changed: 126 additions & 183 deletions

File tree

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
'@electric-ax/agents': patch
3+
'@electric-ax/agents-runtime': patch
4+
---
5+
6+
Delegate built-in reasoning configuration to pi-ai instead of rewriting provider request payloads in the model catalog. Built-in OpenAI, OpenAI Codex, and Anthropic reasoning models now pass `reasoning` through the runtime adapter, preserving existing `auto` defaults and Anthropic thinking budgets while letting pi-ai own provider-specific payload shapes.

‎packages/agents-runtime/src/context-factory.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -785,6 +785,9 @@ export function createHandlerContext<TState extends StateProxy = StateProxy>(
785785

786786
getApiKey: activeAgentConfig.getApiKey,
787787

788+
reasoning: activeAgentConfig.reasoning,
789+
thinkingBudgets: activeAgentConfig.thinkingBudgets,
790+
788791
onPayload: activeAgentConfig.onPayload,
789792

790793
onStepEnd: activeAgentConfig.onStepEnd,

‎packages/agents-runtime/src/pi-adapter.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,8 @@ export interface PiAdapterOptions {
7272
getApiKey?: (
7373
provider: string
7474
) => Promise<string | undefined> | string | undefined
75+
reasoning?: SimpleStreamOptions[`reasoning`]
76+
thinkingBudgets?: SimpleStreamOptions[`thinkingBudgets`]
7577
onPayload?: SimpleStreamOptions[`onPayload`]
7678
// Invoked after each step ends with the token counts reported by the
7779
// provider. Used by goal-budget enforcement to abort mid-run; see
@@ -281,6 +283,8 @@ export function createPiAgentAdapter(
281283
const streamFn: StreamFn = (streamModel, context, streamOptions) =>
282284
baseStreamFn(streamModel, context, {
283285
...streamOptions,
286+
...(opts.reasoning && { reasoning: opts.reasoning }),
287+
...(opts.thinkingBudgets && { thinkingBudgets: opts.thinkingBudgets }),
284288
timeoutMs: modelTimeoutMs,
285289
maxRetries: modelMaxRetries,
286290
})

‎packages/agents-runtime/src/types.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -959,6 +959,8 @@ export interface AgentConfig {
959959
getApiKey?: (
960960
provider: string
961961
) => Promise<string | undefined> | string | undefined
962+
reasoning?: SimpleStreamOptions[`reasoning`]
963+
thinkingBudgets?: SimpleStreamOptions[`thinkingBudgets`]
962964
onPayload?: SimpleStreamOptions[`onPayload`]
963965
// Invoked after each step ends with the provider-reported token counts.
964966
// `input` is the full prompt volume (incl. prompt-cache reads/writes, as

‎packages/agents-runtime/test/pi-adapter.test.ts‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -237,6 +237,71 @@ describe(`createPiAgentAdapter`, () => {
237237
expect(seenOptions).toEqual([{ timeoutMs: 1234, maxRetries: 5 }])
238238
})
239239

240+
it(`passes configured reasoning options to stream calls`, async () => {
241+
const seenOptions: Array<{
242+
reasoning?: string
243+
thinkingBudgets?: unknown
244+
}> = []
245+
const completedMessage: AssistantMessage = {
246+
role: `assistant`,
247+
content: [{ type: `text`, text: `ok` }],
248+
api: `anthropic-messages`,
249+
provider: `anthropic`,
250+
model: `claude-sonnet-4-5-20250929`,
251+
usage: {
252+
input: 0,
253+
output: 0,
254+
cacheRead: 0,
255+
cacheWrite: 0,
256+
totalTokens: 0,
257+
cost: {
258+
input: 0,
259+
output: 0,
260+
cacheRead: 0,
261+
cacheWrite: 0,
262+
total: 0,
263+
},
264+
},
265+
stopReason: `stop`,
266+
timestamp: Date.now(),
267+
}
268+
269+
const thinkingBudgets = {
270+
minimal: 1024,
271+
low: 2048,
272+
medium: 8192,
273+
high: 24576,
274+
}
275+
const factory = createPiAgentAdapter({
276+
systemPrompt: `Test system prompt`,
277+
model: `claude-sonnet-4-5-20250929`,
278+
tools: [],
279+
reasoning: `high`,
280+
thinkingBudgets,
281+
streamFn: (_model, _context, options) => {
282+
seenOptions.push({
283+
reasoning: options?.reasoning,
284+
thinkingBudgets: options?.thinkingBudgets,
285+
})
286+
const stream = createAssistantMessageEventStream()
287+
queueMicrotask(() => stream.end(completedMessage))
288+
return stream
289+
},
290+
})
291+
292+
const handle = factory({
293+
entityUrl: `test/entity-1`,
294+
epoch: 1,
295+
messages: [],
296+
outboundIdSeed: { run: 0, step: 0, msg: 0, tc: 0, reasoning: 0 },
297+
writeEvent: (_event: ChangeEvent) => {},
298+
})
299+
300+
await handle.run(`hello`)
301+
302+
expect(seenOptions).toEqual([{ reasoning: `high`, thinkingBudgets }])
303+
})
304+
240305
it(`preserves existing stream options while injecting timeout and retry options`, async () => {
241306
let capturedSignal: AbortSignal | undefined
242307
const completedMessage: AssistantMessage = {

‎packages/agents/src/model-catalog.ts‎

Lines changed: 32 additions & 125 deletions
Original file line numberDiff line numberDiff line change
@@ -50,15 +50,16 @@ type ExplicitReasoningEffort = Exclude<BuiltinReasoningEffort, `auto`>
5050

5151
export type BuiltinAgentModelConfig = Pick<
5252
AgentConfig,
53-
`model` | `provider` | `onPayload` | `getApiKey`
53+
| `model`
54+
| `provider`
55+
| `reasoning`
56+
| `thinkingBudgets`
57+
| `onPayload`
58+
| `getApiKey`
5459
> & {
5560
reasoningEffort?: ExplicitReasoningEffort
5661
}
5762

58-
type PersistedModelConfig = Pick<AgentConfig, `model` | `provider`> & {
59-
reasoningEffort?: ExplicitReasoningEffort
60-
}
61-
6263
const DEFAULT_ANTHROPIC_MODEL = `claude-sonnet-4-6`
6364
const DEFAULT_OPENAI_MODEL = `gpt-4.1`
6465
const DEFAULT_CODEX_MODEL = `gpt-5.4`
@@ -239,134 +240,34 @@ function filterChoicesByEnabledModels(
239240
}
240241

241242
/**
242-
* Anthropic-specific budget mapping for `reasoningEffort`.
243-
*
244-
* Anthropic's `thinking.budget_tokens` is a hard cap on tokens spent
245-
* inside the thinking block before the model must commit to its
246-
* answer. Docs require ≥ 1024; we scale from there. Numbers tuned so
247-
* `medium` is the spot most "show your work" requests land, and
248-
* `high` covers tougher reasoning without uncapped spend.
249-
*
250-
* Keep in sync with provider doc updates — Anthropic has shifted the
251-
* minimum once already (older models capped lower).
243+
* Keep the previous built-in Anthropic token budget tuning while delegating the
244+
* provider-specific request shape to pi-ai. pi-ai only uses these budgets for
245+
* older budget-based Anthropic thinking models; adaptive-thinking models ignore
246+
* them and use the `reasoning` effort level directly.
252247
*/
253-
const ANTHROPIC_THINKING_BUDGET_BY_EFFORT: Record<
254-
ExplicitReasoningEffort,
255-
number
256-
> = {
248+
const ANTHROPIC_THINKING_BUDGETS = {
257249
minimal: 1024,
258250
low: 2048,
259251
medium: 8192,
260252
high: 24576,
261-
}
262-
263-
/**
264-
* Opus 4.6+/4.7 and Sonnet 4.6 use Anthropic's adaptive thinking API
265-
* (`thinking.type: "adaptive"` + `output_config.effort`); the older budget-based
266-
* `thinking.type: "enabled"` is rejected on Opus 4.7.
267-
*/
268-
function isAdaptiveThinkingModel(modelId: string): boolean {
269-
return /(opus-4[-.]6|opus-4[-.]7|sonnet-4[-.]6)/.test(modelId)
270-
}
253+
} satisfies NonNullable<AgentConfig[`thinkingBudgets`]>
271254

272-
/** `reasoningEffort` → Anthropic adaptive `output_config.effort` level. */
273-
const ANTHROPIC_ADAPTIVE_EFFORT_BY_REASONING: Record<
274-
ExplicitReasoningEffort,
275-
string
276-
> = {
277-
minimal: `low`,
278-
low: `low`,
279-
medium: `medium`,
280-
high: `high`,
281-
}
282-
283-
function withProviderPayloadDefaults(
284-
config: PersistedModelConfig & { getApiKey?: AgentConfig[`getApiKey`] },
255+
function resolveBuiltinReasoning(
285256
choice: BuiltinModelChoice,
286257
reasoningEffort: ExplicitReasoningEffort | null
287-
): BuiltinAgentModelConfig {
288-
if (!choice.reasoning) return config
289-
290-
if (choice.provider === `openai` || choice.provider === `openai-codex`) {
291-
const defaultEffort = choice.provider === `openai-codex` ? `low` : `minimal`
292-
const effort =
293-
reasoningEffort === `minimal` && choice.provider === `openai-codex`
294-
? `low`
295-
: (reasoningEffort ?? defaultEffort)
296-
297-
return {
298-
...config,
299-
onPayload: (payload) => {
300-
if (typeof payload !== `object` || payload === null) return undefined
301-
const body = payload as Record<string, unknown>
302-
const existingReasoning =
303-
typeof body.reasoning === `object` && body.reasoning !== null
304-
? (body.reasoning as Record<string, unknown>)
305-
: {}
306-
307-
return {
308-
...body,
309-
reasoning: {
310-
...existingReasoning,
311-
effort,
312-
},
313-
}
314-
},
315-
}
258+
): AgentConfig[`reasoning`] | undefined {
259+
if (!choice.reasoning) return undefined
260+
if (
261+
choice.provider !== `openai` &&
262+
choice.provider !== `openai-codex` &&
263+
choice.provider !== `anthropic`
264+
) {
265+
return undefined
316266
}
317-
318-
if (choice.provider === `anthropic`) {
319-
// `auto` falls through to a `minimal` default, matching the OpenAI branch.
320-
const effectiveEffort = reasoningEffort ?? `minimal`
321-
// pi-ai writes `thinking: { type: "disabled" }` by default; we merge our
322-
// enabled values last so they win.
323-
const mergeThinking = (
324-
body: Record<string, unknown>,
325-
thinking: Record<string, unknown>,
326-
extra?: Record<string, unknown>
327-
): Record<string, unknown> => {
328-
const existing =
329-
typeof body.thinking === `object` && body.thinking !== null
330-
? (body.thinking as Record<string, unknown>)
331-
: {}
332-
return { ...body, ...extra, thinking: { ...existing, ...thinking } }
333-
}
334-
335-
if (isAdaptiveThinkingModel(String(config.model))) {
336-
const effort = ANTHROPIC_ADAPTIVE_EFFORT_BY_REASONING[effectiveEffort]
337-
return {
338-
...config,
339-
onPayload: (payload) => {
340-
if (typeof payload !== `object` || payload === null) return undefined
341-
const body = payload as Record<string, unknown>
342-
const existingOutputConfig =
343-
typeof body.output_config === `object` &&
344-
body.output_config !== null
345-
? (body.output_config as Record<string, unknown>)
346-
: {}
347-
return mergeThinking(
348-
body,
349-
{ type: `adaptive` },
350-
{ output_config: { ...existingOutputConfig, effort } }
351-
)
352-
},
353-
}
354-
}
355-
356-
const budgetTokens = ANTHROPIC_THINKING_BUDGET_BY_EFFORT[effectiveEffort]
357-
return {
358-
...config,
359-
onPayload: (payload) => {
360-
if (typeof payload !== `object` || payload === null) return undefined
361-
return mergeThinking(payload as Record<string, unknown>, {
362-
type: `enabled`,
363-
budget_tokens: budgetTokens,
364-
})
365-
},
366-
}
267+
if (choice.provider === `openai-codex`) {
268+
return reasoningEffort === `minimal` ? `low` : (reasoningEffort ?? `low`)
367269
}
368-
369-
return config
270+
return reasoningEffort ?? `minimal`
370271
}
371272

372273
function parseReasoningEffort(value: unknown): ExplicitReasoningEffort | null {
@@ -443,10 +344,16 @@ export function resolveBuiltinModelConfig(
443344
: undefined
444345

445346
const choice = selected ?? catalog.defaultChoice
446-
const config = {
347+
const reasoning = resolveBuiltinReasoning(choice, reasoningEffort)
348+
const config: BuiltinAgentModelConfig = {
447349
provider: choice.provider,
448350
model: choice.id,
449351
...(reasoningEffort && { reasoningEffort }),
352+
...(reasoning && { reasoning }),
353+
...(choice.reasoning &&
354+
choice.provider === `anthropic` && {
355+
thinkingBudgets: ANTHROPIC_THINKING_BUDGETS,
356+
}),
450357
...(choice.provider === `openai-codex` && {
451358
getApiKey: () => readCodexAccessToken(),
452359
}),
@@ -455,7 +362,7 @@ export function resolveBuiltinModelConfig(
455362
}),
456363
}
457364

458-
return withProviderPayloadDefaults(config, choice, reasoningEffort)
365+
return config
459366
}
460367

461368
export function modelChoiceValues(

0 commit comments

Comments
 (0)