Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@
- Work is tracked in Beads (`bd`). Check open Beads before starting follow-up work.
- The examples GitHub Environment now provides `OPENAI_API_KEY`, `GEMINI_API_KEY`, and `ANTHROPIC_API_KEY` to `.github/workflows/examples.yml`.
- Use `createClientFromProfile(profile, store)` for generic LLM profile dispatch. It routes `providerId`/detected provider to Anthropic, Gemini, OpenAI Responses, or OpenAI-compatible chat. Product/REST callers should select `LLMProfile` records; use explicit provider factories such as `createOpenAIChatClientFromProfile` only for advanced SDK tests or provider-specific code.
- `Agent.step()` passes only usable `ToolDefinition` instances to `LLMClient.complete`; OpenAI clients serialize native tools and omit the request field when none are present. `npm run live:openai-tools` proves real read/edit/finish dispatch with `gpt-5-nano`.

- Gemini `thoughtSignature` round-trip is verified live for Gemini 3.x models (`gemini-3.5-flash`, `gemini-3.1-pro-preview`) using `thinkingConfig.thinkingLevel`. Gemini 2.5 `thinkingBudget` support is intentionally closed as wont-fix because those models are old/unavailable for this SDK target.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Intentional deviations from Python remain: no ACP runtime, security analyzers, r

This package is tracking the Python `agent-sdk` architecture while staying idiomatic TypeScript. The implemented surfaces currently include focused parity coverage for:

- LLM message/content serialization, OpenAI chat completions/Responses, Anthropic, and Gemini request/response mapping
- LLM message/content serialization, Agent-to-LLM `ToolDefinition` propagation, and provider-owned OpenAI chat completions/Responses, Anthropic, and Gemini request/response mapping
- event schemas and `eventsToMessages` conversion, including parallel tool-call batching behavior
- conversation state, local/remote conversations, pause/resume, restore, parallel execution, and stuck detection
- settings/profiles, profile-selected LLM field hygiene, provider/profile-scoped API key references, and keyring-backed secret storage
Expand Down Expand Up @@ -117,6 +117,7 @@ Runnable TypeScript examples live in [`examples/`](examples/) and are checked by
| Example | Covers |
|---------|--------|
| [`hello-world.ts`](examples/hello-world.ts) | Real OpenAI profile completion through the shared env-backed example profile helper |
| [`native-openai-tools.ts`](examples/native-openai-tools.ts) | Real OpenAI Responses read/edit/finish function calls through Agent tool dispatch |
| [`tools.ts`](examples/tools.ts) | Concrete terminal, file editor, glob, grep, and task tracker tools |
| [`profiles-and-secrets.ts`](examples/profiles-and-secrets.ts) | Provider/profile-scoped LLM API key references and secret store usage |
| [`agent-settings.ts`](examples/agent-settings.ts) | Agent settings/profile validation and profile-selected raw LLM field cleanup |
Expand Down
12 changes: 8 additions & 4 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ View + optional Condenser │
↓ │
eventsToMessages │
↓ │
LLMClient.complete
LLMClient.complete(messages, usable tools)
provider client serializes ToolDefinition schemas
dispatchLlmResponse │
├─ content/reasoning → MessageEvent
└─ tool_calls → ActionEvent(s) → ToolDefinition.execute → ObservationEvent(s)
Expand Down Expand Up @@ -86,12 +86,14 @@ A step performs:
1. build a `View` from `ConversationState.events`;
2. optionally condense the view;
3. render context/system prompt suffixes;
4. call `LLMClient.complete(messages)`;
4. call `LLMClient.complete(messages, tools)` with the agent's usable `ToolDefinition`s;
5. dispatch the result with `dispatchLlmResponse()`.

This matches the pinned Python Agent, which passes its resolved `tools_map` values through `make_llm_completion()`. The TypeScript `LLMClient` remains a thin transport boundary: it receives executable tool definitions but does not reshape them. Provider clients that support native tools own their wire format and derive schemas from `ToolDefinition` helpers; Agent and server code must not construct provider-specific tool DTOs.

`LocalConversation` owns the local run loop around an `Agent` and `ConversationState`. `RemoteConversation` mirrors the public shape for an agent-server-backed runtime. `ConversationState` is the append-only event log plus execution status.

`ParallelToolExecutor` runs batches of tool actions with a configurable concurrency limit. This is distinct from Python's confirmation gates: pending/parallel actions are core execution machinery and are retained; confirmation/security policy execution is deliberately not ported.
`dispatchLlmResponse()` preserves every returned tool call as an `ActionEvent`. `ParallelToolExecutor` then runs pending batches with a configurable concurrency limit, so adding tool definitions to completion does not collapse or bypass multi-tool dispatch. This is distinct from Python's confirmation gates: pending/parallel actions are core execution machinery and are retained; confirmation/security policy execution is deliberately not ported.

`StuckDetector` scans recent events for repeated action/observation loops, repeated action/error loops, or agent monologues after the last user turn.

Expand Down Expand Up @@ -128,6 +130,8 @@ The four provider APIs are implemented as the APIs they actually are, not hidden
- Anthropic Messages owns Anthropic content blocks, prompt caching, and extended-thinking details.
- Gemini owns GenerateContent parts, function-call parts, `thoughtSignature` round-tripping, and Gemini thinking config.

For OpenAI, Chat Completions wraps the schema produced from `ToolDefinition.toResponsesTool()` in its nested function-tool shape, while Responses uses the helper's native top-level shape. Both omit the wire-level `tools` field when the supplied list is empty. These are provider-client concerns; the shared completion interface carries `ToolDefinition`s without a parallel DTO layer.

`oh-tab/packages/agent-sdk` was used as inspiration for product-level profile semantics, key lookup shape, and build/test tooling expectations. It was not copied: the implementation is fresh TypeScript, and the older package remains reference-only.

Compatibility details intentionally covered by tests:
Expand Down
4 changes: 2 additions & 2 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ This directory captures the working documentation for `@smolpaws/openhands-agent

## Current docs

- [`ARCHITECTURE.md`](ARCHITECTURE.md) — main component architecture and data flow.
- [`TRANSPILE_PLAN.md`](TRANSPILE_PLAN.md) — upstream target, parity principles, accepted deviations, and remaining roadmap.
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — main component architecture and data flow, including the Agent → LLM `ToolDefinition` contract and provider serialization ownership.
- [`TRANSPILE_PLAN.md`](TRANSPILE_PLAN.md) — upstream target, parity principles, accepted deviations, closed Agent tool-flow gap, and remaining roadmap.
- [`REASONING_CAPABILITIES.md`](REASONING_CAPABILITIES.md) — provider/model-specific reasoning and thinking controls investigation plus proposed API shape.
- [`PROMPT_CACHE_RETENTION.md`](PROMPT_CACHE_RETENTION.md) — GPT-5.6 prompt-cache retention evidence, live probes, and TypeScript SDK implementation decision.
- [`RELEASE_0.3.2.md`](RELEASE_0.3.2.md) — current 0.3.2 provider-native reasoning docs and GPT-5.6 prompt-cache retention release notes.
Expand Down
2 changes: 2 additions & 0 deletions docs/TRANSPILE_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ The 0.2.0 parity line added compatibility/helper exports for smolpaws, profile-s

Additional 0.3.x work added profile-first LLM client dispatch, live-provider hardening, EventLog/FileStore persistence, restore/idempotent seeding behavior, contiguous-index recovery, synchronous lock caveats, and the `FileStore.lockAsync()` follow-up with async EventLog/ConversationState/LocalConversation append paths for server/runtime code that may encounter lock contention.

The Agent → LLM tool-flow gap is closed: `Agent.step()` passes usable `ToolDefinition`s through the thin `LLMClient.complete()` boundary, and provider clients own native schema serialization. OpenAI Chat Completions and Responses coverage plus a live read/edit/finish example guard this pinned-Python behavior. Tool passing is parity work, not an accepted deviation.

Accepted clarification: low-level LLM client classes may remain exported from the npm package as advanced/testing/building blocks. The product/REST boundary must still be **profile-only**: REST callers select LLM profiles, never raw clients or a Python-style bare `LLM` object.


Expand Down
92 changes: 92 additions & 0 deletions examples/native-openai-tools.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';

import {
Agent,
FinishTool,
LocalConversation,
ToolDefinition,
conversationExecutionStatus,
createClientFromProfile,
llmProfileSchema,
} from '@smolpaws/openhands-agent';
import { z } from 'zod';

import { createExampleLlmSecretStore } from './_shared/exampleProfile.js';

const profile = llmProfileSchema.parse({
profileId: 'native-openai-tools-example',
providerId: 'openai',
model: process.env.OPENAI_TOOL_MODEL?.trim() || 'gpt-5-nano',
openAiApiMode: 'responses',
maxOutputTokens: 1_024,
reasoningEffort: 'low',
});
const store = createExampleLlmSecretStore(profile);

if (store === null) {
console.log('native-openai-tools: set OPENAI_API_KEY to run this live tool-invocation example.');
} else {
const root = await mkdtemp(path.join(os.tmpdir(), 'openhands-native-tools-'));
const target = path.join(root, 'README.md');
const calls: string[] = [];
try {
await writeFile(target, 'ORIGINAL\n', 'utf8');
const readFileTool = new ToolDefinition({
name: 'read_file',
description: 'Read the UTF-8 contents of README.md before editing it.',
inputSchema: z.object({ path: z.literal('README.md') }).strict(),
executor: async () => {
calls.push('read_file');
return { content: await readFile(target, 'utf8') };
},
});
const editFileTool = new ToolDefinition({
name: 'edit_file',
description: 'Replace README.md with the requested UTF-8 content.',
inputSchema: z.object({ path: z.literal('README.md'), content: z.string() }).strict(),
executor: async ({ content }) => {
calls.push('edit_file');
await writeFile(target, content, 'utf8');
return { updated: true };
},
});
const conversation = new LocalConversation({
agent: new Agent({
llm: await createClientFromProfile(profile, store),
tools: [readFileTool, editFileTool, FinishTool.create()],
systemPrompt: 'Use native function tools, never textual imitations. Read README.md, replace it with exactly UPDATED_BY_NATIVE_TOOL followed by a newline, then call finish.',
}),
maxIterations: 8,
});

conversation.sendMessage('Perform the requested README update and finish only after verifying the edit tool succeeded.');
await conversation.run();

const actionNames = conversation.state.events
.filter((event) => event.kind === 'ActionEvent')
.map((event) => event.tool_name);
assert(conversation.state.executionStatus === conversationExecutionStatus.FINISHED, `conversation status was ${conversation.state.executionStatus}`);
assert(await readFile(target, 'utf8') === 'UPDATED_BY_NATIVE_TOOL\n', 'README.md was not updated by the edit tool');
assert(calls.includes('read_file'), 'read_file executor was not invoked');
assert(calls.includes('edit_file'), 'edit_file executor was not invoked');
assert(actionNames.includes('finish'), 'finish was not invoked as a native tool');

console.log(JSON.stringify({
example: 'native-openai-tools',
model: profile.model,
execution_status: conversation.state.executionStatus,
native_action_tools: actionNames,
read_executor_invoked: true,
edit_executor_invoked: true,
file_updated: true,
}));
} finally {
await rm(root, { recursive: true, force: true });
}
}

function assert(condition: unknown, message: string): asserts condition {
if (!condition) throw new Error(message);
}
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
"build:types": "tsc -p tsconfig.json",
"lint": "eslint ./src",
"live:llm": "npm run build && node scripts/live/llm-smoke.mjs",
"live:openai-tools": "npm run build && tsx examples/native-openai-tools.ts",
"live:openai-responses-reasoning": "npm run build && tsx scripts/live/openai-responses-reasoning.ts",
"live:anthropic-cache-smoke": "npm run build && tsx scripts/live/anthropic-cache-smoke.ts",
"live:provider-smokes": "npm run live:openai-responses-reasoning && npm run live:anthropic-cache-smoke",
Expand Down
56 changes: 56 additions & 0 deletions src/agent/__tests__/tool-propagation.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
import { describe, expect, it } from 'vitest';
import { z } from 'zod';

import { ConversationState } from '../../conversation/index.js';
import { llmProfileSchema, messageSchema } from '../../llm/index.js';
import type { LLMClient, LLMCompletionResponse } from '../../llm/client.js';
import { ToolDefinition } from '../../tool/index.js';
import { Agent } from '../agent.js';

describe('Agent tool propagation', () => {
it('passes exactly usable tools and dispatches returned calls to the real tool', async () => {
const executions: string[] = [];
const usable = new ToolDefinition({
name: 'record_value',
description: 'Record one value.',
inputSchema: z.object({ value: z.string() }).strict(),
executor: ({ value }) => {
executions.push(value);
return { recorded: value };
},
});
const unusable = new ToolDefinition({
name: 'hidden_tool',
description: 'Must not be advertised.',
inputSchema: z.object({}).strict(),
usable: false,
});
let receivedTools: readonly ToolDefinition[] | undefined;
const llm: LLMClient = {
profile: llmProfileSchema.parse({ profileId: 'test', providerId: 'test', model: 'test' }),
async complete(_messages, tools): Promise<LLMCompletionResponse> {
receivedTools = tools;
return {
message: messageSchema.parse({
role: 'assistant',
content: [],
tool_calls: [{ id: 'call-1', responses_item_id: null, name: 'record_value', arguments: '{"value":"from-llm"}', origin: 'completion' }],
}),
usage: null,
};
},
};
const agent = new Agent({ llm, tools: [usable, unusable] });

const events = await agent.step(new ConversationState());

expect(receivedTools).toEqual([usable]);
expect(executions).toEqual(['from-llm']);
expect(events.map((event) => event.kind)).toEqual(['ActionEvent', 'ObservationEvent']);
expect(events[1]).toMatchObject({
kind: 'ObservationEvent',
tool_name: 'record_value',
observation: { recorded: 'from-llm' },
});
});
});
2 changes: 1 addition & 1 deletion src/agent/agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ export class Agent {
if (messages === null) {
return [state.events.at(-1)].filter((event): event is Event => event !== undefined);
}
const response = await this.llm.complete(messages);
const response = await this.llm.complete(messages, this.tools.filter((tool) => tool.usable));
return dispatchLlmResponse(response, state, (action) => this.runTool(action), {
maxConcurrency: this.toolConcurrencyLimit,
});
Expand Down
62 changes: 62 additions & 0 deletions src/llm/__tests__/openai-client.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import { describe, expect, it } from 'vitest';
import { z } from 'zod';

import { InMemorySecretStore, llmProviderSecretRef, llmProfileSecretRef } from '../../secrets/index.js';
import { ToolDefinition } from '../../tool/index.js';
import { textContent } from '../index.js';
import {
OpenAIChatClient,
Expand Down Expand Up @@ -142,6 +144,66 @@ describe('profile-resolved OpenAI-compatible chat client', () => {
});
});

describe('OpenAI native tool serialization', () => {
const tool = new ToolDefinition({
name: 'read_file',
description: 'Read a UTF-8 text file.',
inputSchema: z.object({ path: z.string().describe('Absolute file path') }).strict(),
});
const message = { role: 'user' as const, content: [textContent('Read the file.')] };

it('sends Chat Completions function tools in provider-native shape', async () => {
const profile = llmProfileSchema.parse({ profileId: 'chat-tools', providerId: 'openai', model: 'gpt-4.1' });
const calls: FakeFetchCall[] = [];
const client = new OpenAIChatClient(profile, 'test-key', fakeFetch({ content: 'ok' }, calls));

await client.complete([message], [tool]);

expect(calls[0]?.body.tools).toEqual([
{
type: 'function',
function: {
name: tool.name,
description: tool.description,
parameters: tool.toResponsesTool().parameters,
strict: false,
},
},
]);
});

it('sends Responses function tools using ToolDefinition.toResponsesTool()', async () => {
const profile = llmProfileSchema.parse({ profileId: 'responses-tools', providerId: 'openai', model: 'gpt-5-nano', openAiApiMode: 'responses' });
const calls: FakeFetchCall[] = [];
const client = new OpenAIResponsesClient(profile, 'test-key', fakeResponsesFetch({ content: 'ok' }, calls));

await client.complete([message], [tool]);

expect(calls[0]?.body.tools).toEqual([tool.toResponsesTool()]);
});

it('omits tools for no-tools requests instead of sending an empty field', async () => {
const chatCalls: FakeFetchCall[] = [];
const responsesCalls: FakeFetchCall[] = [];
const chat = new OpenAIChatClient(
llmProfileSchema.parse({ profileId: 'chat-no-tools', providerId: 'openai', model: 'gpt-4.1' }),
'test-key',
fakeFetch({ content: 'ok' }, chatCalls),
);
const responses = new OpenAIResponsesClient(
llmProfileSchema.parse({ profileId: 'responses-no-tools', providerId: 'openai', model: 'gpt-5-nano', openAiApiMode: 'responses' }),
'test-key',
fakeResponsesFetch({ content: 'ok' }, responsesCalls),
);

await chat.complete([message]);
await responses.complete([message], []);

expect(chatCalls[0]?.body).not.toHaveProperty('tools');
expect(responsesCalls[0]?.body).not.toHaveProperty('tools');
});
});

describe('OpenAI chat message serialization parity', () => {
it('drops empty assistant content when tool calls are present', () => {
const profile = llmProfileSchema.parse({ profileId: 'default', providerId: 'openai', model: 'gpt-5.1' });
Expand Down
3 changes: 2 additions & 1 deletion src/llm/client.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { z } from 'zod';

import type { ToolDefinition } from '../tool/index.js';
import { messageSchema, type LLMProfile, type Message } from './index.js';

export interface FetchResponseLike {
Expand All @@ -16,7 +17,7 @@ export type FetchLike = (

export interface LLMClient {
readonly profile: LLMProfile;
complete(messages: readonly Message[]): Promise<LLMCompletionResponse>;
complete(messages: readonly Message[], tools?: readonly ToolDefinition[]): Promise<LLMCompletionResponse>;
}

export const llmUsageSchema = z
Expand Down
Loading