Audit remediation: tests, isolate hardening, framework fixes, error hygiene (#465)

* test(ai-code-mode-skills): add unit test coverage for skill library

The package had 13 source files with zero unit tests. Added 116 tests
across 9 files covering trust strategies, memory + file storage, skill
management tools (including name-validation boundaries for register_skill),
bindings, skills-to-tools execution with mocked isolate driver, type
generation, the system-prompt renderer, and skill selection with a
mocked chat adapter.

* feat(ai-isolate-cloudflare): support production deployments and harden tool-name handling

The Worker was documented, commented, and configured as if unsafe_eval
only worked in wrangler dev. Updated src/worker/index.ts, wrangler.toml,
and the README to describe the production path (Cloudflare accounts
with the unsafe_eval binding enabled), and pointed users to auth /
rate limiting as the real production gate.

Also added assertSafeToolName in wrap-code.ts to reject tool names that
would break out of the generated function identifier, e.g.
"foo'); process.exit(1); (function bar() {". Added tests covering
quotes, backticks, whitespace, semicolons, newlines, empty strings,
leading digits, and the valid identifier shapes.

Added a new escape-attempts.test.ts covering JSON.stringify escaping
of adversarial tool-result values and verifying the result lands in
a plain object-literal assignment (never a template literal).

* refactor(ai-ollama): extract tool-converter with test coverage

Tool handling was inlined inside the text adapter with raw type casts.
Extracted into src/tools/function-tool.ts + tool-converter.ts matching
the structure used by ai-openai, ai-anthropic, ai-grok, and ai-groq.
Re-exported as convertFunctionToolToAdapterFormat and
convertToolsToProviderFormat from the package index.

Added 29 unit tests covering the converter, client utilities
(createOllamaClient, getOllamaHostFromEnv, generateId, estimateTokens),
and the text adapter's streaming behaviour: RUN/TEXT_MESSAGE/tool-call
lifecycle events, id synthesis when Ollama omits a tool-call id, tool
forwarding to the SDK in provider format, and structured-output JSON
parsing with error wrapping.

The package previously had 73 source files and zero unit tests.

* fix(frameworks): propagate useChat callback changes after re-render

onResponse, onChunk, and onCustomEvent were captured by reference at
ChatClient creation time. When a parent component re-rendered with
fresh closures, the client kept calling the originals.

- ai-react / ai-preact: wrap the three callbacks the same way
  onFinish/onError already were, reading from optionsRef.current at
  call time.
- ai-vue / ai-solid: wrap the callbacks to read options.xxx at call
  time. This also fixes a subtler bug where using client.updateOptions
  to swap callbacks could not clear them (the "!== undefined" guard
  silently skipped undefined values).
- ai-svelte: documented the capture-at-creation behaviour — Svelte's
  createChat runs once per instance and there's no per-render hook, so
  callbacks are frozen unless the caller mutates the options object or
  calls client.updateOptions imperatively.

Added a React regression test that rerenders with a new onChunk and
verifies the new callback fires while the original does not.

* refactor(ai, ai-openai): narrow error handling and stop logging raw errors

The three catch blocks that convert thrown values into RUN_ERROR events
(stream-to-response.ts, activities/stream-generation-result.ts,
activities/generateVideo/index.ts) were using catch(error: any) and
dereferencing .message / .code without checks. Added a shared
toRunErrorPayload(error, fallback) helper under activities/ that accepts
Error instances, plain objects with message/code fields, or bare strings,
and funnels all three sites through it with a per-site fallback message.

Removed four console.error calls in the OpenAI text adapter's chatStream
that dumped the full error object to stdout. SDK errors can carry the
original request (including auth headers), so the library no longer logs
them; upstream callers should convert errors into structured events.

Added 8 unit tests for toRunErrorPayload including a leaked-properties
test confirming the helper does not expose extra fields.

* test(isolates): add sandbox escape-attempt tests for Node and QuickJS drivers

Covers the attack surface a malicious skill / code-mode snippet might
probe: process/require/fetch should be unavailable, prototype pollution
must not leak to the host or between contexts, synchronous CPU-spin
loops must be interrupted by the timeout (not hang), and Function-
constructor escape attempts must execute inside the isolate (never
returning a real host process object).

QuickJS also gets a test that globalThis mutations inside one context
do not bleed into a sibling context.

* ci: apply automated fixes

* fix: address PR review feedback

- ai-preact: forward onCustomEvent in useChat (changeset claimed the fix
  covered preact but it was silently dropped before reaching ChatClient).
- ai-isolate-cloudflare: reject JS reserved keywords as tool names
  (return, class, function, if, await, ...) so the wrapper fails fast
  at generation time instead of with a cryptic SyntaxError at eval.
- ai/src/activities/error-payload: apply typeof string check to the
  Error branch's code field, matching the plain-object branch.
  Some SDKs attach numeric or Symbol codes to Error instances.
- ai-ollama text-adapter test: strengthen OLLAMA_HOST assertion by
  tracking the mocked Ollama constructor args, so the test fails if
  the env var is ignored.
- ai-ollama utils test: rename 'when OLLAMA_HOST is unset' to 'empty'
  since the setup stubs an empty string.
- ai-code-mode-skills file-storage test: use vi.useFakeTimers() for
  the createdAt/updatedAt round-trip instead of a 5ms real sleep.

* ci: apply automated fixes

* fix(ai, ai-ollama): merge-driven regressions from CR

Address CR findings after merging main:

- ai-ollama tests: inject testLogger (from resolveDebugOption(false))
  into every adapter.chatStream and adapter.structuredOutput call —
  main's #467 made `logger` required on TextOptions, the PR's new tests
  were written against the pre-#467 contract and crashed at runtime on
  `logger.errors`/`logger.request` dereference.
- generateVideo: narrow `error` via toRunErrorPayload before handing it
  to logger.errors. Previously passed the raw error object through the
  logger meta, which would surface SDK request state (headers, payloads)
  to any user-supplied logger — defeating the hardening the PR applies
  to the RUN_ERROR event.
- error-narrowing changeset: update wording to match actual code. The
  OpenAI text adapter's chatStream still logs under the merge, but now
  through the narrowed `{message, code}` payload rather than raw errors.
  Changeset previously claimed "the library now re-throws without
  logging", which didn't match shipped behavior.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

Author: AlemTuzlak

.changeset/error-narrowing.md

@@ -0,0 +1,10 @@
+---
+'@tanstack/ai': patch
+'@tanstack/ai-openai': patch
+---
+
+refactor(ai, ai-openai): narrow error handling before logging
+
+`catch (error: any)` sites in `stream-to-response.ts`, `activities/stream-generation-result.ts`, and `activities/generateVideo/index.ts` are now narrowed to `unknown` and funnel through a shared `toRunErrorPayload(error, fallback)` helper that extracts `message` / `code` without leaking the original error object (which can carry request state from an SDK).
+
+Replaced four `console.error` calls in the OpenAI text adapter's `chatStream` catch block that dumped the full error object to stdout. SDK errors can carry the original request including auth headers, so the library now logs only the narrowed `{ message, code }` payload via the internal logger — any user-supplied logger receives the sanitized shape, not the raw SDK error.

.changeset/isolate-cloudflare-production.md

@@ -0,0 +1,7 @@
+---
+'@tanstack/ai-isolate-cloudflare': patch
+---
+
+feat(ai-isolate-cloudflare): support production deployments and close tool-name injection vector
+
+The Worker now documents production-capable `unsafe_eval` usage (previously the code, wrangler.toml, and README all described it as dev-only). Tool names are validated against a strict identifier regex before being interpolated into the generated wrapper code, so a malicious tool name like `foo'); process.exit(1); (function bar() {` is rejected at generation time rather than breaking out of the wrapping function.

.changeset/ollama-tool-converter.md

@@ -0,0 +1,7 @@
+---
+'@tanstack/ai-ollama': patch
+---
+
+refactor(ai-ollama): extract tool conversion into `src/tools/` matching peer adapters
+
+Tool handling lived inline inside the text adapter with raw type casts. It is now split into a dedicated `tool-converter.ts` / `function-tool.ts` pair (mirroring the structure used by `ai-openai`, `ai-anthropic`, `ai-grok`, and `ai-groq`) and re-exported from the package index as `convertFunctionToolToAdapterFormat` and `convertToolsToProviderFormat`. Runtime behavior is unchanged.

.changeset/useChat-callback-propagation.md

@@ -0,0 +1,10 @@
+---
+'@tanstack/ai-react': patch
+'@tanstack/ai-preact': patch
+'@tanstack/ai-vue': patch
+'@tanstack/ai-solid': patch
+---
+
+fix(ai-react, ai-preact, ai-vue, ai-solid): propagate `useChat` callback changes
+
+`onResponse`, `onChunk`, and `onCustomEvent` were captured by reference at client creation time. When a parent component re-rendered with fresh closures, the `ChatClient` kept calling the originals. Every framework now wraps these callbacks so the latest `options.xxx` is read at call time (via `optionsRef.current` in React/Preact, and direct option access in Vue/Solid, matching the pattern already used for `onFinish` / `onError`). Clearing a callback (setting it to `undefined`) now correctly no-ops instead of continuing to invoke the stale handler.

packages/typescript/ai-code-mode-skills/tests/create-skill-management-tools.test.ts

@@ -0,0 +1,302 @@
+import { describe, expect, it, vi } from 'vitest'
+import { createSkillManagementTools } from '../src/create-skill-management-tools'
+import { createMemorySkillStorage } from '../src/storage/memory-storage'
+import {
+  createAlwaysTrustedStrategy,
+  createDefaultTrustStrategy,
+} from '../src/trust-strategies'
+
+const mockContext = () => ({ emitCustomEvent: vi.fn() })
+
+function getTool(
+  tools: ReturnType<typeof createSkillManagementTools>,
+  name: string,
+) {
+  const tool = tools.find((t) => t.name === name)
+  if (!tool) throw new Error(`Tool ${name} not found`)
+  return tool
+}
+
+function validRegisterInput(
+  overrides: Partial<{
+    name: string
+    description: string
+    code: string
+    inputSchema: string
+    outputSchema: string
+    usageHints: Array<string>
+    dependsOn: Array<string>
+  }> = {},
+) {
+  return {
+    name: 'fetch_data',
+    description: 'A skill',
+    code: 'return input;',
+    inputSchema: '{"type":"object","properties":{}}',
+    outputSchema: '{"type":"object","properties":{}}',
+    usageHints: ['Use for fetching'],
+    dependsOn: [],
+    ...overrides,
+  }
+}
+
+describe('createSkillManagementTools', () => {
+  it('exposes search_skills, get_skill, and register_skill', () => {
+    const storage = createMemorySkillStorage([])
+    const tools = createSkillManagementTools({ storage })
+    expect(tools.map((t) => t.name).sort()).toEqual([
+      'get_skill',
+      'register_skill',
+      'search_skills',
+    ])
+  })
+
+  describe('search_skills', () => {
+    it('returns lightweight matching entries', async () => {
+      const storage = createMemorySkillStorage([
+        {
+          id: '1',
+          name: 'github_stats',
+          description: 'GitHub stats',
+          code: 'secret',
+          inputSchema: {},
+          outputSchema: {},
+          usageHints: ['for github'],
+          dependsOn: [],
+          trustLevel: 'untrusted',
+          stats: { executions: 0, successRate: 0 },
+          createdAt: '',
+          updatedAt: '',
+        },
+      ])
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'search_skills')
+      const results = (await tool.execute!(
+        { query: 'github', limit: 5 },
+        mockContext() as any,
+      )) as Array<Record<string, unknown>>
+      expect(results).toHaveLength(1)
+      expect(results[0]).not.toHaveProperty('code')
+      expect(results[0]!.name).toBe('github_stats')
+    })
+
+    it('respects the limit parameter', async () => {
+      const storage = createMemorySkillStorage([
+        {
+          id: 'a',
+          name: 'data_one',
+          description: '',
+          code: '',
+          inputSchema: {},
+          outputSchema: {},
+          usageHints: [],
+          dependsOn: [],
+          trustLevel: 'untrusted',
+          stats: { executions: 0, successRate: 0 },
+          createdAt: '',
+          updatedAt: '',
+        },
+        {
+          id: 'b',
+          name: 'data_two',
+          description: '',
+          code: '',
+          inputSchema: {},
+          outputSchema: {},
+          usageHints: [],
+          dependsOn: [],
+          trustLevel: 'untrusted',
+          stats: { executions: 0, successRate: 0 },
+          createdAt: '',
+          updatedAt: '',
+        },
+      ])
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'search_skills')
+      const results = (await tool.execute!(
+        { query: 'data', limit: 1 },
+        mockContext() as any,
+      )) as Array<unknown>
+      expect(results).toHaveLength(1)
+    })
+  })
+
+  describe('get_skill', () => {
+    it('returns an error object for a missing skill', async () => {
+      const storage = createMemorySkillStorage([])
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'get_skill')
+      const result = (await tool.execute!(
+        { name: 'missing' },
+        mockContext() as any,
+      )) as { error?: string }
+      expect(result.error).toContain('not found')
+    })
+
+    it('returns the full skill including code when found', async () => {
+      const storage = createMemorySkillStorage([
+        {
+          id: '1',
+          name: 'alpha',
+          description: 'Alpha',
+          code: 'return 1;',
+          inputSchema: { type: 'object' },
+          outputSchema: { type: 'number' },
+          usageHints: ['hint'],
+          dependsOn: [],
+          trustLevel: 'untrusted',
+          stats: { executions: 0, successRate: 0 },
+          createdAt: '',
+          updatedAt: '',
+        },
+      ])
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'get_skill')
+      const result = (await tool.execute!(
+        { name: 'alpha' },
+        mockContext() as any,
+      )) as {
+        name?: string
+        code?: string
+        inputSchema?: string
+      }
+      expect(result.name).toBe('alpha')
+      expect(result.code).toBe('return 1;')
+      expect(result.inputSchema).toBe('{"type":"object"}')
+    })
+  })
+
+  describe('register_skill', () => {
+    it('rejects names starting with external_', async () => {
+      const storage = createMemorySkillStorage([])
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'register_skill')
+      const result = (await tool.execute!(
+        validRegisterInput({ name: 'external_evil' }),
+        mockContext() as any,
+      )) as { error?: string }
+      expect(result.error).toContain("cannot start with 'external_'")
+    })
+
+    it('rejects names starting with skill_ (redundant prefix)', async () => {
+      const storage = createMemorySkillStorage([])
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'register_skill')
+      const result = (await tool.execute!(
+        validRegisterInput({ name: 'skill_duplicate' }),
+        mockContext() as any,
+      )) as { error?: string }
+      expect(result.error).toContain("should not include the 'skill_' prefix")
+    })
+
+    it('rejects malformed JSON inputSchema', async () => {
+      const storage = createMemorySkillStorage([])
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'register_skill')
+      const result = (await tool.execute!(
+        validRegisterInput({ inputSchema: 'not valid json' }),
+        mockContext() as any,
+      )) as { error?: string }
+      expect(result.error).toContain('inputSchema must be a valid JSON string')
+    })
+
+    it('rejects malformed JSON outputSchema', async () => {
+      const storage = createMemorySkillStorage([])
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'register_skill')
+      const result = (await tool.execute!(
+        validRegisterInput({ outputSchema: '{' }),
+        mockContext() as any,
+      )) as { error?: string }
+      expect(result.error).toContain('outputSchema must be a valid JSON string')
+    })
+
+    it('rejects a duplicate name', async () => {
+      const storage = createMemorySkillStorage([
+        {
+          id: '1',
+          name: 'existing',
+          description: '',
+          code: '',
+          inputSchema: {},
+          outputSchema: {},
+          usageHints: [],
+          dependsOn: [],
+          trustLevel: 'untrusted',
+          stats: { executions: 0, successRate: 0 },
+          createdAt: '',
+          updatedAt: '',
+        },
+      ])
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'register_skill')
+      const result = (await tool.execute!(
+        validRegisterInput({ name: 'existing' }),
+        mockContext() as any,
+      )) as { error?: string }
+      expect(result.error).toContain('already exists')
+    })
+
+    it('persists a valid skill with defaults', async () => {
+      const storage = createMemorySkillStorage([])
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'register_skill')
+      const result = (await tool.execute!(
+        validRegisterInput({ name: 'valid_skill' }),
+        mockContext() as any,
+      )) as { success?: boolean; skillId?: string }
+      expect(result.success).toBe(true)
+      expect(result.skillId).toMatch(/^[0-9a-f-]{36}$/)
+
+      const saved = await storage.get('valid_skill')
+      expect(saved).not.toBeNull()
+      expect(saved!.stats).toEqual({ executions: 0, successRate: 0 })
+    })
+
+    it('applies the trust strategy to set initial trust level', async () => {
+      const storage = createMemorySkillStorage([])
+      const tools = createSkillManagementTools({
+        storage,
+        trustStrategy: createAlwaysTrustedStrategy(),
+      })
+      const tool = getTool(tools, 'register_skill')
+      await tool.execute!(
+        validRegisterInput({ name: 's1' }),
+        mockContext() as any,
+      )
+      const saved = await storage.get('s1')
+      expect(saved!.trustLevel).toBe('trusted')
+    })
+
+    it('prefers explicit trustStrategy over storage.trustStrategy', async () => {
+      const storage = createMemorySkillStorage({
+        trustStrategy: createAlwaysTrustedStrategy(),
+      })
+      const tools = createSkillManagementTools({
+        storage,
+        trustStrategy: createDefaultTrustStrategy(),
+      })
+      const tool = getTool(tools, 'register_skill')
+      await tool.execute!(
+        validRegisterInput({ name: 's1' }),
+        mockContext() as any,
+      )
+      const saved = await storage.get('s1')
+      expect(saved!.trustLevel).toBe('untrusted')
+    })
+
+    it('falls back to storage.trustStrategy when none provided', async () => {
+      const storage = createMemorySkillStorage({
+        trustStrategy: createAlwaysTrustedStrategy(),
+      })
+      const tools = createSkillManagementTools({ storage })
+      const tool = getTool(tools, 'register_skill')
+      await tool.execute!(
+        validRegisterInput({ name: 's1' }),
+        mockContext() as any,
+      )
+      const saved = await storage.get('s1')
+      expect(saved!.trustLevel).toBe('trusted')
+    })
+  })
+})