Skip to content

OpenAPI connections pass through default/example values that contradict the schema's own type, causing models to emit invalid tool input #2369

Description

@Josh-moreton

Summary

defineOpenAPIConnection copies each property's default and example into the model-facing JSON Schema verbatim, even when those values contradict the property's declared type. Models follow the example, produce input that fails the generated zod schema, and the tool call dies.

Real specs do this. Xero's official xero_accounting.yaml has 10 boolean properties whose default and example are quoted strings.

Reproduction

Minimal spec exhibiting the pattern:

export default defineOpenAPIConnection({
  baseUrl: "https://api.example.com",
  description: "Repro.",
  spec: {
    openapi: "3.0.3",
    info: { title: "Repro", version: "1.0.0" },
    paths: {
      "/things": {
        post: {
          operationId: "createThing",
          requestBody: {
            content: {
              "application/json": {
                schema: {
                  type: "object",
                  properties: {
                    HasAttachments: {
                      type: "boolean",
                      default: "false",   // string, contradicts type
                      example: "false",   // string, contradicts type
                    },
                  },
                },
              },
            },
          },
          responses: { "200": { description: "OK" } },
        },
      },
    },
  },
});

The model reliably emits "HasAttachments": "false" and the call fails:

AI_InvalidToolInputError: Invalid input for tool example__createThing:
AI_TypeValidationError: Type validation failed
  path: ["body","HasAttachments"]
  "Invalid input: expected boolean, received string"

Real-world case

Xero's spec, pinned at commit 853dc01:

HasAttachments:
  description: boolean to indicate if an invoice has an attachment
  readOnly: true
  type: boolean
  default: "false"
  example: "false"

Scanning that spec for booleans carrying string default/example:

ApprovedForSending, FromIsReconciled, HasAccount, HasAttachments,
HasErrors, HasValidationErrors, IncludePDF, MarkAsSent, SendCopy,
ToIsReconciled

MarkAsSent, IncludePDF and SendCopy are emailInvoice parameters — a model is meant to set those, so it cannot simply be told to omit them.

This bit us in production on a createInvoices call. Four fields failed at once, all from copied examples.

Where

dist/src/runtime/connections/openapi-schema.js. derefSchema walks every key and recurses:

let a={};
for(let[e,o]of Object.entries(n)) a[e]=derefSchema(t,o,r+1,i);
return normalizeSchemaType(a), normalizeNullable(a), a

normalizeSchemaType strips type values that aren't valid JSON Schema types, and normalizeNullable folds nullable into type — so there is already a normalization pass here, it just doesn't look at default or example.

Suggested fix

In that same pass, drop default and example when they don't validate against the node's own type. A schema that contradicts itself is a spec bug, and the annotation is the wrong half to trust — dropping it costs the model nothing, while keeping it actively teaches the wrong type.

readOnly: true properties appearing in request schemas at all is arguably a second, separate issue.

Versions

  • eve 0.39.0
  • ai 7.0.66
  • Node 24, deployed on Vercel

Metadata

Metadata

Assignees

No one assigned

    Labels

    apibugSomething isn't workingp1

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions