Skip to content

Send a field's numeric and length bounds to the provider - #1683

Merged
Muizzkolapo merged 3 commits into
mock-answers-in-the-shape-a-prompt-offersfrom
schema-bounds-reach-the-provider
Nov 8, 2026
Merged

Muizzkolapo merged 3 commits into
mock-answers-in-the-shape-a-prompt-offersfrom
schema-bounds-reach-the-provider

Conversation

@Muizzkolapo

@Muizzkolapo Muizzkolapo commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • The schema compiler copied a fixed list of field keys. That list left out minimum, maximum, minLength and maxLength, so no provider was told a field's range.
  • compile_field now copies the four keys for every vendor. No vendor strips them: no provider documents a rejection of them when strict mode is off, and agac never sets strict. exclusiveMinimum, exclusiveMaximum and multipleOf are still not sent.
  • Two checks read the schema as it is sent, so they now hold the bounds too: a declared endpoint fails a reply outside minimum or maximum, and a tool action refuses output that breaks any of the four. An LLM action's own output check reads the schema file and already held minimum and maximum.
  • The schema reference has a per-vendor table of how the schema is sent and whether the vendor enforces the bounds.

Verification

Gate Result
RED (tests/unit/output/response/test_schema_bounds_reach_the_provider.py on the parent) 14 or more assertion failures
Review (risk LOW, 1 blind verifier) NO: the code was right, but the changelog and docs named the wrong checks and did not announce the tool-output refusal
Fix and re-review Docs and changelog rewritten; tests for the tool refusal and for the keys that stay out. Re-review NO on the cohere and gemini rows of the vendor table, fixed, then YES
Full suite, ruff, mypy 17327 passed, 37 skipped
  • The reviewer measured the two new checks against the parent: a declared-endpoint reply with score: 9 (maximum 4) passed on the parent and fails at this layer; a tool output with a too-long label passed on the parent and is refused here.
  • One suite run failed test_a_mapping_at_the_end_of_a_deep_chain_is_placed, a wall-clock test (elapsed < 2.0), at machine load average about 16. It passes alone 3 of 3, and the rerun passed. It is noted on the coordination board.
  • Smoke ran on the bottom layer only, as asked.

@Muizzkolapo
Muizzkolapo added this pull request to stack #1685 November 7, 2026 19:20
@Muizzkolapo
Muizzkolapo force-pushed the schema-bounds-reach-the-provider branch from d79009a to 145b309 Compare November 7, 2026 19:29
@Muizzkolapo
Muizzkolapo force-pushed the schema-bounds-reach-the-provider branch from 145b309 to 424a04e Compare November 8, 2026 01:16
@Muizzkolapo
Muizzkolapo marked this pull request as ready for review November 8, 2026 09:59
@Muizzkolapo
Muizzkolapo force-pushed the schema-bounds-reach-the-provider branch 2 times, most recently from 71e4310 to c78709b Compare November 8, 2026 14:41
The schema compiler copied a fixed list of field keys. That list
left out minimum, maximum, minLength and maxLength, so no provider
was told a field's range. The schema file accepted these keys and
the docs showed them.

compile_field now copies the four keys for every vendor. No vendor
strips them: no provider documents a rejection of them when strict
mode is off, and agac never sets strict. exclusiveMinimum,
exclusiveMaximum and multipleOf are still not sent.

The schema reference now has a table that shows how each vendor
receives the schema and whether it enforces the bounds.
The Validation Constraints table still said minimum, maximum,
minLength and maxLength are not sent to the provider. The section
below it on the same page says they are. The table now says they
are sent, and lists exclusiveMaximum and multipleOf with
exclusiveMinimum as not sent.
The changelog and docs credited the compiled schema with checks
that read the schema file: an LLM action's output check and repair
already held minimum and maximum before this change. They also said
string length is never checked on output, and they did not say that
tool output is now refused.

Two checks read the schema as it is sent and now hold the bounds: a
declared endpoint fails a reply outside minimum or maximum, and a
tool action refuses output that breaks any of the four. The docs
say so, the constraint table is scoped to LLM actions, and the
vendor table gives cohere and gemini their own rows. Tests pin the
tool refusal and that exclusiveMinimum, exclusiveMaximum and
multipleOf stay out of the compiled schema.
@Muizzkolapo
Muizzkolapo force-pushed the schema-bounds-reach-the-provider branch from c78709b to c7e3c93 Compare November 8, 2026 14:54
@Muizzkolapo
Muizzkolapo merged commit f89987e into main Nov 8, 2026
10 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 8, 2026
@Muizzkolapo
Muizzkolapo deleted the schema-bounds-reach-the-provider branch November 8, 2026 17:08
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant