cl-llm-provider-api

Portable Common Lisp protocols and value types for language-model providers and bounded inference inputs.

The library defines semantic provider events and results, concrete wire clients, context assembly, structured-output contracts, shared inference budgets, read-only views, and content-addressed context objects. Applications own credential stores, model catalogs, registries, prompt policy, conversation projection, and tool dispatch.

Systems and package

• ASDF system: cl-llm-provider-api
• Optional concrete protocols: cl-llm-provider-api/wire
• Optional HTTP condition adapter: cl-llm-provider-api/dexador
• Test system: cl-llm-provider-api/tests
• Package: CL-LLM-PROVIDER-API
• Nickname: LLM-PROVIDER-API

Install the locked dependencies and run all tests:

./script/bootstrap
./script/check

Projected requests and execution

Load cl-llm-provider-api/wire for Responses, Chat Completions, and Anthropic Messages encoding and stream assembly. Use hash tables for JSON objects, vectors for arrays, and portable Responses-shaped transcript items. Subclass the existing provider classes to specialize wire naming, instruction placement, and provider metadata without introducing application objects into the wire layer.

Construct WIRE-REQUEST with :model, :items, :prefix, :suffix, and an :options property list. Supply already-projected history and instruction texts. Call PROVIDER-REQUEST-OBJECT with a provider, the projection, and tool vector. Responses accepts namespace declarations; Chat Completions and Messages accept encoded tools. Use PROVIDER-WIRE-TOOLS for protocol-specific tool projection.

Shared options include :maximum-output-tokens. Responses additionally accepts :reasoning-effort, :reasoning-summary, and :fields (alternating wire keys and values). Chat accepts :stream-usage-p, :reasoning-parameter, :reasoning-effort, and :output-ceiling-field. Messages accepts :cache-p and :ephemeral-items for separately projected volatile history.

Call PROVIDER-EXECUTE-REQUEST with :transport, :event-callback, and optional :secrets, :cleanup, :call-with-deadline, and :completion hooks. The transport accepts the request object and returns a stream, HTTP status, and response headers. The library normalizes HTTP and terminal failures and redacts supplied secrets from detached headers, events, results, and diagnostics. Use :cleanup to dispose of each acquired stream; the default abort-closes it. Cleanup is best effort: its errors and deadline failures do not replace the request outcome. Completion runs after valid terminal consumption and cleanup.

Load cl-llm-provider-api/dexador and wrap the request thunk with PROVIDER--CALL-WITH-TRANSPORT-NORMALIZATION to classify socket, TLS, and deadline conditions from transport opening and SSE line reads. Application callbacks run outside that classifier. Pass :terminal-errors-p t only when wrapping a transport opening directly. Convert Dexador HTTP conditions with PROVIDER-SIGNAL-HTTP-FAILURE. Supply credentials and authenticated request headers from the application. Resampling and transient retries use separate budgets in CALL-WITH-BOUNDED-RETRIES. Supply :maximum-retries, :delay-function (retry number and condition), and :call-with-attempt for a custom delay policy or attempt scope.

Context assembly and output contracts

Create bounded CONTEXT-CONTRIBUTION values with MAKE-CONTEXT-CONTRIBUTION. Use MAKE-CONTEXT-RESOLVER and CONTEXT-RESOLVE to select contributions by lifetime, supersession, deduplication, conflict group, mandatory class, priority, and available budget. A CONTEXT-SELECTION includes selected and omitted values. Call CONTEXT-SELECTION-COMPLETE only after a successful request to acknowledge successful-delivery lifetimes. Applications supply request identity, token costs, rendering, and evidence trust boundaries.

Use OUTPUT-SCHEMA-NORMALIZE for the supported native schema subset and OUTPUT-SCHEMA->JSON for its JSON representation. Validate values with OUTPUT-SCHEMA-VALID-P. Decode exact JSON through OUTPUT-JSON-DECODE and convert portable tagged values through OUTPUT-JSON->SEXP and OUTPUT-SEXP->JSON. Object properties and required keys, arrays, scalar JSON types, enums, alternatives, and additional-property constraints are supported. Invalid contracts and values signal OUTPUT-CONTRACT-ERROR and OUTPUT-VALUE-ERROR with field diagnostics.

Provider values

All values are ordinary CLOS instances created with MAKE-INSTANCE.

• PROVIDER-EVENT is the root event class.
• ASSISTANT-DELTA-EVENT and REASONING-DELTA-EVENT carry streamed text.
• PROVIDER-ITEM-EVENT carries one completed provider-native item.
• PROVIDER-COMPLETED-EVENT carries response identity, usage, and turn state.
• PROVIDER-RETRY-EVENT carries bounded reconnect metadata.
• PROVIDER-RESULT carries ordered output items, tool calls, usage, routing state, and :CONTINUE, :END, or :UNSPECIFIED turn completion.

(make-instance 'llm-provider-api:provider-result
               :response-id "response-1"
               :output-items (list message)
               :tool-calls calls
               :usage '(:total-tokens 42)
               :turn-completion :end)

Output items and usage metadata deliberately have type T. A provider may use hash tables, association lists, structures, or application-specific objects.

Provider protocol

Subclass MODEL-PROVIDER and implement PROVIDER-STREAM-TURN. The callback receives semantic PROVIDER-EVENT instances and the method returns one PROVIDER-RESULT.

(defclass example-provider (llm-provider-api:model-provider) ())

(defmethod llm-provider-api:provider-stream-turn
    ((provider example-provider) conversation
     &key tool-namespaces event-callback goal-context compaction-p)
  (declare (ignore provider tool-namespaces goal-context compaction-p))
  (funcall event-callback
           (make-instance 'llm-provider-api:assistant-delta-event
                          :text "hello"))
  (make-instance 'llm-provider-api:provider-result
                 :output-items (list conversation)
                 :turn-completion :end))

The portable provider protocol also exposes:

• PROVIDER-FAMILY
• PROVIDER-WITH-CONFIGURATION
• PROVIDER-CONSUME-STREAM
• PROVIDER-NATIVE-COMPACT-CONVERSATION
• PROVIDER-SET-REASONING-SUMMARIES
• PROVIDER-OUTPUT-CEILING-P
• PROVIDER-NORMALIZE-OUTPUT-ITEM

Conservative defaults return :CUSTOM, NIL, the unchanged provider, or the unchanged item where a safe default exists.

Wire protocol

RESPONSES-API-PROVIDER and CHAT-COMPLETIONS-PROVIDER identify common wire families. PROVIDER-WIRE-PROTOCOL returns :RESPONSES-API, :CHAT-COMPLETIONS, or :CUSTOM.

Request implementations can specialize:

• PROVIDER-WIRE-TOOL-NAME, PROVIDER-WIRE-TOOL, and PROVIDER-WIRE-TOOLS
• PROVIDER-WIRE-INPUT-ITEM
• PROVIDER-RESPONSES-WIRE-EFFORT
• PROVIDER-RESPONSES-REASONING-SUMMARY
• PROVIDER-RESPONSES-HOSTED-TOOLS
• PROVIDER-RESPONSES-INSTRUCTIONS-PLACEMENT
• PROVIDER-RESPONSES-REQUEST-NAMESPACES
• PROVIDER-RESPONSES-REQUEST-FIELDS
• PROVIDER-REQUEST-OBJECT

The Responses API default joins a namespace and tool name with a dot. Wire objects remain application-defined values, so the library does not impose a JSON representation. Responses instructions use conversation input items by default. Providers may return :TOP-LEVEL from PROVIDER-RESPONSES-INSTRUCTIONS-PLACEMENT to use a top-level instructions request field instead.

Stream engine

READ-SSE-DATA decodes one server-sent event’s joined data field from a character stream. Lines and events are bounded by *SSE-MAXIMUM-LINE-CHARACTERS* and *SSE-MAXIMUM-EVENT-CHARACTERS*; violations signal the class named by *STREAM-LIMIT-ERROR-CLASS*, PROVIDER-STREAM-LIMIT-ERROR by default. A drained stream returns *SSE-END-OF-STREAM*. Hosts needing runtime-specific inactivity deadlines install a wrapper as *SSE-READ-LINE-FUNCTION*.

READ-CHARACTER-SEQUENCE fills a string like READ-SEQUENCE using requests no larger than *CHARACTER-READ-SEQUENCE-WINDOW*, staying on the portable buffered path on runtimes whose large decoded reads misbehave.

CALL-WITH-BOUNDED-RETRIES retries an attempt function over the *BOUNDED-RETRY-DELAYS* backoff schedule when it signals PROVIDER-RETRYABLE-ERROR, reports each wait through PROVIDER-RETRY-EVENT callbacks, and restarts immediately on PROVIDER-RESAMPLE-REQUESTED. Hosts bridge their own condition types by inheriting from these classes.

Bounded inference

Budgets

RLM-BUDGET-CREATE creates a subtree budget with shared call and token counters and frame-local recursion depth. RLM-BUDGET-ACQUIRE-REQUEST atomically reserves one call and a bounded output tranche. Settle it with RLM-BUDGET-SETTLE-OUTPUT. Descendants created by RLM-BUDGET-DESCEND share the same counters.

(let* ((budget (llm-provider-api:rlm-budget-create
                :calls 8 :tokens 80000 :depth 2))
       (tranche (llm-provider-api:rlm-budget-acquire-request budget)))
  (unwind-protect
       (call-provider :maximum-output-tokens tranche)
    (llm-provider-api:rlm-budget-settle-output budget tranche usage-total)))

Exhaustion signals RLM-BUDGET-EXHAUSTED with a :CALLS, :TOKENS, or :DEPTH dimension and an optional task label.

Views

RLM-VIEW-MATERIALIZE accepts a string, pathname, RLM-VIEW, or a plist with :CONTENT or :PATH and an optional :LABEL. RLM-VIEWS-MATERIALIZE numbers duplicate labels. RLM-VIEWS-RENDER emits digest-delimited read-only blocks.

Context objects

RLM-CONTEXT-STORE-CREATE opens a directory-backed content-addressed store. RLM-CONTEXT-INTERN and RLM-CONTEXT-INTERN-PATHNAME publish UTF-8 content atomically under its SHA-256 digest. Reads and lookups verify the digest and signal RLM-VIEW-ERROR if stored content is corrupt. Store roots are canonical absolute directories. Object reads reject noncanonical cache paths, and passing an object to another store re-interns its verified content there.

(let* ((store (llm-provider-api:rlm-context-store-create #P"context/"))
       (object (llm-provider-api:rlm-context-intern
                store "large immutable input" :label "corpus")))
  (values (llm-provider-api:rlm-context-object-digest object)
          (llm-provider-api:rlm-context-object-content object)))

Dependencies

• Babel
• Bordeaux Threads
• Ironclad
• ASDF/UIOP

License

COLL-Attribution, copyright 2026 Lambda Symbolics OÜ. See LICENSE.lisp.