rlm.org

Autolith RLM

Recursive inference

A recursive language model treats long context as an external environment instead of a prompt. The model decomposes a task, runs bounded inference over selected snippets, and composes the returned values. Autolith realizes this as a middle layer between a tool call and a child agent, so recursion becomes an ordinary Lisp operation:

(infer "List the invariants this file maintains."
       :context (list #p"src/conversation/store.lisp")
       :contract '(:type :object
                   :properties
                   (("invariants" (:type :array :items (:type :string))))
                   :required ("invariants"))
       :budget (rlm-budget-create :calls 16 :tokens 80000 :depth 3))

Autolith already has the outer level. Task children are persistent agents with conversations, tool registries, and orchestrator scheduling. RLM adds the two inner levels plus the paper-defining root operation, rlm-complete, described below:

• An inference call. One bounded inference operation over explicit context. Returns a Lisp value. Never touches the caller’s conversation.
• An inference frame. One task with a private ephemeral conversation, a contract, and a budget subtree. A frame may recurse into child frames until the budget runs out. Only the frame’s value and a trace reference escape.

A frame never inherits the parent conversation, the agent identity, or unrestricted capabilities. It receives the task, the materialized context views, the contract, and the remaining budget. This containment is what makes recursion cheap and safe: a hundred frames cannot pollute the primary conversation or the prompt cache.

Context views

Context is passed explicitly, and the designator set is split by authority. Programmatic Lisp callers in the active image may pass:

• a string, passed verbatim,
• a pathname, read at call time,
• a plist (:label ... :content ...) for labeled literals.

The model-visible tools never accept raw filesystem paths. Their views carry literal text, a resource URI resolved through the resource protocol under the caller’s authority (so restricted frames stay confined to their readable scheme set), or a stored context object reference as context:<sha256>. Each view materializes into a labeled block with a content digest, so a trace records exactly what the frame saw. Views are read-only and fixed for the frame’s lifetime.

Contracts

A contract names the shape of the value a frame must return. Contracts reuse the task output schema language documented for task agent roles; task-output-schema-normalize validates the definition and task-output-schema-valid-p validates the value. Two contract forms exist:

• :text, the default. The frame returns its answer as a string.
• a schema, for example (:type :object :properties (("claims" (:type :array :items (:type :string)))) :required ("claims")). The frame is instructed to answer with exactly one JSON value of any type, object and array included, and the parsed, validated value is returned as portable tagged native data.

A malformed answer is repaired by re-asking within the remaining call budget. When the budget is gone the failure signals a condition, so callers can retry with more context or a different model; partial completed results are a property of rlm-map, which returns finished frames alongside failed ones.

Budgets

Every inference runs under an rlm-budget: remaining calls, tokens, and recursion depth, with thread-safe accounting. Recursive frames share the parent’s counters and decrement depth. Every provider request atomically reserves, in one critical section, one call and an output token tranche; the tranche doubles as the request’s provider output ceiling where the wire protocol accepts one (elsewhere it stays an accounting reservation) and is settled against the reported usage afterwards, with failed or usage-less requests refunding the output tranche in full while the admitted call stays consumed. A tranche takes at most a share of the remaining pool (a quarter by default, capped at 16000 tokens) and never more than the pool holds, so concurrent siblings keep headroom and combined reservations cannot oversubscribe the allocation; only input tokens stay post hoc, gating the next reservation rather than the current one. Exhaustion signals rlm-budget-exhausted instead of silently degrading.

Root completions

rlm-complete is the paper-faithful realization: the complete input remains outside the root model context, and only selected bounded slices are supplied to sub-inference contexts. The context designator is interned as an immutable content-addressed object under data/inferences/objects/ and the root model receives only its label, size, and digest. A dedicated heap-isolated Lisp environment holds the content, every intermediate value, and the recursion:

(rlm-complete "Summarize every incident in this log."
              :context #p"/var/log/huge.log"
              :budget (rlm-budget-create :calls 24 :tokens 240000
                                         :depth 2))

The root model drives the environment through the env.eval tool, one form per call, observing only bounded printed values and output. Inside the environment, (context-length), (context-slice start end), and (context-search pattern) inspect the external object; (infer ...) and (rlm-map ...) proxy sub-inferences back to the host over an authenticated loopback endpoint, where budgets, contracts, and traces are enforced host-side because environment code is model-authored and untrusted; (finish value) records the final answer. Every proxied call descends the run’s shared budget subtree, so model-authored loops can never outspend the root allocation. Finish is terminal: the first finish wins, later operations are refused, and the value must be portable readable data because it crosses the environment boundary as printed text. The run returns the recorded Lisp value plus the root trace identifier. The model reaches the same operation through the rlm.complete tool.

Every proxied operation also appends one replay-safe :rlm-call metadata record to the root trace, carrying the operation, the child trace identifiers, and the remaining budget. The root trace therefore holds the run’s machine-readable invocation tree even when environment code discards the trace identifiers it receives, which is what trace-driven policy distillation reads.

The defining litmus property is tested end to end: an input larger than the provider window is processed almost completely, with the decomposition authored by the root model as Lisp, no provider request exceeding the window, intermediate values staying in the environment, and the exact answer returned.

Parallel maps

rlm-map fans a list of tasks out as frames sharing one budget subtree, each task a string or a (:task ... :context ...) plist. Results keep task order, and a failed frame reports its error in place instead of discarding the finished ones, so exhausting the shared budget degrades to partial results. The model reaches the same fan-out through the rlm.map tool.

Policies

Decomposition and synthesis are generic functions dispatching on a policy keyword, so live self modification can promote strategies distilled from successful traces as new eql-specialized methods:

• rlm-decompose-inference-task returns subtask plists for a task, or NIL to run it directly.
• rlm-synthesize-inference-results composes the final answer from the subtask results; the default method for every policy runs one more frame whose views are the results, failures included.

rlm-run drives one task through a policy: decompose, fan the subtasks out through rlm-map, and synthesize under the caller’s contract, all inside one budget subtree. The default :direct policy never decomposes.

Traces

Frames persist their private conversations under data/inferences/ in the same append-only readable format as regular conversations, one file per frame. The caller’s conversation only ever sees the returned value. The trace answers “why did this inference conclude that” after the fact: task, views with digests, model parameters, raw exchanges, and usage are all in the file.

Traces stay addressable after the fact: inference:<trace-id> URIs read a bounded trace log back through resource.read, and the compaction instructions tell summaries to reference frames by trace identifier instead of restating their content, so compaction keeps evidence reachable instead of inlining it.

Recursion surfaces

The lisp.* tools run in heap-isolated workers with no provider access, so model-authored worker code cannot call infer. The surfaces are:

• The infer, rlm-map, rlm-run, and rlm-complete functions in the active image, for user REPL forms and for self-modification code (policies, agenda automation).
• The rlm.infer, rlm.map, and rlm.complete tools for the model, taking tasks, views or one external context, an optional JSON Schema contract for the frame tools, and call, token, and depth allowances.

Long-running tool calls publish one compact live activity hint. rlm.infer shows provider request count and remaining calls, rlm.map adds the active frame index and fan width, and rlm.complete reports environment startup before root request progress. These hints are transient status rows, not conversation records.

Both surfaces accept :capabilities (the tools’ capabilities argument). A frame without capabilities is a pure call over its views. A read frame runs as a restricted agent turn: it may use resource.read (workspace scheme only), the search tools, and nested rlm.infer and rlm.map sharing the same budget subtree, with tool rounds bounded per round. Inside a frame the nested tools ignore allowance arguments and descend the enclosing budget, so fan-out can never outspend the root allocation.

Design decisions

Settled trade-offs, recorded so review rounds do not relitigate them.

The environment is privileged and unsandboxed

env.eval runs model-authored Lisp in a heap-isolated worker with the user’s privileges, exactly like lisp.* always has; Autolith’s stance is that sandboxing is no substitute for oversight. Opaque context handles with host-proxied context-slice and context-search were considered and rejected: they add a socket round trip per operation while confining nothing, because arbitrary worker Lisp can still open any file the user can. Confinement is why frames may not launch environments, not a property handles would provide. The store is protected differently: objects are read-only on disk, host-side lookup re-hashes their content and rejects corruption, and re-interning the original content repairs a corrupted object. Revisit only alongside real OS sandboxing of the worker.

Finish and shutdown do not block

Admission shares one lock with the terminal state, so nothing starts after a finish commits, but a finish never waits for admitted operations: they only overlap when environment code spawned worker threads, they stay budget-bounded, and a blocking finish could deadlock the run. Endpoint shutdown likewise joins handlers briefly and then abandons stragglers; an abandoned handler is a budget-bounded provider call whose response write fails harmlessly, which is cheaper than any safe cancellation.

Context objects live in the heap

Interning and the environment’s context functions materialize whole objects in memory. This is a documented capacity bound, not a design error: the RLM property only requires the content to stay outside model contexts, and content addressing already provides stable identity. Streaming, mmap, or indexes arrive when a workload actually exceeds the heap, not before.

Token accounting is exact for output, soft for input

Reservations bound what concurrent requests may produce, but exact input usage is only known after a response, so input tokens settle post hoc and gate the next reservation. Providers that report no usage consume reservations but settle as free; deterministic test providers depend on this, and real providers report usage.

Implementation

• Budgets: src/inference/budget.lisp.
• Context views and materialization: src/inference/view.lisp.
• The inference frame and infer: src/inference/frame.lisp, with the compact frame system prompts and trace persistence.
• The rlm.infer and rlm.map tools, frame registries, and the contract conversion from JSON Schema: src/inference/tools.lisp.
• Parallel maps: src/inference/map.lisp.
• Policies and rlm-run: src/inference/policy.lisp.
• The inference: trace resource: src/inference/resource.lisp, with the trace reference guidance in the compaction instructions.
• Content-addressed context objects: src/inference/object.lisp.
• The loopback host endpoint for environment calls: src/inference/endpoint.lisp.
• Root completions, the environment prelude, and env.eval: src/inference/environment.lisp.
• Model-decided auto-mode command permissions: src/inference/permission.lisp.

Per-call model routing

Pass optional model and effort arguments to rlm.infer, rlm.map, or rlm.complete to select a model and supported reasoning effort for the entire subtree. For a map, this selection applies to every frame; for a root completion, it applies to the environment and child inferences. Make this selection in the outermost call. Nested calls use the enclosing selection without overrides.