Skip to content

Per-role decoding config is loaded but never applied (dead config) #189

Description

@real-venus

Summary

Every model-pool config ships a per-role decoding block, e.g. configs/models.openrouter.light.yaml:

decoding:
  thinker:   { temperature: 0.7, top_p: 0.95, max_tokens: 4096 }
  worker:    { temperature: 0.2, top_p: 0.95, max_tokens: 4096 }
  verifier:  { temperature: 0.0, top_p: 1.0,  max_tokens: 2048 }

OpenAICompatiblePool.__init__ loads it (self.decoding = cfg.get("decoding", {}), openai_compatible_pool.py:135) — but nothing ever reads self.decoding. A grep over src/ finds the attribute is write-only:

$ git grep -n "\.decoding" src/
src/trinity/llm/openai_compatible_pool.py:135:  self.decoding: dict = cfg.get("decoding", {})

The actual decoding params come from hardcoded defaults in orchestration/session.py::run_trajectory (temperature=0.0, top_p=1.0, max_tokens=4096), passed identically for all three roles.

Impact

  • The documented per-role settings (worker 0.2, thinker 0.7, verifier 2048 max_tokens, …) are silently ignored — every role runs at the session default (temp 0.0 / top_p 1.0 / max_tokens 4096).
  • A miner who tunes the decoding: block to shape routing behavior sees no effect, with no error or warning.
  • The config surface is misleading: it looks tunable but is inert.

Proposed fix

Wire per-role decoding through, backward-compatibly:

  • Expose a resolver on the pool (e.g. decoding_for(role)) that returns the config block for that role, falling back to the current defaults for any missing role/field.
  • Have run_trajectory select temperature/top_p/max_tokens per turn from that resolver instead of fixed args (keeping the explicit-arg override for callers/tests).
  • When a config has no decoding: block, behavior is unchanged (defaults preserved).

(Alternative, if the block is intentionally unused: remove the dead self.decoding load and the decoding: blocks from the configs so the surface isn't misleading. Either resolves the inconsistency.)

Acceptance

  • Per-role decoding values from the config are applied to the corresponding turns
  • Absent/partial decoding blocks fall back to the current defaults (no behavior change)
  • Unit test asserting the resolved (temperature, top_p, max_tokens) per role from a sample config, plus the fallback path

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions