Skip to content

Model pins don't propagate: fleet members and agent profiles keep a retired id after the provider moves on #6035

Description

@Hmbown

Problem

A model id is pinned independently in at least six places, with no shared owner and no migration when a vendor retires an id. On one real machine, after DeepSeek released V4.1 Flash as deepseek-flash and dropped deepseek-v4-flash from the account roster:

~/.codewhale/config.toml:2      default_text_model = "deepseek-v4-flash-vision-exp"
~/.codewhale/config.toml:15     [providers.deepseek]      model = "deepseek-flash"
~/.codewhale/settings.toml:38   default_model             = "deepseek-v4-flash-vision-exp"
~/.codewhale/settings.toml:51   [provider_models] deepseek = "deepseek-v4-flash-vision-exp"
~/.codewhale/agents/scout.toml:4   model = "deepseek-v4-flash"   provider = "deepseek"
~/.codewhale/agents/custom.toml:4  model = "deepseek-v4-flash"   provider = "deepseek"
~/.codewhale/fleets/default.toml:19 model = "deepseek-v4-flash"  provider = "deepseek"

14 of 19 DeepSeek pins named ids the account roster no longer lists. Changing the provider's model in one place moved the main session to deepseek-flash while every subagent kept running deepseek-v4-flash — visibly, in the Agents panel, three workers on a retired id while the parent was on the new one.

The nuance that makes this not a simple "purge old ids"

deepseek-v4-flash is still served — the subagents worked, producing 277k/142k/270k tokens. It is absent from /v1/models but the endpoint still answers. So this is soft-deprecation, not breakage.

It also legitimately still exists in Codewhale's catalog, because third-party hosts (Fireworks, OpenCode Zen, SiliconFlow, Together, Baseten…) serve their own copies and are unaffected by a first-party roster change. A fleet pinning deepseek-v4-flash on one of those providers is entirely correct.

So the rule cannot be "retire the id globally". It has to be per route: this id is no longer offered on this provider.

What is missing

  1. No single owner for "which model does this role use". Provider default, global default, per-provider setting, agent profile and fleet member each hold their own copy.
  2. No propagation. Changing a provider's model does not reach agent profiles or fleet members that name the same provider.
  3. No deprecation signal. Nothing notices that a pinned id vanished from the roster of the provider it is pinned to.
  4. No visible drift. The only way this surfaced was a human reading model names in the Agents panel.

Proposed direction

  • When a pinned (provider, model) no longer appears in that provider's live roster, say so — in /status, in the fleet view, and where the pin is edited. Not a silent rewrite: AGENTS.md and C04 both forbid silent route rewrites, and a fleet member's model is a deliberate choice.
  • Offer a one-action migration ("3 fleet members and 2 agent profiles pin deepseek-v4-flash, which deepseek no longer lists — move them to deepseek-flash?"), scoped to the affected provider only.
  • Consider making a fleet member's model inherit from the provider by default, with an explicit pin as the opt-out. Most members do not want an independent choice; they want "whatever this provider is set to".
  • A vendor-published successor hint (deepseek-v4-flashdeepseek-flash) belongs in the catalog or the cloud-facts channel, not in a Rust table — see Model capability is decided by hardcoded prefixes and a literal list; it belongs to the catalog #6032.

Founder's framing

"it would be through fleet and whatnot i think? so we have to have the fleet models auto-update with it … but if deepseek-v4 still exists IN codewhale somewhere it doesn't NOT make sense … but yeah the fleet stuff is a mess! it's a lot!"

Both halves are right: propagation is missing, and the old id is still legitimately present for other routes. The fix has to respect both.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    • Status
      Backlog

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions