Skip to content

feat(agent-core-v2): advertise model capabilities in the subagent model picker #2591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
7Sageer merged 3 commits into from
Aug 4, 2026
Merged

feat(agent-core-v2): advertise model capabilities in the subagent model picker #2591

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d621fdd
feat(agent-core-v2): advertise model capabilities in the subagent mod…
7Sageer Aug 4, 2026
6b9079b
chore(agent-core-v2): follow header-only comment convention
7Sageer Aug 4, 2026
b3c3890
docs(agent-core-v2): note the model catalog collaborator in the Agent…
7Sageer Aug 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions packages/agent-core-v2/src/agent/tools/agent-swarm/agentSwarmTool.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,11 @@
* `ISessionAgentProfileCatalog`, and the caller's `IAgentProfileService`) and
* threads it through the swarm tasks; otherwise binding is left to the
* service, which keeps its own "no model bound" check and inherit-caller
* fallback. Swarm mode is entered through `IAgentSwarmService`; the caller's
* agent id comes from `IAgentScopeContext`. Pure tool — owns no scoped state.
* fallback. The advertised `model` parameter lists the secondary/primary
* pair via `buildSubagentModelDescriptions`, suffixing each line with the
* entry's capability flags resolved through `IModelCatalog`. Swarm mode is
* entered through `IAgentSwarmService`; the caller's agent id comes from
* `IAgentScopeContext`. Pure tool — owns no scoped state.
*
* Registered via the module-level `registerAgentToolService(IAgentSwarmTool,
* AgentSwarmTool)` at the bottom of this file — the same "import = register"
Expand All @@ -31,6 +34,7 @@ import { registerAgentToolService } from '#/agent/toolRegistry/toolContribution'
import { toInputJsonSchema } from '#/tool/input-schema';
import { IConfigService } from '#/app/config/config';
import { IFlagService } from '#/app/flag/flag';
import { IModelCatalog } from '#/kosong/model/catalog';
import { ISessionSwarmService, type SessionSwarmTask } from '#/session/swarm/sessionSwarm';
import { ISessionAgentProfileCatalog } from '#/session/sessionAgentProfileCatalog/sessionAgentProfileCatalog';
import { IAgentProfileService } from '#/agent/profile/profile';
Expand Down Expand Up @@ -112,6 +116,7 @@ export class AgentSwarmTool implements IAgentSwarmTool {
@IFlagService private readonly flags: IFlagService,
@ISessionAgentProfileCatalog private readonly catalog: ISessionAgentProfileCatalog,
@IAgentProfileService private readonly profile: IAgentProfileService,
@IModelCatalog private readonly modelCatalog: IModelCatalog,
) {
this.callerAgentId = scopeContext.agentId;
}
Expand All @@ -121,6 +126,7 @@ export class AgentSwarmTool implements IAgentSwarmTool {
this.config,
this.flags,
this.profile.data().modelAlias,
this.modelCatalog,
);
return modelLines === undefined
? AGENT_SWARM_DESCRIPTION
Expand Down
1 change: 1 addition & 0 deletions packages/agent-core-v2/src/agent/tools/agent/agentTool.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ export class SubagentTool implements ISubagentTool {
this.config,
this.flags,
this.profile.data().modelAlias,
this.modelCatalog,
);
if (modelLines !== undefined) {
description += `\n\n${modelLines}`;
Expand Down
43 changes: 39 additions & 4 deletions packages/agent-core-v2/src/session/subagent/configSection.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,10 @@
* naturally (global thinking config → the bound model's default effort)
* rather than inheriting the caller's level. Both tools resolve spawn
* bindings through `resolveSubagentBinding`, advertise the pair via
* `buildSubagentModelDescriptions`, and wrap spawn failures with
* `buildSubagentModelDescriptions` (each line suffixed with the entry's
* resolved capability flags, so the parent can route multimodal or
* thinking-heavy subagent tasks instead of guessing from the model id),
* and wrap spawn failures with
* `wrapSubagentModelError`; while the experiment is off they also strip the
* no-op `model` parameter from their advertised schemas via
* `stripSubagentModelParameter`. Self-registered at module load via
Expand Down Expand Up @@ -52,6 +55,8 @@ import {
type IConfigService,
} from '#/app/config/config';
import { registerConfigSection } from '#/app/config/configSectionContributions';
import type { ModelCapability } from '#/kosong/contract/capability';
import type { IModelCatalog } from '#/kosong/model/catalog';

import { SECONDARY_MODEL_FLAG_ID } from './flag';

Expand Down Expand Up @@ -127,16 +132,46 @@ export function buildSubagentModelDescriptions(
config: IConfigService,
flags: IFlagService,
callerModelAlias: string | undefined,
modelCatalog: IModelCatalog,
): string | undefined {
const secondaryModel = resolveSecondaryModel(config, flags)?.model;
const secondary = resolveSecondaryModel(config, flags);
const secondaryModel = secondary?.model;
if (secondaryModel === undefined || callerModelAlias === undefined) return undefined;
const boundSecondary =
secondaryModelPatch(secondary) === undefined ? secondaryModel : SECONDARY_DERIVED_MODEL_ID;
return [
'Available models (pass via model):',
`- secondary: ${secondaryModel} (default) — the configured secondary model; prefer it for routine subagent tasks`,
`- primary: ${callerModelAlias} — the main model you are running on; use it for hard, quality-sensitive subagent tasks`,
`- secondary: ${secondaryModel} (default) — the configured secondary model; prefer it for routine subagent tasks${capabilitiesSuffix(resolvedCapabilities(modelCatalog, boundSecondary))}`,
`- primary: ${callerModelAlias} — the main model you are running on; use it for hard, quality-sensitive subagent tasks${capabilitiesSuffix(resolvedCapabilities(modelCatalog, callerModelAlias))}`,
].join('\n');
}

const ADVERTISED_CAPABILITY_FLAGS = [
'image_in',
'video_in',
'audio_in',
'thinking',
'tool_use',
'dynamically_loaded_tools',
] as const satisfies readonly (keyof ModelCapability)[];

function capabilitiesSuffix(capability: ModelCapability | undefined): string {
if (capability === undefined) return '';
const names = ADVERTISED_CAPABILITY_FLAGS.filter((flag) => capability[flag] === true);
return `; capabilities: ${names.length === 0 ? 'none' : names.join(', ')}`;
}

function resolvedCapabilities(
modelCatalog: IModelCatalog,
model: string,
): ModelCapability | undefined {
try {
return modelCatalog.get(model).capabilities;
} catch {
return undefined;
}
}

/**
* Strip the `model` property from a subagent collaboration tool's advertised
* JSON schema. While the `secondary-model` experiment is off the parameter is
Expand Down
77 changes: 61 additions & 16 deletions packages/agent-core-v2/test/agent/swarm/swarm.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import type {
ResolvedToolExecutionHookContext,
} from '#/agent/toolExecutor/toolHooks';
import type { ToolCall } from '#/kosong/contract/message';
import type { ModelCapability } from '#/kosong/contract/capability';
import { IModelCatalog } from '#/kosong/model/catalog';
import type { ExecutableToolContext } from '#/tool/toolContract';
import { IAgentToolRegistryService } from '#/agent/toolRegistry/toolRegistry';
import { AgentToolRegistryService } from '#/agent/toolRegistry/toolRegistryService';
Expand Down Expand Up @@ -146,6 +148,19 @@ function stubCallerProfile(
} as unknown as IAgentProfileService;
}

function stubModelCatalog(
capabilities: Readonly<Record<string, ModelCapability>> = {},
): IModelCatalog {
return {
_serviceBrand: undefined,
get: (id: string) => {
const capability = capabilities[id];
if (capability === undefined) throw new Error(`Model "${id}" is not configured.`);
return { capabilities: capability };
},
} as unknown as IModelCatalog;
}

describe('AgentSwarmService', () => {
let disposables: DisposableStore;
let ix: TestInstantiationService;
Expand Down Expand Up @@ -342,7 +357,7 @@ describe('AgentSwarmTool', () => {
]),
});
const swarmMode = mockSwarmMode();
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), swarmMode, stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile());
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), swarmMode, stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile(), stubModelCatalog());
const input = {
description: 'Review files',
prompt_template: 'Review {{item}}',
Expand Down Expand Up @@ -439,7 +454,7 @@ describe('AgentSwarmTool', () => {

it('does not expose permission rule argument matching', () => {
const host = mockSwarmHost();
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile());
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile(), stubModelCatalog());
const execution = tool.resolveExecution({
description: 'Review files',
prompt_template: 'Review {{item}}',
Expand All @@ -454,7 +469,7 @@ describe('AgentSwarmTool', () => {

it('description states the enforced input requirements', () => {
const host = mockSwarmHost();
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile());
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile(), stubModelCatalog());
expect(tool.description).toContain('at least 2');
expect(tool.description).toContain('{{item}}');
expect(tool.description.toLowerCase()).toContain('distinct');
Expand All @@ -476,6 +491,7 @@ describe('AgentSwarmTool', () => {
stubFlag(true),
stubSwarmCatalog(caller),
stubCallerProfile({ profileName: 'deleted-profile', subagents: ['explore'] }),
stubModelCatalog(),
);

const result = await executeTool(
Expand Down Expand Up @@ -539,7 +555,7 @@ describe('AgentSwarmTool', () => {

for (const testCase of cases) {
const host = mockSwarmHost();
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile());
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile(), stubModelCatalog());

const result = await executeTool(tool, context(testCase.input));

Expand Down Expand Up @@ -572,7 +588,7 @@ describe('AgentSwarmTool', () => {
async ({ agentId }: { readonly agentId: string }) => persistedItems[agentId],
);
const host = mockSwarmHost({ run, getSwarmItem });
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile());
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile(), stubModelCatalog());
const input = {
description: 'Finish review',
subagent_type: 'explore',
Expand Down Expand Up @@ -692,7 +708,7 @@ describe('AgentSwarmTool', () => {
);
const getSwarmItem = vi.fn(async () => 'src/old-a.ts');
const host = mockSwarmHost({ run, getSwarmItem });
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile());
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile(), stubModelCatalog());
const input = {
description: 'Resume review',
resume_agent_ids: {
Expand Down Expand Up @@ -755,7 +771,7 @@ describe('AgentSwarmTool', () => {
},
]),
});
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile());
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile(), stubModelCatalog());

const result = await executeTool(
tool,
Expand All @@ -781,7 +797,7 @@ describe('AgentSwarmTool', () => {

it('passes the configured subagent timeout to swarm tasks', async () => {
const host = mockSwarmHost();
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig({ timeoutMs: 5_000 }), stubFlag(true), stubSwarmCatalog(), stubCallerProfile());
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig({ timeoutMs: 5_000 }), stubFlag(true), stubSwarmCatalog(), stubCallerProfile(), stubModelCatalog());

await executeTool(
tool,
Expand All @@ -804,7 +820,7 @@ describe('AgentSwarmTool', () => {

it('resolves spawn task bindings from the configured secondary model', async () => {
const host = mockSwarmHost();
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig({ model: 'provider/secondary', defaultEffort: 'low' }), stubFlag(true), stubSwarmCatalog(), stubCallerProfile({ modelAlias: 'main-model', thinkingLevel: 'high' }));
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig({ model: 'provider/secondary', defaultEffort: 'low' }), stubFlag(true), stubSwarmCatalog(), stubCallerProfile({ modelAlias: 'main-model', thinkingLevel: 'high' }), stubModelCatalog());

await executeTool(
tool,
Expand Down Expand Up @@ -833,7 +849,7 @@ describe('AgentSwarmTool', () => {
modelPreference: 'secondary',
systemPrompt: () => 'coder',
};
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig({ model: 'provider/secondary', defaultEffort: 'low' }), stubFlag(true), stubSwarmCatalog(DEFAULT_CALLER_PROFILE, [secondaryCoder]), stubCallerProfile({ modelAlias: 'main-model', thinkingLevel: 'high' }));
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig({ model: 'provider/secondary', defaultEffort: 'low' }), stubFlag(true), stubSwarmCatalog(DEFAULT_CALLER_PROFILE, [secondaryCoder]), stubCallerProfile({ modelAlias: 'main-model', thinkingLevel: 'high' }), stubModelCatalog());

await executeTool(
tool,
Expand All @@ -857,17 +873,46 @@ describe('AgentSwarmTool', () => {

it('advertises both selectable models in the description only when configured', async () => {
const host = mockSwarmHost();
const configured = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig({ model: 'provider/secondary' }), stubFlag(true), stubSwarmCatalog(), stubCallerProfile({ modelAlias: 'main-model' }));
const configured = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig({ model: 'provider/secondary' }), stubFlag(true), stubSwarmCatalog(), stubCallerProfile({ modelAlias: 'main-model' }), stubModelCatalog({
'provider/secondary': { image_in: true, video_in: false, audio_in: false, thinking: true, tool_use: true, max_context_tokens: 262_144 },
'main-model': { image_in: false, video_in: false, audio_in: false, thinking: false, tool_use: true, max_context_tokens: 262_144 },
}));

expect(configured.description).toContain('Available models (pass via model):');
expect(configured.description).toContain('- secondary: provider/secondary (default)');
expect(configured.description).toContain('- primary: main-model');
expect(configured.description).toContain(
'- secondary: provider/secondary (default) — the configured secondary model; prefer it for routine subagent tasks; capabilities: image_in, thinking, tool_use',
);
expect(configured.description).toContain(
'- primary: main-model — the main model you are running on; use it for hard, quality-sensitive subagent tasks; capabilities: tool_use',
);

const unconfigured = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile({ modelAlias: 'main-model' }));
const unconfigured = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile({ modelAlias: 'main-model' }), stubModelCatalog());

expect(unconfigured.description).not.toContain('Available models');
});

it('reads secondary capabilities from the derived entry when the recipe carries patch fields', async () => {
const host = mockSwarmHost();
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig({ model: 'provider/secondary', defaultEffort: 'low' }), stubFlag(true), stubSwarmCatalog(), stubCallerProfile({ modelAlias: 'main-model' }), stubModelCatalog({
[SECONDARY_DERIVED_MODEL_ID]: { image_in: false, video_in: false, audio_in: false, thinking: true, tool_use: true, max_context_tokens: 131_072 },
'main-model': { image_in: true, video_in: false, audio_in: false, thinking: false, tool_use: true, max_context_tokens: 262_144 },
}));

expect(tool.description).toContain(
'- secondary: provider/secondary (default) — the configured secondary model; prefer it for routine subagent tasks; capabilities: thinking, tool_use',
);
expect(tool.description).toContain('capabilities: image_in, tool_use');
});

it('omits the capabilities suffix for models the catalog cannot resolve', async () => {
const host = mockSwarmHost();
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig({ model: 'provider/secondary' }), stubFlag(true), stubSwarmCatalog(), stubCallerProfile({ modelAlias: 'main-model' }), stubModelCatalog());

expect(tool.description).toContain('- secondary: provider/secondary (default)');
expect(tool.description).toContain('- primary: main-model');
expect(tool.description).not.toContain('capabilities:');
});

it('omits resume hint when incomplete subagents have no agent ids', async () => {
const host = mockSwarmHost({
run: vi.fn().mockImplementation(async ({ tasks }) => [
Expand All @@ -883,7 +928,7 @@ describe('AgentSwarmTool', () => {
},
]),
});
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile());
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile(), stubModelCatalog());

const result = await executeTool(
tool,
Expand Down Expand Up @@ -930,7 +975,7 @@ describe('AgentSwarmTool', () => {
},
]),
});
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile());
const tool = new AgentSwarmTool(host.swarmService, makeAgentScopeContext({ agentId: host.callerAgentId, agentScope: '' }), mockSwarmMode(), stubConfig(), stubFlag(true), stubSwarmCatalog(), stubCallerProfile(), stubModelCatalog());

const result = await executeTool(
tool,
Expand Down
Loading
Loading