Skip to content

Commit e2cdd3c

Browse files
feat(ollama): add native thinking/reasoning support (#832)
* feat(ollama): add native thinking/reasoning support - Add Enable Thinking checkbox + ThinkingBudget selector to Ollama settings - Map reasoning effort to Ollama native think param (low/medium/high) - Stream message.thinking as reasoning chunks - Round-trip prior reasoning/thinking blocks for multi-turn thinking - Add unit tests for native-ollama and Ollama settings component - Add i18n strings for thinking toggle Closes #831 * i18n: add Ollama thinking translations for all locales Adds thinking and thinkingHelp strings for the new Ollama native thinking toggle across all 17 non-English webview-ui locales. Part of #831 * fix(ollama): resolve check-types error, gate think param on opt-in, address CodeRabbit comments - Fix TS2367 in native-ollama.ts: remove unreachable 'image' branch from assistant message conversion (Anthropic ContentBlock union has no 'image'; assistant cannot send images). User-message image handling is preserved. - Gate getOllamaThinkParam on enableReasoningEffort === true so a stale reasoningEffort inherited from another provider config cannot silently emit a think param when the Ollama UI checkbox is unchecked. - Extract buildChatRequestOptions() helper shared by createMessage and completePrompt to eliminate duplicated request-option assembly. - Ollama settings UI: preserve reasoningEffort across checkbox toggles instead of wiping it to undefined when unchecked; restore the prior value on re-enable rather than forcing 'medium'. - Add tests for the stale-config case, the disable+opt-in case, and the completePrompt non-Error throw branch. Coverage now 83.89% (>= 80%). * docs(ollama): add explanatory comment for removed Anthropic image branch in assistant messages * test(ollama): improve patch coverage for native thinking support Add tests covering previously uncovered branches in native-ollama.ts: - Anthropic-protocol thinking block round-tripping (block.type === thinking) - Concatenation of multiple reasoning/thinking blocks with newline joins - Empty reasoning/thinking blocks (length > 0 false branch + reasoningText || undefined) - Plain assistant message without reasoning blocks (falsy reasoningText branch) - Unknown reasoningEffort value (default switch branch returns undefined) - Stream processing error wrapping (catch streamError branch + Unknown error fallback) - Non-ECONNREFUSED non-404 error rethrow (fall-through throw error branch) Also strengthen the Ollama.spec.tsx toggle-off assertion per CodeRabbit nitpick: assert no call with reasoningEffort as the first argument at all, instead of only ruling out undefined. Patch coverage for native-ollama.ts rises from 71.05% to ~100% of new lines, clearing the codecov/patch 80% threshold.
1 parent ff3730a commit e2cdd3c

22 files changed

Lines changed: 1017 additions & 36 deletions

File tree

src/api/providers/__tests__/native-ollama.spec.ts

Lines changed: 541 additions & 0 deletions
Large diffs are not rendered by default.

src/api/providers/native-ollama.ts

Lines changed: 171 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,15 @@ interface OllamaChatOptions {
1414
num_ctx?: number
1515
}
1616

17+
// Narrow local types for non-Anthropic content blocks that may be carried in
18+
// the conversation history. The Anthropic SDK union does not include the
19+
// custom `reasoning` block (used by non-Anthropic protocols) or the
20+
// Anthropic-protocol `thinking` block, so we declare them here to keep type
21+
// checking intact for the rest of the union instead of falling back to `any`.
22+
type ReasoningContentBlock = { type: "reasoning"; text: string }
23+
type ThinkingContentBlock = { type: "thinking"; thinking: string }
24+
type AssistantContentBlock = Anthropic.ContentBlock | ReasoningContentBlock | ThinkingContentBlock
25+
1726
function convertToOllamaMessages(anthropicMessages: Anthropic.Messages.MessageParam[]): Message[] {
1827
const ollamaMessages: Message[] = []
1928

@@ -97,32 +106,64 @@ function convertToOllamaMessages(anthropicMessages: Anthropic.Messages.MessagePa
97106
})
98107
}
99108
} else if (anthropicMessage.role === "assistant") {
100-
const { nonToolMessages, toolMessages } = anthropicMessage.content.reduce<{
101-
nonToolMessages: (Anthropic.TextBlockParam | Anthropic.ImageBlockParam)[]
109+
// Assistant message conversion: only `text`, `tool_use`, and the
110+
// custom `reasoning`/`thinking` blocks are relevant here.
111+
//
112+
// Note on the removed `image` branch: the previous code checked
113+
// `block.type === "image"` and typed `nonToolMessages` as
114+
// `(Anthropic.TextBlockParam | Anthropic.ImageBlockParam)[]`. This
115+
// was removed because:
116+
// 1. The Anthropic API only accepts image blocks in *user*
117+
// messages — assistants cannot produce or send images, so
118+
// the branch was dead code (the original comment even said
119+
// "impossible as the assistant cannot send images").
120+
// 2. `Anthropic.ContentBlock` (the response/output union) does
121+
// not include an `image` variant, so the comparison
122+
// `block.type === "image"` triggered TS2367 ("this comparison
123+
// appears to be unintentional because the types have no
124+
// overlap").
125+
// 3. Image handling for *user* messages (where images are
126+
// actually sent to the model) is preserved unchanged in the
127+
// `anthropicMessage.role === "user"` branch above.
128+
const { nonToolMessages, toolMessages, reasoningText } = anthropicMessage.content.reduce<{
129+
nonToolMessages: Anthropic.TextBlockParam[]
102130
toolMessages: Anthropic.ToolUseBlockParam[]
131+
reasoningText: string
103132
}>(
104133
(acc, part) => {
105-
if (part.type === "tool_use") {
106-
acc.toolMessages.push(part)
107-
} else if (part.type === "text" || part.type === "image") {
108-
acc.nonToolMessages.push(part)
134+
// `part` is typed as an Anthropic content block, but the
135+
// conversation history may also carry custom `reasoning`
136+
// blocks (used by non-Anthropic protocols) or Anthropic
137+
// `thinking` blocks that are not part of the SDK union.
138+
// Cast to the augmented union to access them while
139+
// preserving type safety for the rest of the block.
140+
const block = part as AssistantContentBlock
141+
if (block.type === "tool_use") {
142+
acc.toolMessages.push(block)
143+
} else if (block.type === "text") {
144+
acc.nonToolMessages.push(block)
145+
} else if (block.type === "reasoning") {
146+
// Non-Anthropic protocols store reasoning as a block
147+
// with a `text` field. Pass it back so Ollama can
148+
// preserve thinking context across turns.
149+
if (block.text.length > 0) {
150+
acc.reasoningText += (acc.reasoningText ? "\n" : "") + block.text
151+
}
152+
} else if (block.type === "thinking") {
153+
// Anthropic-protocol thinking blocks carry `thinking`.
154+
if (block.thinking.length > 0) {
155+
acc.reasoningText += (acc.reasoningText ? "\n" : "") + block.thinking
156+
}
109157
} // assistant cannot send tool_result messages
110158
return acc
111159
},
112-
{ nonToolMessages: [], toolMessages: [] },
160+
{ nonToolMessages: [], toolMessages: [], reasoningText: "" },
113161
)
114162

115163
// Process non-tool messages
116164
let content: string = ""
117165
if (nonToolMessages.length > 0) {
118-
content = nonToolMessages
119-
.map((part) => {
120-
if (part.type === "image") {
121-
return "" // impossible as the assistant cannot send images
122-
}
123-
return part.text
124-
})
125-
.join("\n")
166+
content = nonToolMessages.map((part) => part.text).join("\n")
126167
}
127168

128169
// Convert tool_use blocks to Ollama tool_calls format
@@ -140,6 +181,8 @@ function convertToOllamaMessages(anthropicMessages: Anthropic.Messages.MessagePa
140181
role: "assistant",
141182
content,
142183
tool_calls: toolCalls,
184+
// Round-trip prior reasoning so multi-turn thinking is preserved.
185+
thinking: reasoningText || undefined,
143186
})
144187
}
145188
}
@@ -203,6 +246,90 @@ export class NativeOllamaHandler extends BaseProvider implements SingleCompletio
203246
}))
204247
}
205248

249+
/**
250+
* Maps the configured reasoning effort setting to Ollama's native `think`
251+
* request parameter (boolean | "high" | "medium" | "low").
252+
*
253+
* Requires an explicit Ollama opt-in (`enableReasoningEffort === true`)
254+
* before translating `reasoningEffort`. This prevents inherited
255+
* `apiConfiguration.reasoningEffort` values (left over from another
256+
* provider) from silently emitting a `think` param when the Ollama UI
257+
* checkbox is unchecked.
258+
*
259+
* Returns undefined when reasoning is not explicitly enabled, leaving
260+
* the model/Modelfile in control (preserving prior behavior where models
261+
* that emit think/thought tags in content are still handled by TagMatcher).
262+
*
263+
* Note: The Ollama API itself also accepts `"max"` (see
264+
* https://docs.ollama.com/capabilities/thinking), but the installed
265+
* `ollama` SDK (v0.6.x) only types `think` as
266+
* `boolean | "high" | "medium" | "low"`. Until the SDK types catch up,
267+
* "xhigh"/"max" efforts are clamped to "high".
268+
*
269+
* - enableReasoningEffort !== true -> undefined (no think param sent)
270+
* - "disable" -> false (thinking off)
271+
* - "none" / "minimal" -> true (enable thinking with default budget)
272+
* - "low" / "medium" / "high" -> the matching effort level
273+
* - "xhigh" / "max" -> "high" (highest level the SDK currently supports)
274+
*/
275+
private getOllamaThinkParam(): boolean | "high" | "medium" | "low" | undefined {
276+
// Require an explicit Ollama opt-in before mapping reasoningEffort.
277+
// Without this guard, a stale reasoningEffort inherited from another
278+
// provider config could still emit a think param when the UI checkbox
279+
// is unchecked.
280+
if (this.options.enableReasoningEffort !== true) {
281+
return undefined
282+
}
283+
284+
const effort = this.options.reasoningEffort
285+
if (effort === undefined) {
286+
return undefined
287+
}
288+
289+
switch (effort) {
290+
case "disable":
291+
return false
292+
case "none":
293+
case "minimal":
294+
return true
295+
case "low":
296+
return "low"
297+
case "medium":
298+
return "medium"
299+
case "high":
300+
case "xhigh":
301+
case "max":
302+
return "high"
303+
default:
304+
return undefined
305+
}
306+
}
307+
308+
/**
309+
* Builds the shared chat request options (temperature, num_ctx) and the
310+
* conditional `think` parameter used by both `createMessage` and
311+
* `completePrompt`. Centralizing this avoids drift between the streaming
312+
* and single-shot request paths.
313+
*
314+
* Returns a tuple of `[chatOptions, thinkParam]` where `thinkParam` is
315+
* `undefined` when no `think` field should be sent to Ollama.
316+
*/
317+
private buildChatRequestOptions(
318+
useR1Format: boolean,
319+
): [OllamaChatOptions, boolean | "high" | "medium" | "low" | undefined] {
320+
const chatOptions: OllamaChatOptions = {
321+
temperature: this.options.modelTemperature ?? (useR1Format ? DEEP_SEEK_DEFAULT_TEMPERATURE : 0),
322+
}
323+
324+
// Only include num_ctx if explicitly set via ollamaNumCtx
325+
if (this.options.ollamaNumCtx !== undefined) {
326+
chatOptions.num_ctx = this.options.ollamaNumCtx
327+
}
328+
329+
const thinkParam = this.getOllamaThinkParam()
330+
return [chatOptions, thinkParam]
331+
}
332+
206333
override async *createMessage(
207334
systemPrompt: string,
208335
messages: Anthropic.Messages.MessageParam[],
@@ -227,23 +354,24 @@ export class NativeOllamaHandler extends BaseProvider implements SingleCompletio
227354
)
228355

229356
try {
230-
// Build options object conditionally
231-
const chatOptions: OllamaChatOptions = {
232-
temperature: this.options.modelTemperature ?? (useR1Format ? DEEP_SEEK_DEFAULT_TEMPERATURE : 0),
233-
}
234-
235-
// Only include num_ctx if explicitly set via ollamaNumCtx
236-
if (this.options.ollamaNumCtx !== undefined) {
237-
chatOptions.num_ctx = this.options.ollamaNumCtx
238-
}
239-
240-
// Create the actual API request promise
357+
// Build the shared chat options and conditional think parameter.
358+
// Conditionally enabling Ollama's native think parameter lets
359+
// reasoning models (qwen3, deepseek-r1, etc.) emit thinking via
360+
// the dedicated message.thinking field instead of (or in addition
361+
// to) think/thought tags embedded in content.
362+
const [chatOptions, thinkParam] = this.buildChatRequestOptions(useR1Format)
363+
364+
// Create the actual API request promise. The `stream: true` literal
365+
// is kept inline so TypeScript selects the streaming overload of
366+
// client.chat. The `think` parameter is spread conditionally to
367+
// avoid sending an explicit `think: undefined` to the runtime.
241368
const stream = await client.chat({
242369
model: modelId,
243370
messages: ollamaMessages,
244371
stream: true,
245372
options: chatOptions,
246373
tools: this.convertToolsToOllama(metadata?.tools),
374+
...(thinkParam !== undefined ? { think: thinkParam } : {}),
247375
})
248376

249377
let totalInputTokens = 0
@@ -255,6 +383,18 @@ export class NativeOllamaHandler extends BaseProvider implements SingleCompletio
255383

256384
try {
257385
for await (const chunk of stream) {
386+
// Process Ollama's native thinking field. When the think
387+
// parameter is enabled (or the model thinks by default),
388+
// Ollama streams reasoning via message.thinking separately
389+
// from message.content. Surface it as a reasoning chunk so
390+
// it is rendered and preserved like other providers.
391+
if (typeof chunk.message.thinking === "string" && chunk.message.thinking.length > 0) {
392+
yield {
393+
type: "reasoning",
394+
text: chunk.message.thinking,
395+
}
396+
}
397+
258398
if (typeof chunk.message.content === "string" && chunk.message.content.length > 0) {
259399
// Process content through matcher for reasoning detection
260400
for (const matcherChunk of matcher.update(chunk.message.content)) {
@@ -353,21 +493,17 @@ export class NativeOllamaHandler extends BaseProvider implements SingleCompletio
353493
const { id: modelId } = await this.fetchModel()
354494
const useR1Format = modelId.toLowerCase().includes("deepseek-r1")
355495

356-
// Build options object conditionally
357-
const chatOptions: OllamaChatOptions = {
358-
temperature: this.options.modelTemperature ?? (useR1Format ? DEEP_SEEK_DEFAULT_TEMPERATURE : 0),
359-
}
360-
361-
// Only include num_ctx if explicitly set via ollamaNumCtx
362-
if (this.options.ollamaNumCtx !== undefined) {
363-
chatOptions.num_ctx = this.options.ollamaNumCtx
364-
}
496+
// Reuse the shared request-option builder so single-shot
497+
// completions respect the same reasoning configuration as the
498+
// streaming path.
499+
const [chatOptions, thinkParam] = this.buildChatRequestOptions(useR1Format)
365500

366501
const response = await client.chat({
367502
model: modelId,
368503
messages: [{ role: "user", content: prompt }],
369504
stream: false,
370505
options: chatOptions,
506+
...(thinkParam !== undefined ? { think: thinkParam } : {}),
371507
})
372508

373509
return response.message?.content || ""

webview-ui/src/components/settings/providers/Ollama.tsx

Lines changed: 46 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,17 @@
11
import { useState, useCallback, useMemo, useEffect } from "react"
22
import { useEvent } from "react-use"
33
import { VSCodeTextField } from "@vscode/webview-ui-toolkit/react"
4+
import { Checkbox } from "vscrui"
45

5-
import type { ProviderSettings, ExtensionMessage, ModelRecord } from "@roo-code/types"
6+
import { type ProviderSettings, type ExtensionMessage, type ModelRecord, ollamaDefaultModelInfo } from "@roo-code/types"
67

78
import { useAppTranslation } from "@src/i18n/TranslationContext"
89
import { useRouterModels } from "@src/components/ui/hooks/useRouterModels"
910
import { vscode } from "@src/utils/vscode"
1011

1112
import { inputEventTransform } from "../transforms"
1213
import { ModelPicker } from "../ModelPicker"
14+
import { ThinkingBudget } from "../ThinkingBudget"
1315

1416
type OllamaProps = {
1517
apiConfiguration: ProviderSettings
@@ -131,6 +133,49 @@ export const Ollama = ({ apiConfiguration, setApiConfigurationField }: OllamaPro
131133
{t("settings:providers.ollama.numCtxHelp")}
132134
</div>
133135
</VSCodeTextField>
136+
<div className="flex flex-col gap-1">
137+
<Checkbox
138+
checked={apiConfiguration.enableReasoningEffort ?? false}
139+
onChange={(checked: boolean) => {
140+
setApiConfigurationField("enableReasoningEffort", checked)
141+
142+
if (checked) {
143+
// Restore the last selected effort level if one was
144+
// previously chosen; otherwise default to "medium" so
145+
// the request actually enables Ollama's native think
146+
// parameter. Without a value, the ThinkingBudget Select
147+
// would show "None" (disable) and getOllamaThinkParam()
148+
// would return undefined, sending no think parameter
149+
// despite the checkbox being on. Preserving the prior
150+
// value avoids wiping the user's effort choice when
151+
// toggling the checkbox off and back on.
152+
setApiConfigurationField("reasoningEffort", apiConfiguration.reasoningEffort ?? "medium")
153+
}
154+
// When unchecked, leave reasoningEffort untouched so the
155+
// user's prior selection is preserved across toggles. The
156+
// handler gates on enableReasoningEffort === true, so a
157+
// stale reasoningEffort value will not emit a think param
158+
// while the checkbox is off.
159+
}}>
160+
{t("settings:providers.ollama.thinking")}
161+
</Checkbox>
162+
<div className="text-xs text-vscode-descriptionForeground mt-1">
163+
{t("settings:providers.ollama.thinkingHelp")}
164+
</div>
165+
{!!apiConfiguration.enableReasoningEffort && (
166+
<ThinkingBudget
167+
apiConfiguration={apiConfiguration}
168+
setApiConfigurationField={setApiConfigurationField}
169+
// Ollama models don't advertise reasoning capabilities, so
170+
// synthesize a model info that exposes the effort levels
171+
// Ollama's native `think` parameter supports (low/medium/high).
172+
modelInfo={{
173+
...ollamaDefaultModelInfo,
174+
supportsReasoningEffort: true,
175+
}}
176+
/>
177+
)}
178+
</div>
134179
<div className="text-sm text-vscode-descriptionForeground">
135180
{t("settings:providers.ollama.description")}
136181
<span className="text-vscode-errorForeground ml-1">{t("settings:providers.ollama.warning")}</span>

0 commit comments

Comments
 (0)