Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
afc9f74
refactor(llm): extract stream content parsing into a named seam
dsapandora Jul 26, 2026
3bb5136
feat(llm): add normalized provider Adapter interface
dsapandora Jul 26, 2026
4cfc8e1
refactor(llm): move stream content parser to the adapter module
dsapandora Jul 26, 2026
502ea6e
feat(llm): add LangChainAdapter
dsapandora Jul 26, 2026
695b53d
feat(llm): add AnthropicAdapter
dsapandora Jul 26, 2026
44f2e67
feat(llm): add OpenAIAdapter
dsapandora Jul 26, 2026
1956036
feat(llm): add drive_adapter Event→callback shim
dsapandora Jul 26, 2026
c2c67e4
feat(llm): drive the LangChain streaming path through the adapter
dsapandora Jul 26, 2026
1de1e3d
fix(llm): flatten non-streaming _chat content to a string
dsapandora Jul 27, 2026
d779495
feat(llm): route the native Anthropic path through an adapter
dsapandora Jul 27, 2026
ce853f9
feat(llm): route the OpenAI Responses path through an adapter
dsapandora Jul 27, 2026
4f943ef
refactor(llm): unify the non-streaming path through the adapter
dsapandora Jul 28, 2026
e1ae7d9
feat(llm_anthropic): per-node extended-thinking control with context …
dsapandora Jul 28, 2026
1fd66cd
test(agent_crewai): drive NativeAnthropicAdapter for native stop-thre…
dsapandora Jul 28, 2026
8c488f9
fix(sync-models): make the extendedThinking toggle capabilities-aware
dsapandora Jul 28, 2026
ae6df02
refactor(llm): address CodeRabbit — dead adapters, store=False, Respo…
dsapandora Jul 28, 2026
3b61c38
fix(sync-models): scope the extendedThinking toggle to nodes that def…
dsapandora Jul 29, 2026
5daf0cc
docs(llm_anthropic): soften the extendedThinking field description
dsapandora Jul 29, 2026
3f6dc36
fix(llm): keep the non-streaming path on invoke via adapter.collect()
dsapandora Jul 30, 2026
9c1da26
fix(llm): address review — parse_bool, opus-5 toggle, custom, Respons…
dsapandora Jul 30, 2026
8fc1992
fix(llm): close Responses stream + honor Haiku 4.5 extended thinking
dsapandora Jul 31, 2026
96468a4
refactor(llm): reuse LangChainAdapter.collect() in the Responses fall…
dsapandora Jul 31, 2026
989a6e7
refactor(llm): expose collect() reasoning so the fallback keeps its lane
dsapandora Jul 31, 2026
fe43637
refactor(llm): read the block vocabulary from flatten_content_blocks
dsapandora Jul 31, 2026
9c318a0
docs(llm_anthropic): describe the extended-thinking toggle as opt-in
dsapandora Jul 31, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions nodes/src/nodes/llm_anthropic/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ directly for save-time validation. The configured `modelOutputTokens` is passed
the model as `max_tokens`. Token counts for budgeting are estimated at roughly
4 characters per token.

When the selected model is flagged as reasoning-capable, the node enables
extended thinking automatically and streams the model's reasoning over the
`thinking` SSE lane; see "Extended thinking" below.
Extended thinking is opt-in per node via the `extendedThinking` field and is off by
default. When it is enabled on a reasoning-capable model, the node streams the model's
reasoning over the `thinking` SSE lane on the interactive streaming path only — the
agent / `expectJson` path never requests it; see "Extended thinking" below.

---

Expand All @@ -37,6 +38,7 @@ extended thinking automatically and streams the model's reasoning over the
| `modelSource` | string | Where the model definition comes from (`manual` or `openrouter`) |
| `model` | string (custom profile only) | Anthropic model ID, used only when `profile` is `custom` |
| `modelTotalTokens` | number (custom profile only) | Total context tokens for the custom profile; must be greater than 0 |
| `extendedThinking` | boolean, `false` | Request extended thinking for this node. Ignored unless the model is reasoning-capable |

The model ID and token limits for named profiles are fixed by the profile. Only the
`custom` profile exposes `model` and `modelTotalTokens` directly.
Expand All @@ -61,14 +63,16 @@ Default profile: **Claude Sonnet 4.6**.

## Extended thinking

Whether thinking is requested is driven by the model's `capabilities.reasoning`
flag in the node configuration (stamped from OpenRouter model sync). When set,
the node builds provider-correct thinking parameters based on the model name.
Whether thinking is requested is driven by two things: the model's
`capabilities.reasoning` flag in the node configuration (stamped from OpenRouter
model sync) **and** the node's own `extendedThinking` toggle, which is off by
default. Both must be on. When they are, the node builds provider-correct
thinking parameters based on the model name.
Routing prefixes such as `openrouter/anthropic/` are stripped before matching.

| Model | Thinking parameters sent |
| -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Any Haiku model | None. Haiku has no extended thinking; sending parameters would return a 400. |
| Legacy Claude 3 / 3.5 Haiku | None. Those models have no extended thinking; sending parameters would return a 400. Haiku 4.5 and newer are not excluded and follow the row below. |
| `claude-opus-4-7` / `claude-opus-4-8` | `thinking: {type: "adaptive", display: "summarized"}` (adaptive thinking) |
| Other Claude models | `thinking: {type: "enabled", budget_tokens: N}` plus the `interleaved-thinking-2025-05-14` beta header, where `N` is half the output-token limit (minimum 2,048, always below `max_tokens`). Skipped entirely if the output window is too small for a valid budget. |

Expand Down
37 changes: 15 additions & 22 deletions nodes/src/nodes/llm_anthropic/anthropic.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,10 @@
from ai.common.chat import ChatBase
from ai.common.config import Config
from ai.common.llm_native_stream import build_anthropic_thinking_kwargs, gate_model_name
from ai.common.utils import parse_bool
from langchain_anthropic import ChatAnthropic


# Interim: extended thinking is disabled at the node until the normalized provider adapter
# lands (RFC #1679). With thinking on, Anthropic returns typed content blocks (thinking + text)
# that the agent / expectJson path cannot consume (incident #1658; flatten defense-in-depth in
# #1659). We gate activation here rather than flipping capabilities.reasoning, because that flag
# is stamped by tools/sync_models and re-stamps to true on the next sync. Flip back to re-enable.
_EXTENDED_THINKING_ENABLED = False


def _estimate_token_ids(text: str) -> list:
"""Estimate token ids at ~4 chars/token."""
return [0] * max(1, (len(text) + 3) // 4)
Expand Down Expand Up @@ -78,23 +71,23 @@ def __init__(self, provider: str, connConfig: Dict[str, Any], bag: Dict[str, Any
# Init the chat base
super().__init__(provider, connConfig, bag)

# Get the LLM
kwargs: Dict[str, Any] = {
'model': model,
'api_key': apikey,
'max_tokens': self._modelOutputTokens,
'custom_get_token_ids': _estimate_token_ids,
}
if self._is_reasoning and _EXTENDED_THINKING_ENABLED:
kwargs.update(build_anthropic_thinking_kwargs(model_gate, self._modelOutputTokens))

self._extended_thinking = bool(kwargs.get('thinking'))
# Only route through the native handler when thinking is actually on;
# non-reasoning models stay on the default LangChain path.
# Extended thinking is opt-in per node (config extendedThinking, default off) and only
# for reasoning models. It is NOT baked into the client: the native streaming adapter adds
# it per call, so thinking activates on the interactive streaming path only — never on the
# agent / expectJson path (which has no streaming and stays on the plain LangChain client).
self._thinking_mode_kwargs: Dict[str, Any] = {}
if self._is_reasoning and parse_bool(config.get('extendedThinking')):
Comment thread
dsapandora marked this conversation as resolved.
self._thinking_mode_kwargs = build_anthropic_thinking_kwargs(model_gate, self._modelOutputTokens)
self._extended_thinking = bool(self._thinking_mode_kwargs)
if self._extended_thinking:
self._native_stream_provider = 'anthropic'

self._llm = ChatAnthropic(**kwargs)
self._llm = ChatAnthropic(
model=model,
api_key=apikey,
max_tokens=self._modelOutputTokens,
custom_get_token_ids=_estimate_token_ids,
)

# Save our chat class into the bag
bag['chat'] = self
26 changes: 26 additions & 0 deletions nodes/src/nodes/llm_anthropic/services.json
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,12 @@
"title": "Tokens",
"description": "Total Tokens"
},
"extendedThinking": {
"type": "boolean",
"title": "Extended thinking",
"default": false,
"description": "Enable Anthropic extended thinking (reasoning) for this node. Off by default. Applies to reasoning-capable models on the interactive chat path."
},
"anthropic.custom": {
"object": "custom",
"properties": [
Expand All @@ -345,34 +351,39 @@
"object": "claude-sonnet-4-6",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-opus-4-6": {
"object": "claude-opus-4-6",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-haiku-4-5": {
"object": "claude-haiku-4-5",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
Comment thread
coderabbitai[bot] marked this conversation as resolved.
},
"anthropic.claude-sonnet-4-5": {
"object": "claude-sonnet-4-5",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-opus-4-5": {
"object": "claude-opus-4-5",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
Expand Down Expand Up @@ -523,6 +534,7 @@
"object": "claude-opus-4-7",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
Expand All @@ -537,97 +549,111 @@
"object": "claude-fable-5",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-fable-latest": {
"object": "claude-fable-latest",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-haiku-latest": {
"object": "claude-haiku-latest",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-opus-4": {
"object": "claude-opus-4",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-opus-4-1": {
"object": "claude-opus-4-1",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-opus-4-7-fast": {
"object": "claude-opus-4-7-fast",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-opus-4-8": {
"object": "claude-opus-4-8",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-opus-4-8-fast": {
"object": "claude-opus-4-8-fast",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-opus-latest": {
"object": "claude-opus-latest",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-sonnet-4": {
"object": "claude-sonnet-4",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-sonnet-5": {
"object": "claude-sonnet-5",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-sonnet-latest": {
"object": "claude-sonnet-latest",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-opus-5": {
Comment thread
asclearuc marked this conversation as resolved.
"object": "claude-opus-5",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
},
"anthropic.claude-opus-5-fast": {
"object": "claude-opus-5-fast",
"properties": [
"llm.cloud.apikey",
"extendedThinking",
"llm.cloud.modelSource"
]
}
Expand Down
9 changes: 3 additions & 6 deletions nodes/test/agent_crewai/test_stop_words.py
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,7 @@ def _load_native_stream():


def _run_native_capture(ns, stop_value):
"""Drive _stream_anthropic_messages_api with fakes; return the stop the payload got."""
"""Drive the native Anthropic adapter with fakes; return the stop the payload got."""
captured: dict = {}

class _FakeLLM:
Expand All @@ -258,11 +258,8 @@ class _FakeChat:
ns._open_raw_message_stream = lambda client, payload: iter(()) # no events -> no text
token = ns.STOP_SEQUENCES_VAR.set(stop_value)
try:
# Produces no text, so the function raises after building the payload — we only
# assert the payload wiring, which happens before any streaming.
ns._stream_anthropic_messages_api(_FakeChat(), 'prompt', lambda t: None, None, None)
except RuntimeError:
pass
# Drain the adapter; the payload (with the stop) is built before any streaming.
list(ns.NativeAnthropicAdapter(_FakeChat()).stream('prompt'))
finally:
ns.STOP_SEQUENCES_VAR.reset(token)
return captured.get('stop', 'UNSET')
Expand Down
100 changes: 100 additions & 0 deletions nodes/test/llm_anthropic/test_extended_thinking_init.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# =============================================================================
# MIT License
# Copyright (c) 2026 Aparavi Software AG
# =============================================================================

"""Node-init tests for the llm_anthropic extended-thinking toggle.

`test_thinking_mode_injected_into_payload_per_call` sets `_thinking_mode_kwargs`
by hand, so it starts after the interesting part. These tests drive the real
chain instead:

config['extendedThinking'] -> parse_bool -> _is_reasoning gate
-> build_anthropic_thinking_kwargs(model_gate, ...) -> _thinking_mode_kwargs
-> _native_stream_provider = 'anthropic'

Four ways that chain can break and nothing else would catch: `parse_bool`
reverted to `bool`, the `_is_reasoning` gate dropped, the wrong `model_gate`
passed, or `_native_stream_provider` left unset so the native handler never runs.
"""

import importlib.util
import os
import sys
import types

import pytest

from ai.common.config import Config

_HERE = os.path.dirname(os.path.abspath(__file__))
_MOD_PATH = os.path.join(_HERE, '..', '..', 'src', 'nodes', 'llm_anthropic', 'anthropic.py')

_MODEL = 'claude-sonnet-4-6'


def _load_node_module():
"""Load anthropic.py standalone, stubbing the langchain_anthropic client."""
saved = sys.modules.get('langchain_anthropic')
stub = types.ModuleType('langchain_anthropic')
stub.ChatAnthropic = type('ChatAnthropic', (), {'__init__': lambda self, **kw: None})
sys.modules['langchain_anthropic'] = stub
try:
spec = importlib.util.spec_from_file_location('_llm_anthropic_node', _MOD_PATH)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
finally:
if saved is None:
sys.modules.pop('langchain_anthropic', None)
else:
sys.modules['langchain_anthropic'] = saved


def _build_chat(monkeypatch, *, reasoning: bool, toggle=None):
"""Instantiate the node over a controlled config and return it."""
config = {
'model': _MODEL,
'apikey': 'sk-ant-test',
'modelTotalTokens': 200000,
'modelOutputTokens': 8192,
'capabilities': {'reasoning': reasoning},
}
if toggle is not None:
config['extendedThinking'] = toggle

# Both the node and ChatBase read their config through this one call.
monkeypatch.setattr(Config, 'getNodeConfig', staticmethod(lambda *a, **k: dict(config)))
return _load_node_module().Chat('anthropic', {}, {})


def test_toggle_absent_leaves_thinking_off(monkeypatch):
chat = _build_chat(monkeypatch, reasoning=True)

assert chat._thinking_mode_kwargs == {}
assert chat._native_stream_provider == ''


def test_toggle_on_reasoning_model_arms_the_native_handler(monkeypatch):
chat = _build_chat(monkeypatch, reasoning=True, toggle=True)

assert chat._thinking_mode_kwargs != {}
assert chat._native_stream_provider == 'anthropic'


def test_toggle_on_non_reasoning_model_stays_off(monkeypatch):
# The capability gate wins: the toggle alone must not request thinking.
chat = _build_chat(monkeypatch, reasoning=False, toggle=True)

assert chat._thinking_mode_kwargs == {}
assert chat._native_stream_provider == ''


@pytest.mark.parametrize('falsy', ['false', 'False', '0', ''])
def test_string_false_from_the_form_stays_off(monkeypatch, falsy):
# The case that fails the moment parse_bool is swapped back for bool():
# a non-empty string like 'false' is truthy to bool().
chat = _build_chat(monkeypatch, reasoning=True, toggle=falsy)

assert chat._thinking_mode_kwargs == {}
assert chat._native_stream_provider == ''
Loading
Loading