diff --git a/code_review.md b/code_review.md index ad211cba4..40fdd739e 100644 --- a/code_review.md +++ b/code_review.md @@ -42,7 +42,7 @@ Guides load on demand, so the general passes here stay short. | `runtime/` state and recovery | serde and replay type fidelity, real failure-path tests | [review-guides/runtime-state-recovery.md](review-guides/runtime-state-recovery.md) | | Python-Java bridge | cross-language parity, type mapping across Pemja | [review-guides/python-java-bridge.md](review-guides/python-java-bridge.md) | | `api/` contract | API shape, compatibility policy, deprecation | [review-guides/api-contract.md](review-guides/api-contract.md) | -| `dist` and dependency | shading, LICENSE and NOTICE, dist registration | planned | +| `dist` and dependency | shading, LICENSE and NOTICE, dist registration | [review-guides/dist-dependency.md](review-guides/dist-dependency.md) | | docs-only | facts match their source of truth | [review-guides/docs-only.md](review-guides/docs-only.md) | ## Required Review Passes diff --git a/dist/src/main/resources/META-INF/NOTICE b/dist/src/main/resources/META-INF/NOTICE index a1cf484cf..aa53540ab 100644 --- a/dist/src/main/resources/META-INF/NOTICE +++ b/dist/src/main/resources/META-INF/NOTICE @@ -130,9 +130,9 @@ See bundled license files for details. - io.github.ollama4j:ollama4j:1.1.5 - org.jsoup:jsoup:1.21.2 -- com.anthropic:anthropic-java:2.11.1 -- com.anthropic:anthropic-java-client-okhttp:2.11.1 -- com.anthropic:anthropic-java-core:2.11.1 +- com.anthropic:anthropic-java:2.12.0 +- com.anthropic:anthropic-java-client-okhttp:2.12.0 +- com.anthropic:anthropic-java-core:2.12.0 - org.reactivestreams:reactive-streams:1.0.4 - io.modelcontextprotocol.sdk:mcp:0.16.0 - io.modelcontextprotocol.sdk:mcp-json-jackson2:0.16.0 diff --git a/docs/README.md b/docs/README.md index dcbf4611c..a15fe0764 100644 --- a/docs/README.md +++ b/docs/README.md @@ -22,12 +22,16 @@ $ docker run -v $(pwd):/src -p 1313:1313 jakejarvis/hugo-extended:latest server #### Local Hugo installation: -Make sure you have installed [Hugo](https://gohugo.io/getting-started/installing/) on your system. +`setup_hugo.sh` installs Hugo 0.110.0, the version the docs build is pinned to. A newer Hugo release removed an internal template the pinned `hugo-book` theme calls, so building with a current Hugo fails with `no such template "_internal/google_analytics_async.html"`. + +On Linux, install it with: ```sh $ ./setup_hugo.sh ``` +`setup_hugo.sh` downloads a Linux binary, x86_64 or ARM64. On other platforms, install the extended Hugo 0.110.0 from the [Hugo releases page](https://github.com/gohugoio/hugo/releases/tag/v0.110.0) yourself. + Then build the docs from source: ```sh diff --git a/docs/content.zh/_index.md b/docs/content.zh/_index.md index 33bbd1084..b25810aeb 100644 --- a/docs/content.zh/_index.md +++ b/docs/content.zh/_index.md @@ -41,7 +41,7 @@ under the License. ### Community Sync -我们每周会举行一次在线同步会议，欢迎所有人参与。请查看此 [GitHub 讨论页面](https://github.com/apache/flink-agents/discussions/66) 获取下次会议的时间表、议程以及往期会议记录。 +我们每周会举行一次在线同步会议，欢迎所有人参与。请查看此 [GitHub 讨论页面](https://github.com/apache/flink-agents/discussions/419) 获取下次会议的时间表、议程以及往期会议记录。 {{< /columns >}} diff --git a/docs/content/_index.md b/docs/content/_index.md index 4ed63a5a0..2adfcb438 100644 --- a/docs/content/_index.md +++ b/docs/content/_index.md @@ -30,6 +30,18 @@ under the License. If you’re interested in playing around with Flink Agents, check out our [get started documentation]({{< ref "docs/get-started/overview" >}}). It provides a step by step introduction on how to install Flink Agents and build agents with it. +### Quickstart + +* [Workflow Agent]({{< ref "docs/get-started/quickstart/workflow_agent" >}}) - Define agent logic as an explicit event-driven workflow +* [ReAct Agent]({{< ref "docs/get-started/quickstart/react_agent" >}}) - Let the model plan and call tools in a reason-act loop +* [YAML Agent]({{< ref "docs/get-started/quickstart/yaml_agent" >}}) - Configure an agent declaratively in YAML + +### Build and Operate + +* [Chat Models]({{< ref "docs/development/chat_models" >}}), [Tools]({{< ref "docs/development/tool_use" >}}) and [Memory]({{< ref "docs/development/memory/overview" >}}) - The building blocks of an agent +* [Integrate with Flink]({{< ref "docs/development/integrate_with_flink" >}}) - Run an agent inside a Flink job +* [Deployment]({{< ref "docs/operations/deployment" >}}) and [Monitoring]({{< ref "docs/operations/monitoring" >}}) - Take an agent to production + <---> ## Get Help with Flink Agents @@ -44,7 +56,7 @@ See the [Apache Flink website](https://flink.apache.org/what-is-flink/community/ ### Community Sync -There is a weekly online sync. Everyone is welcome to join. Please find the schedule, agenda for the next sync, and records of previous syncs in this [github discussion page](https://github.com/apache/flink-agents/discussions/66). +There is a weekly online sync. Everyone is welcome to join. Please find the schedule, agenda for the next sync, and records of previous syncs in this [github discussion page](https://github.com/apache/flink-agents/discussions/419). {{< /columns >}} diff --git a/docs/content/docs/development/chat_models.md b/docs/content/docs/development/chat_models.md index aad8c12a4..8f9962aa1 100644 --- a/docs/content/docs/development/chat_models.md +++ b/docs/content/docs/development/chat_models.md @@ -292,6 +292,7 @@ Anthropic provides cloud-based chat models featuring the Claude family, known fo | `tools` | List[str] | None | List of tool names available to the model | | `max_tokens` | int | `1024` | Maximum number of tokens to generate | | `temperature` | float | `0.1` | Sampling temperature (0.0 to 1.0) | +| `json_prefill` | bool | `False` | Prefill assistant response with "{" to enforce JSON output (applies only on models that accept assistant-message prefilling; disabled when tools are used, or when the request carries an `output_config`) | | `additional_kwargs` | dict | `{}` | Additional Anthropic API parameters | {{< /tab >}} @@ -306,9 +307,9 @@ Anthropic provides cloud-based chat models featuring the Claude family, known fo | `tools` | List | None | List of tool names available to the model | | `max_tokens` | long | `1024` | Maximum number of tokens to generate | | `temperature` | double | `0.1` | Sampling temperature (0.0 to 1.0) | -| `json_prefill` | boolean | `true` | Prefill assistant response with "{" to enforce JSON output (disabled when tools are used) | +| `json_prefill` | boolean | `false` | Prefill assistant response with "{" to enforce JSON output (applies only on models that accept assistant-message prefilling; disabled when tools are used, or when the request carries an `output_config`) | | `strict_tools` | boolean | `false` | Enable strict mode for tool calling schemas | -| `additional_kwargs` | Map | `{}` | Additional Anthropic API parameters (top_k, top_p, stop_sequences) | +| `additional_kwargs` | Map | `{}` | Additional Anthropic API parameters (top_k, top_p, stop_sequences); an `output_config` supplied here takes precedence over one derived from an output schema | {{< /tab >}} diff --git a/docs/content/docs/get-started/overview.md b/docs/content/docs/get-started/overview.md index 4c99eb84c..2fbef0c32 100644 --- a/docs/content/docs/get-started/overview.md +++ b/docs/content/docs/get-started/overview.md @@ -47,3 +47,15 @@ To get started with Apache Flink Agents, you can checkout the following quicksta - [ReAct Agent Quickstart]({{< ref "docs/get-started/quickstart/react_agent" >}}) - [Skills Agent Quickstart]({{< ref "docs/get-started/quickstart/skills_agent" >}}) - [YAML Agent Quickstart]({{< ref "docs/get-started/quickstart/yaml_agent" >}}) + +## Preview Status + +{{< hint warning >}} +Apache Flink Agents 0.x releases are preview versions. They may contain known +or unknown issues, including potential security risks. The APIs and +configuration are experimental and may change in backward-incompatible ways +before 1.0. Review the [publicly disclosed known issues](https://github.com/apache/flink-agents/issues) +and assess these risks before production use. Report suspected undisclosed +security vulnerabilities privately through the [ASF Security Team](https://www.apache.org/security/) +instead of creating a public GitHub issue. +{{< /hint >}} diff --git a/docs/setup_hugo.sh b/docs/setup_hugo.sh index 5f3fa2902..015b228b0 100644 --- a/docs/setup_hugo.sh +++ b/docs/setup_hugo.sh @@ -18,8 +18,13 @@ ################################################################################ # setup hugo -HUGO_REPO=https://github.com/gohugoio/hugo/releases/download/v0.110.0/hugo_extended_0.110.0_Linux-64bit.tar.gz -HUGO_ARTIFACT=hugo_extended_0.110.0_Linux-64bit.tar.gz +case "$(uname -m)" in + x86_64) HUGO_ARCH=Linux-64bit ;; + aarch64) HUGO_ARCH=linux-arm64 ;; + *) echo "No Hugo binary for architecture $(uname -m)" ; exit 1 ;; +esac +HUGO_ARTIFACT=hugo_extended_0.110.0_${HUGO_ARCH}.tar.gz +HUGO_REPO=https://github.com/gohugoio/hugo/releases/download/v0.110.0/${HUGO_ARTIFACT} if ! curl --fail -OL $HUGO_REPO ; then echo "Failed to download Hugo binary" exit 1 diff --git a/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java b/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java index 79917140b..bf8e0afe0 100644 --- a/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java +++ b/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java @@ -19,6 +19,7 @@ import com.anthropic.client.AnthropicClient; import com.anthropic.client.okhttp.AnthropicOkHttpClient; +import com.anthropic.core.JsonSchemaLocalValidation; import com.anthropic.core.JsonValue; import com.anthropic.models.messages.ContentBlock; import com.anthropic.models.messages.ContentBlockParam; @@ -26,6 +27,7 @@ import com.anthropic.models.messages.MessageCreateParams; import com.anthropic.models.messages.MessageParam; import com.anthropic.models.messages.Model; +import com.anthropic.models.messages.OutputConfig; import com.anthropic.models.messages.TextBlockParam; import com.anthropic.models.messages.Tool; import com.anthropic.models.messages.ToolResultBlockParam; @@ -48,6 +50,7 @@ import java.util.List; import java.util.Map; import java.util.Optional; +import java.util.Set; import java.util.stream.Collectors; /** @@ -114,24 +117,173 @@ public void close() { this.client.close(); } + // Models Anthropic documents native structured-output support for. Source of truth: + // https://platform.claude.com/docs/en/build-with-claude/structured-outputs + // + // The documented rule is generational rather than a per-snapshot list: structured outputs are + // generally available for Claude 4.5 and later models, and for Claude Mythos Preview. Names + // from the 4.6 generation onward carry no date and are pinned, so the name is itself the + // snapshot and is matched exactly. + // + // The three 4.5-generation names are aliases that front a dated snapshot, so a request may + // carry either the alias or the snapshot behind it and both have to match. Those match the + // alias itself or a name continuing with a "-" separator, which covers + // claude-sonnet-4-5-20250929. A name that extends the alias without that separator is a + // different minor version and is capable only if the exact set names it. The alias also has + // to retain the minor version: "claude-opus-4" would capture claude-opus-4-1-20250805, which + // predates the cutoff and is not capable. + // + // A name outside both sets reports not-capable and degrades to the prompt-engineering + // fallback rather than failing at the provider. + private static final Set NATIVE_STRUCTURED_OUTPUT_MODELS = + Set.of( + "claude-opus-4-6", + "claude-opus-4-7", + "claude-opus-4-8", + "claude-opus-5", + "claude-sonnet-4-6", + "claude-sonnet-5", + "claude-fable-5", + "claude-mythos-5", + "claude-mythos-preview"); + + private static final Set NATIVE_STRUCTURED_OUTPUT_ALIAS_PREFIXES = + Set.of("claude-opus-4-5", "claude-sonnet-4-5", "claude-haiku-4-5"); + + /** + * Whether Anthropic documents native structured-output support for {@code effectiveModel}. + * + *

See the allowlists above for the source of truth and for why a 4.5-generation alias also + * matches the dated snapshot behind it while every other name is matched exactly. An + * unrecognized name reports {@code false} so that it degrades to the prompt-engineering + * fallback rather than failing at the provider. + * + *

Reads no instance state, so capability stays answerable independently of how the + * connection was configured. + */ + @Override + protected boolean supportsNativeStructuredOutput(String effectiveModel) { + // Load-bearing: the allowlist is an immutable Set, whose contains(null) throws rather than + // reporting absence. + if (effectiveModel == null) { + return false; + } + return NATIVE_STRUCTURED_OUTPUT_MODELS.contains(effectiveModel) + || NATIVE_STRUCTURED_OUTPUT_ALIAS_PREFIXES.stream() + .anyMatch( + prefix -> + effectiveModel.equals(prefix) + || effectiveModel.startsWith(prefix + "-")); + } + + // Models Anthropic documents as rejecting assistant-message prefilling. Source of truth: + // https://platform.claude.com/docs/en/build-with-claude/working-with-messages#putting-words-in-claudes-mouth + // + // Prefilling is not supported from the Claude 4.6 generation onward, nor on Claude Mythos + // Preview, Claude Fable 5 or Claude Mythos 5; a request that prefills one of them is answered + // with a 400 rather than a completion. Anthropic publishes no programmatic signal for prefill + // support the way it does for structured outputs, so the rule has to be a maintained list of + // names. Those names carry no date and are pinned, so the name is itself the snapshot and is + // matched exactly, and a name outside the list is treated as accepting the prefill. + // + // Kept in its own storage rather than derived from the structured-output allowlists above, + // whose contents it currently coincides with. The two encode different documented boundaries: + // structured output starts at the 4.5 generation while prefill rejection starts at 4.6, so the + // three 4.5-generation names are structured-output capable and still accept a prefill. Sharing + // one list would hold only until a model moves one boundary without moving the other. + private static final Set PREFILL_UNSUPPORTED_MODELS = + Set.of( + "claude-opus-4-6", + "claude-opus-4-7", + "claude-opus-4-8", + "claude-opus-5", + "claude-sonnet-4-6", + "claude-sonnet-5", + "claude-fable-5", + "claude-mythos-5", + "claude-mythos-preview"); + + /** + * Whether {@code effectiveModel} accepts the prefilled assistant {@code "{"} message. + * + *

See the list above for the source of truth and for why it is matched exactly and kept + * apart from the structured-output allowlists. An unrecognized name reports {@code true}, which + * matches the documented rule: prefilling is the long-standing behaviour and only the listed + * names withdraw it. The cost of that default runs the opposite way to {@link + * #supportsNativeStructuredOutput}: a rejecting model this list has not caught up with is + * prefilled and answered with a 400, where an unrecognized name on the structured-output path + * degrades silently to the prompt-engineering fallback instead. + */ + static boolean supportsJsonPrefill(String effectiveModel) { + // Load-bearing: the list is an immutable Set, whose contains(null) throws rather than + // reporting absence. + if (effectiveModel == null) { + return true; + } + return !PREFILL_UNSUPPORTED_MODELS.contains(effectiveModel); + } + + /** + * Derives the native {@code output_config} for a POJO class through the SDK's typed + * structured-output builder. + * + *

The Kotlin facade {@code StructuredOutputsKt.outputFormatFromClass} would produce this + * directly, but it is compiled {@code ACC_SYNTHETIC} and so cannot be named from Java. The + * typed builder generates the same schema; the config is extracted from the throwaway request + * it produces and reattached to the real one, which also avoids that overload's side effect of + * retyping the request and the response as {@code StructuredMessageCreateParams} and {@code + * StructuredMessage}. The throwaway request is never sent, so its placeholder model, message + * and token limit only have to satisfy the builder's required-field check. + * + *

Local schema validation is off so that the provider, not the client, is the authority on + * which schemas it accepts. + */ + private static OutputConfig toNativeOutputConfig(Class schemaClass) { + return MessageCreateParams.builder() + .model(Model.of("")) + .addUserMessage("") + .maxTokens(1) + .outputConfig(schemaClass, JsonSchemaLocalValidation.NO) + .build() + .rawParams() + .outputConfig() + .orElseThrow( + () -> + new IllegalStateException( + "Anthropic SDK did not produce an output_config for schema " + + schemaClass.getName())); + } + @Override public ChatMessage chat( List messages, List tools, Map modelParams) { + return chat(messages, tools, modelParams, null); + } + + /** + * Translates {@code outputSchema} into Anthropic's native {@code output_config.format} when it + * is a POJO {@link Class}, the effective model is one Anthropic documents structured-output + * support for, and the caller has not already supplied its own {@code output_config}. Any other + * combination sends no derived schema, so the request carries only the output configuration the + * caller supplied, if any, and a schema that cannot be sent natively degrades to the + * prompt-engineering fallback rather than failing at the provider. + * + *

A request that ends up carrying an {@code output_config} — whether derived here or + * supplied by the caller — also suppresses the {@code json_prefill} parameter, since Anthropic + * documents message prefilling as incompatible with structured outputs. + */ + @Override + public ChatMessage chat( + List messages, + List tools, + Map modelParams, + Object outputSchema) { try { - // Check if JSON prefill is requested before building request (modelParams may be - // modified). - boolean jsonPrefillRequested = - modelParams != null && Boolean.TRUE.equals(modelParams.get("json_prefill")); - // JSON prefill is automatically disabled when tools are passed in the request, - // because it interferes with native tool calling. - boolean hasToolsInRequest = tools != null && !tools.isEmpty(); - boolean jsonPrefillApplied = jsonPrefillRequested && !hasToolsInRequest; - - MessageCreateParams params = buildRequest(messages, tools, modelParams); - Message response = client.messages().create(params); - ChatMessage result = convertResponse(response, jsonPrefillApplied); + BuiltRequest built = buildRequest(messages, tools, modelParams, outputSchema); + Message response = client.messages().create(built.params); + ChatMessage result = convertResponse(built, response); // Stash token usage String modelName = null; @@ -153,10 +305,19 @@ public ChatMessage chat( } } - private MessageCreateParams buildRequest( + /** + * Builds the request and reports the JSON prefill decision it made. + * + *

Whether the prefilled assistant {@code "{"} message was appended cannot be recomputed from + * the request alone, and {@link #convertResponse} must know it to reconstruct the full JSON + * document. Deciding once here and carrying the answer out keeps the request and the response + * conversion from disagreeing. + */ + BuiltRequest buildRequest( List messages, List tools, - Map rawModelParams) { + Map rawModelParams, + Object outputSchema) { Map modelParams = rawModelParams != null ? new HashMap<>(rawModelParams) : new HashMap<>(); @@ -216,19 +377,72 @@ private MessageCreateParams buildRequest( applyAdditionalKwargs(builder, additionalKwargs); } - // Handle JSON prefill - append a prefilled assistant message with "{" to enforce JSON - // output. Note: JSON prefill is incompatible with tool use as it forces the model to output - // JSON text instead of using native tool_use content blocks. Automatically disable - // json_prefill when tools are actually passed in the request. + // Read here rather than inside the native structured-output branch below because it governs + // the JSON prefill too, and a caller can supply an output_config without supplying any + // output schema for that branch to look at. + boolean callerSuppliedOutputConfig = + additionalKwargs != null && additionalKwargs.containsKey("output_config"); + + // Native structured output applies only for a POJO Class schema on a model Anthropic + // documents as capable; a RowTypeInfo (wrapped in OutputSchema) or an incapable model keeps + // the prompt-engineering fallback. A caller-supplied output_config is the caller being + // explicit about the exact parameter this branch writes, so it wins and the schema falls + // back to prompt engineering rather than the two competing on the same request. + // + // TODO(#912): the requested strategy is not visible here, so this re-check cannot tell an + // explicit NATIVE request apart from one that merely resolved to native. A caller asking + // for NATIVE on a model this predicate rejects therefore degrades silently to the + // prompt-engineering fallback instead of getting an error. Once strategy resolution is + // wired up, NATIVE must either bypass this capability re-check or fail explicitly. + boolean nativeSchemaApplied = false; + if (outputSchema instanceof Class + && supportsNativeStructuredOutput(modelName) + && !callerSuppliedOutputConfig) { + builder.outputConfig(toNativeOutputConfig((Class) outputSchema)); + nativeSchemaApplied = true; + } + + // JSON prefill appends a prefilled assistant "{" message to steer the model into emitting a + // JSON document. It applies only when the request carries none of three features: + // - tool use, because the prefill forces JSON text instead of native tool_use blocks; + // - structured outputs, which Anthropic documents as incompatible with message prefilling + // — output_config already has the provider enforcing the very document the prefill + // exists to coax out of the model; + // - a model that rejects prefilling outright, which answers with a 400 rather than a + // completion. + // The output_config test covers both ways one can reach the request: derived from + // outputSchema above, or supplied by the caller through additional_kwargs. It keys on what + // the request ends up carrying rather than on what was supplied, so a schema that could not + // be sent natively keeps the prefill its prompt-engineering fallback depends on — unless + // the caller supplied an output_config of its own. Object jsonPrefill = modelParams.remove("json_prefill"); boolean hasToolsInRequest = tools != null && !tools.isEmpty(); - if (Boolean.TRUE.equals(jsonPrefill) && !hasToolsInRequest) { + boolean requestCarriesOutputConfig = nativeSchemaApplied || callerSuppliedOutputConfig; + boolean jsonPrefillApplied = + Boolean.TRUE.equals(jsonPrefill) + && !hasToolsInRequest + && !requestCarriesOutputConfig + && supportsJsonPrefill(modelName); + if (jsonPrefillApplied) { anthropicMessages.add( MessageParam.builder().role(MessageParam.Role.ASSISTANT).content("{").build()); + // The builder copies the list it is given, so appending to the local list after the + // earlier messages(...) call is not enough - the list has to be handed over again. builder.messages(anthropicMessages); } - return builder.build(); + return new BuiltRequest(builder.build(), jsonPrefillApplied); + } + + /** A built request together with the JSON prefill decision applied while building it. */ + static final class BuiltRequest { + final MessageCreateParams params; + final boolean jsonPrefillApplied; + + BuiltRequest(MessageCreateParams params, boolean jsonPrefillApplied) { + this.params = params; + this.jsonPrefillApplied = jsonPrefillApplied; + } } private List extractSystemMessages(List messages) { @@ -362,7 +576,17 @@ private Tool.InputSchema parseToolInputSchema(String schemaJson) { } } - private ChatMessage convertResponse(Message response, boolean jsonPrefillApplied) { + /** + * Converts a response into a {@link ChatMessage}, reconstructing the leading {@code "{"} when + * the request carried the JSON prefill. + * + *

Takes the whole {@link BuiltRequest} rather than the prefill flag on its own so the flag + * travels with the request it was derived from, instead of being computed separately at the call + * site where the two can drift apart. A flag that disagrees with the request either prepends a + * stray {@code "{"} or drops a required one, and the resulting JSON is malformed in a way the + * response itself gives no sign of. + */ + ChatMessage convertResponse(BuiltRequest built, Message response) { List contentBlocks = response.content(); if (contentBlocks.isEmpty()) { throw new IllegalStateException("Anthropic response did not contain any content."); @@ -370,7 +594,7 @@ private ChatMessage convertResponse(Message response, boolean jsonPrefillApplied StringBuilder textContent = new StringBuilder(); // If JSON prefill was used, prepend "{" since the response only contains the continuation - if (jsonPrefillApplied) { + if (built.jsonPrefillApplied) { textContent.append("{"); } List> toolCalls = new ArrayList<>(); @@ -423,9 +647,10 @@ private ChatMessage convertResponse(Message response, boolean jsonPrefillApplied * Extracts JSON content from a string that may contain markdown code blocks. * *

Claude often wraps JSON responses in markdown code blocks like {@code ```json ... ```}, - * especially when tools are configured (since json_prefill is disabled). This method extracts - * the JSON content from such responses. If no code block is found, the original content is - * returned unchanged. + * especially on a response no JSON prefill was applied to, since an assistant turn already + * opened with {@code "{"} cannot be continued into a fence. This method extracts the JSON + * content from such responses. If no code block is found, the original content is returned + * unchanged. * * @param content The response content that may contain markdown-wrapped JSON * @return The extracted JSON string, or the original content if no code block is found diff --git a/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelSetup.java b/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelSetup.java index cf2a79358..b7636f8f8 100644 --- a/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelSetup.java +++ b/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelSetup.java @@ -41,11 +41,15 @@ *

  • temperature (optional): Sampling temperature 0.0-1.0 (default: 0.1) *
  • max_tokens (optional): Maximum tokens in response (default: 1024) *
  • json_prefill (optional): When true, prefills assistant response with "{" to enforce - * JSON output. Automatically disabled when tools are passed. (default: true) + * JSON output. Applies only on models Anthropic documents as accepting assistant-message + * prefilling, and is automatically disabled when tools are passed, or when the request + * carries an output_config, whether that was derived from an output schema or supplied + * through additional_kwargs. (default: false) *
  • strict_tools (optional): When true, tool calls adhere strictly to the JSON schema. * (default: false) *
  • tools (optional): List of tool names available for the model to use - *
  • additional_kwargs (optional): Additional parameters (top_k, top_p, stop_sequences) + *
  • additional_kwargs (optional): Additional parameters (top_k, top_p, stop_sequences). + * An output_config supplied here takes precedence over one derived from an output schema. * * *

    Example usage: @@ -74,7 +78,7 @@ public class AnthropicChatModelSetup extends BaseChatModelSetup { private static final String DEFAULT_MODEL = "claude-sonnet-4-20250514"; private static final double DEFAULT_TEMPERATURE = 0.1d; private static final long DEFAULT_MAX_TOKENS = 1024L; - private static final boolean DEFAULT_JSON_PREFILL = true; + private static final boolean DEFAULT_JSON_PREFILL = false; private static final boolean DEFAULT_STRICT_TOOLS = false; private final Double temperature; diff --git a/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java b/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java new file mode 100644 index 000000000..091b900b5 --- /dev/null +++ b/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java @@ -0,0 +1,609 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.agents.integrations.chatmodels.anthropic; + +import com.anthropic.models.messages.Message; +import com.anthropic.models.messages.MessageParam; +import com.anthropic.models.messages.Model; +import com.anthropic.models.messages.OutputConfig; +import com.anthropic.models.messages.TextBlock; +import com.anthropic.models.messages.Usage; +import com.fasterxml.jackson.core.type.TypeReference; +import com.fasterxml.jackson.databind.ObjectMapper; +import org.apache.flink.agents.api.chat.messages.ChatMessage; +import org.apache.flink.agents.api.chat.messages.MessageRole; +import org.apache.flink.agents.api.resource.ResourceContext; +import org.apache.flink.agents.api.resource.ResourceDescriptor; +import org.apache.flink.agents.api.tools.Tool; +import org.apache.flink.agents.api.tools.ToolMetadata; +import org.apache.flink.agents.api.tools.ToolParameters; +import org.apache.flink.agents.api.tools.ToolResponse; +import org.apache.flink.agents.api.tools.ToolType; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; +import org.junit.jupiter.params.provider.NullSource; +import org.junit.jupiter.params.provider.ValueSource; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; +import java.util.concurrent.atomic.AtomicReference; +import java.util.stream.Stream; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +/** + * Unit tests for {@link AnthropicChatModelConnection}'s request construction, its native + * structured-output capability check, and the response conversion that consumes the request. No + * test issues a request: they inspect what {@code buildRequest} produced, call the capability + * predicate directly, and feed {@code convertResponse} a hand-built response, so they need no + * credentials, no network, and no mocking framework. + */ +class AnthropicChatModelConnectionTest { + + private static final ResourceContext NOOP = ResourceContext.fromGetResource((a, b) -> null); + + private static final ObjectMapper MAPPER = new ObjectMapper(); + + private static final TypeReference> MAP_TYPE = new TypeReference<>() {}; + + /** The continuation an assistant returns after a "{" prefill, and the document it completes. */ + private static final String CONTINUATION = "\"ok\": true}"; + + private static final String COMPLETED = "{" + CONTINUATION; + + private static ResourceDescriptor descriptor(String model) { + return ResourceDescriptor.Builder.newBuilder(AnthropicChatModelConnection.class.getName()) + .addInitialArgument("api_key", "test-key") + .addInitialArgument("model", model) + .build(); + } + + private static AnthropicChatModelConnection connection() { + return new AnthropicChatModelConnection(descriptor("claude-sonnet-4-20250514"), NOOP); + } + + private static Map params(Object jsonPrefill) { + Map params = new HashMap<>(); + params.put("max_tokens", 256); + if (jsonPrefill != null) { + params.put("json_prefill", jsonPrefill); + } + return params; + } + + private static List userMessage() { + return List.of(new ChatMessage(MessageRole.USER, "hi")); + } + + /** An assistant response carrying a single text block. */ + private static Message textResponse(String text) { + Usage usage = + Usage.builder() + .inputTokens(1) + .outputTokens(1) + .cacheCreation(Optional.empty()) + .cacheCreationInputTokens(Optional.empty()) + .cacheReadInputTokens(Optional.empty()) + .serverToolUse(Optional.empty()) + .serviceTier(Optional.empty()) + .build(); + return Message.builder() + .id("msg_test") + .model(Model.of("claude-sonnet-4-20250514")) + .addContent(TextBlock.builder().text(text).citations(Optional.empty()).build()) + .stopReason(Optional.empty()) + .stopSequence(Optional.empty()) + .usage(usage) + .build(); + } + + /** True when the built request ends with the prefilled assistant "{" message. */ + private static boolean requestCarriesPrefill(AnthropicChatModelConnection.BuiltRequest built) { + List messages = built.params.messages(); + MessageParam last = messages.get(messages.size() - 1); + return last.role().equals(MessageParam.Role.ASSISTANT) + && last.content().string().isPresent() + && "{".equals(last.content().string().get()); + } + + /** + * Asserts that the recorded decision, the request content, and the converted response all agree + * with each other and with {@code expectedApplied}. + * + *

    The agreement is the invariant that matters: a decision that does not match what the + * request actually carries makes the conversion either prepend a stray {@code "{"} or drop a + * required one, yielding malformed JSON the response gives no sign of. + */ + private static void assertPrefillDecision( + Object jsonPrefill, List tools, boolean expectedApplied) { + AnthropicChatModelConnection connection = connection(); + AnthropicChatModelConnection.BuiltRequest built = + connection.buildRequest(userMessage(), tools, params(jsonPrefill), null); + + assertThat(built.jsonPrefillApplied).isEqualTo(expectedApplied); + assertThat(requestCarriesPrefill(built)).isEqualTo(expectedApplied); + assertThat(connection.convertResponse(built, textResponse(CONTINUATION)).getContent()) + .isEqualTo(expectedApplied ? COMPLETED : CONTINUATION); + } + + @Test + @DisplayName("json_prefill applied when requested with no tools") + void testPrefillAppliedWithoutTools() { + assertPrefillDecision(true, List.of(), true); + } + + @Test + @DisplayName("json_prefill not applied when tools are present") + void testPrefillNotAppliedWithTools() { + assertPrefillDecision(true, List.of(new StubTool()), false); + } + + @Test + @DisplayName("json_prefill not applied when the parameter is absent") + void testPrefillNotAppliedWhenAbsent() { + assertPrefillDecision(null, List.of(), false); + } + + @Test + @DisplayName("json_prefill not applied when the parameter is false") + void testPrefillNotAppliedWhenFalse() { + assertPrefillDecision(false, List.of(), false); + } + + @Test + @DisplayName("json_prefill applied when the tools list is null") + void testPrefillAppliedWithNullTools() { + assertPrefillDecision(true, null, true); + } + + @Test + @DisplayName("null model params are copied rather than dereferenced") + void testNullModelParamsAreCopied() { + // With no params there is no max_tokens either, so the SDK's own required-field check is + // the first thing that can fail. Reaching it at all is the assertion: a regression in the + // null handling would surface earlier, as a NullPointerException. + assertThatThrownBy(() -> connection().buildRequest(userMessage(), List.of(), null, null)) + .isInstanceOf(IllegalStateException.class); + } + + @Test + @DisplayName("request build failures surface as a wrapped RuntimeException") + void testBuildFailureIsWrapped() { + List messages = List.of(new ChatMessage(MessageRole.TOOL, "result")); + + assertThatThrownBy(() -> connection().chat(messages, List.of(), params(null))) + .isInstanceOf(RuntimeException.class) + .hasMessageContaining("Failed to call Anthropic messages API.") + .hasRootCauseInstanceOf(IllegalArgumentException.class); + } + + // --------------------------------------------------------------------------------------- + // Native structured output + // --------------------------------------------------------------------------------------- + + /** + * A model the provider documents native structured-output support for. + * + *

    Deliberately a 4.5-generation name, which is the only generation that is both + * structured-output capable and still accepts a JSON prefill. The output_config tests below + * assert that a prefill is suppressed; on a 4.6-or-later name the prefill capability guard + * would suppress it as well, so those assertions would hold even with the output_config + * suppression removed. + */ + private static final String CAPABLE_MODEL = "claude-sonnet-4-5"; + + /** The connection's own default model, which predates the structured-output cutoff. */ + private static final String INCAPABLE_MODEL = "claude-sonnet-4-20250514"; + + /** + * The models the provider documents native structured-output support for, in the order the + * connection lists them: the exact-matched names first, then the prefix-matched aliases. + * Mirroring that order keeps the two lists comparable side by side, so a name added to one and + * not the other stands out. + */ + private static Stream capableModels() { + return Stream.of( + "claude-opus-4-6", + "claude-opus-4-7", + "claude-opus-4-8", + "claude-opus-5", + "claude-sonnet-4-6", + "claude-sonnet-5", + "claude-fable-5", + "claude-mythos-5", + "claude-mythos-preview", + "claude-opus-4-5", + "claude-sonnet-4-5", + "claude-haiku-4-5"); + } + + /** + * Names that must not be treated as capable. {@code claude-opus-4-1-20250805} and {@code + * claude-opus-4} are the reason the alias prefixes retain their minor version: truncating + * {@code claude-opus-4-5} to {@code claude-opus-4} would admit both. + * + *

    The empty name is reachable — a blank configured model survives the blank-check in the + * request builder and arrives here unchanged — and it is the shortest name the predicate has to + * answer for, so a rewrite that indexes into the name rather than matching it whole breaks on + * it. + */ + private static Stream incapableModels() { + return Stream.of( + "claude-opus-4-1-20250805", + "claude-opus-4", + "claude-sonnet-4-20250514", + "claude-3-5-sonnet-latest", + ""); + } + + /** A POJO the SDK can derive a JSON schema from. */ + public static class Answer { + public String verdict; + } + + private static Map paramsWithModel(String model, Object jsonPrefill) { + Map params = params(jsonPrefill); + params.put("model", model); + return params; + } + + private static AnthropicChatModelConnection.BuiltRequest build( + String model, Object outputSchema, Object jsonPrefill) { + return connection() + .buildRequest( + userMessage(), + List.of(), + paramsWithModel(model, jsonPrefill), + outputSchema); + } + + /** The property names of the JSON schema the request carries, or empty when it carries none. */ + private static Set nativeSchemaProperties( + AnthropicChatModelConnection.BuiltRequest built) { + return built.params + .outputConfig() + .flatMap(OutputConfig::format) + .map(format -> format.schema()._additionalProperties()) + .map(schema -> schema.get("properties")) + .map(properties -> MAPPER.convertValue(properties, MAP_TYPE).keySet()) + .orElse(Set.of()); + } + + @ParameterizedTest + @ValueSource(strings = {"claude-sonnet-4-5", "claude-opus-4-6"}) + @DisplayName("a POJO schema on a capable model is sent as output_config") + void testNativeSchemaAppliedOnCapableModel(String model) { + // One name from each way the capability check can match: a 4.5-generation alias reached by + // prefix, and a 4.6 name reached by exact match. The request-build site consults the check + // as a whole, so covering only one branch would let it be narrowed to that branch while + // silently dropping native structured output for every model on the other. + AnthropicChatModelConnection.BuiltRequest built = build(model, Answer.class, null); + + // Asserting the property name rather than mere presence: an output_config built from the + // wrong class, or from an empty placeholder, would still be present. + assertThat(nativeSchemaProperties(built)).containsExactly("verdict"); + } + + @Test + @DisplayName("a POJO schema on an incapable model keeps the prompt fallback") + void testNativeSchemaNotAppliedOnIncapableModel() { + assertThat(build(INCAPABLE_MODEL, Answer.class, null).params.outputConfig()).isEmpty(); + } + + @Test + @DisplayName("no output_config is sent when no schema is supplied") + void testNativeSchemaNotAppliedWithoutSchema() { + assertThat(build(CAPABLE_MODEL, null, null).params.outputConfig()).isEmpty(); + } + + @Test + @DisplayName("a schema that is not a Class keeps the prompt fallback") + void testNonClassSchemaKeepsFallback() { + // The RowTypeInfo case arrives wrapped rather than as a Class; anything but a Class has no + // native translation and must degrade rather than fail. + assertThat(build(CAPABLE_MODEL, "not-a-class", null).params.outputConfig()).isEmpty(); + } + + @Test + @DisplayName("the native path adds no anthropic-beta header") + void testNativePathSendsNoBetaHeader() { + // Structured outputs are generally available; the beta header the neighbouring strict_tools + // path sends must not leak onto this one. + assertThat(build(CAPABLE_MODEL, Answer.class, null).params._additionalHeaders().names()) + .doesNotContain("anthropic-beta"); + } + + @ParameterizedTest + @MethodSource("capableModels") + @DisplayName("every documented model reports capable") + void testCapableModelsReportCapable(String model) { + assertThat(connection().supportsNativeStructuredOutput(model)).isTrue(); + } + + @ParameterizedTest + @NullSource + @MethodSource("incapableModels") + @DisplayName("an undocumented model reports not capable") + void testIncapableModelsReportNotCapable(String model) { + assertThat(connection().supportsNativeStructuredOutput(model)).isFalse(); + } + + @Test + @DisplayName("an alias prefix also matches the dated snapshot behind it") + void testAliasPrefixMatchesDatedSnapshot() { + // The three 4.5-generation names are aliases, so a request may carry the snapshot instead. + // Converting the prefixes to exact matches would still satisfy the capable-models test. + assertThat(connection().supportsNativeStructuredOutput("claude-sonnet-4-5-20250929")) + .isTrue(); + } + + @Test + @DisplayName("an alias prefix does not match a longer minor version") + void testAliasPrefixDoesNotMatchLongerMinorVersion() { + // A dated snapshot continues the alias with a "-" separator. A name that extends the + // alias without one is a different minor version, whose capability is not the alias's to + // answer for. + assertThat(connection().supportsNativeStructuredOutput("claude-sonnet-4-50")).isFalse(); + } + + @Test + @DisplayName("capability does not depend on the connection's configured model") + void testCapabilityReadsNoInstanceState() { + AnthropicChatModelConnection configuredCapable = + new AnthropicChatModelConnection(descriptor(CAPABLE_MODEL), NOOP); + + // connection() is configured with an incapable default. Both must answer for the argument + // alone, so a predicate that consulted the configured model would disagree with itself. + assertThat(configuredCapable.supportsNativeStructuredOutput(INCAPABLE_MODEL)) + .isEqualTo(connection().supportsNativeStructuredOutput(INCAPABLE_MODEL)); + assertThat(configuredCapable.supportsNativeStructuredOutput(CAPABLE_MODEL)) + .isEqualTo(connection().supportsNativeStructuredOutput(CAPABLE_MODEL)); + } + + @Test + @DisplayName("a caller-supplied output_config wins and the schema falls back") + void testCallerOutputConfigWinsOverSchema() { + Map params = paramsWithModel(CAPABLE_MODEL, true); + params.put("additional_kwargs", Map.of("output_config", Map.of("format", Map.of()))); + + AnthropicChatModelConnection.BuiltRequest built = + connection().buildRequest(userMessage(), List.of(), params, Answer.class); + + assertThat(built.params.outputConfig()).isEmpty(); + assertThat(built.params._additionalBodyProperties()).containsKey("output_config"); + // The request still carries an output_config, just the caller's, and a prefill alongside + // one is a combination the provider documents as unsupported. + assertThat(built.jsonPrefillApplied).isFalse(); + assertThat(requestCarriesPrefill(built)).isFalse(); + } + + @Test + @DisplayName("a caller-supplied output_config suppresses json_prefill with no schema supplied") + void testCallerOutputConfigSuppressesPrefillWithoutSchema() { + // No output schema, so nothing derives an output_config and the caller's is the only one on + // the request. Asserted on both a capable and an incapable model because suppression has to + // follow the config the request carries rather than the model's structured-output + // capability, which a check folded in beside the capability test would get wrong. + for (String model : List.of(CAPABLE_MODEL, INCAPABLE_MODEL)) { + Map params = paramsWithModel(model, true); + params.put("additional_kwargs", Map.of("output_config", Map.of("format", Map.of()))); + + AnthropicChatModelConnection.BuiltRequest built = + connection().buildRequest(userMessage(), List.of(), params, null); + + assertThat(built.jsonPrefillApplied).as(model).isFalse(); + assertThat(requestCarriesPrefill(built)).as(model).isFalse(); + } + } + + @Test + @DisplayName("the three-argument chat forwards its arguments and no output schema") + void testThreeArgChatForwardsNoSchema() { + // Existing framework callers reach the connection through the three-argument overload. It + // has to keep forwarding no schema: a non-null argument here would switch every capable + // model to native structured output without any caller asking for it. The other three + // arguments have to arrive unchanged, since dropping any of them is silent - the request + // still builds, just without the caller's tools or model parameters. Overriding the + // four-argument overload is what makes the forwarded values observable without a network + // call, since the real one issues the request. + Object notCalled = new Object(); + AtomicReference forwardedSchema = new AtomicReference<>(notCalled); + AtomicReference> forwardedMessages = new AtomicReference<>(); + AtomicReference> forwardedTools = new AtomicReference<>(); + AtomicReference> forwardedParams = new AtomicReference<>(); + AnthropicChatModelConnection connection = + new AnthropicChatModelConnection(descriptor(CAPABLE_MODEL), NOOP) { + @Override + public ChatMessage chat( + List messages, + List tools, + Map modelParams, + Object outputSchema) { + forwardedSchema.set(outputSchema); + forwardedMessages.set(messages); + forwardedTools.set(tools); + forwardedParams.set(modelParams); + return ChatMessage.assistant(""); + } + }; + + List messages = userMessage(); + List tools = List.of(new StubTool()); + Map modelParams = params(null); + connection.chat(messages, tools, modelParams); + + assertThat(forwardedSchema.get()).isNotSameAs(notCalled); + assertThat(forwardedSchema.get()).isNull(); + assertThat(forwardedMessages.get()).isSameAs(messages); + assertThat(forwardedTools.get()).isSameAs(tools); + assertThat(forwardedParams.get()).isSameAs(modelParams); + } + + @Test + @DisplayName("json_prefill is suppressed when the schema is applied natively") + void testJsonPrefillSuppressedWhenNativeApplies() { + AnthropicChatModelConnection connection = connection(); + AnthropicChatModelConnection.BuiltRequest built = + connection.buildRequest( + userMessage(), + List.of(), + paramsWithModel(CAPABLE_MODEL, true), + Answer.class); + + assertThat(built.jsonPrefillApplied).isFalse(); + assertThat(requestCarriesPrefill(built)).isFalse(); + // The provider returns a complete document, so nothing may be prepended to it. + assertThat(connection.convertResponse(built, textResponse(COMPLETED)).getContent()) + .isEqualTo(COMPLETED); + } + + @Test + @DisplayName("json_prefill survives when a schema falls back to prompt engineering") + void testJsonPrefillAppliedWhenSchemaFallsBack() { + // Suppression keys on whether the schema was applied, not on whether one was supplied. + // Keying it on the schema instead would strip the prefill the fallback still depends on. + AnthropicChatModelConnection connection = connection(); + AnthropicChatModelConnection.BuiltRequest built = + connection.buildRequest( + userMessage(), + List.of(), + paramsWithModel(INCAPABLE_MODEL, true), + Answer.class); + + assertThat(built.jsonPrefillApplied).isTrue(); + assertThat(requestCarriesPrefill(built)).isTrue(); + assertThat(connection.convertResponse(built, textResponse(CONTINUATION)).getContent()) + .isEqualTo(COMPLETED); + } + + // --------------------------------------------------------------------------------------- + // JSON prefill model capability + // --------------------------------------------------------------------------------------- + + /** + * The models the provider documents as rejecting assistant-message prefilling, in the order the + * connection lists them. Mirroring that order keeps the two lists comparable side by side, so a + * name added to one and not the other stands out. + */ + private static Stream prefillUnsupportedModels() { + return Stream.of( + "claude-opus-4-6", + "claude-opus-4-7", + "claude-opus-4-8", + "claude-opus-5", + "claude-sonnet-4-6", + "claude-sonnet-5", + "claude-fable-5", + "claude-mythos-5", + "claude-mythos-preview"); + } + + /** + * Names that accept a prefill. The three 4.5-generation names are the load-bearing ones: they + * are documented as structured-output capable, so folding the two rules onto one list would + * silently withdraw the prefill from exactly these models. + * + *

    {@code claude-sonnet-4-5-20250929} is the dated snapshot behind one of those aliases, and + * {@code claude-3-5-sonnet-latest} stands for every name the list does not mention, which keeps + * the prefill because only the listed names withdraw it. + */ + private static Stream prefillSupportedModels() { + return Stream.of( + "claude-opus-4-5", + "claude-sonnet-4-5", + "claude-haiku-4-5", + "claude-sonnet-4-5-20250929", + "claude-sonnet-4-20250514", + "claude-3-5-sonnet-latest", + ""); + } + + /** + * Asserts the prefill decision, the request content and the converted response for a request + * that asks for the prefill on {@code model} and gives the decision no other reason to go + * either way: no tools and no output configuration. + */ + private static void assertPrefillDecisionForModel(String model, boolean expectedApplied) { + AnthropicChatModelConnection connection = connection(); + AnthropicChatModelConnection.BuiltRequest built = + connection.buildRequest( + userMessage(), List.of(), paramsWithModel(model, true), null); + + assertThat(built.jsonPrefillApplied).isEqualTo(expectedApplied); + assertThat(requestCarriesPrefill(built)).isEqualTo(expectedApplied); + assertThat(connection.convertResponse(built, textResponse(CONTINUATION)).getContent()) + .isEqualTo(expectedApplied ? COMPLETED : CONTINUATION); + } + + @ParameterizedTest + @MethodSource("prefillUnsupportedModels") + @DisplayName("every model documented as rejecting prefilling reports unsupported") + void testPrefillUnsupportedModelsReportUnsupported(String model) { + assertThat(AnthropicChatModelConnection.supportsJsonPrefill(model)).isFalse(); + } + + @ParameterizedTest + @NullSource + @MethodSource("prefillSupportedModels") + @DisplayName("a model outside that list reports prefill supported") + void testPrefillSupportedModelsReportSupported(String model) { + assertThat(AnthropicChatModelConnection.supportsJsonPrefill(model)).isTrue(); + } + + @Test + @DisplayName("json_prefill is suppressed on a model that rejects prefilling") + void testPrefillSuppressedOnUnsupportedModel() { + assertPrefillDecisionForModel("claude-opus-4-6", false); + } + + @Test + @DisplayName("json_prefill is applied on a structured-output capable model that accepts it") + void testPrefillAppliedOnStructuredOutputCapableModel() { + // The two capability rules draw different lines, and this model sits between them: the + // provider documents structured-output support from the 4.5 generation on but withdraws + // prefilling only from 4.6 on. Deriving the prefill rule from the structured-output + // allowlists would strip the prefill here, where the provider still accepts it. + assertThat(connection().supportsNativeStructuredOutput("claude-sonnet-4-5")).isTrue(); + + assertPrefillDecisionForModel("claude-sonnet-4-5", true); + } + + /** Minimal tool stub; only its presence in the tools list matters. */ + private static class StubTool extends Tool { + StubTool() { + super(new ToolMetadata("add", "adds", "{\"type\":\"object\"}")); + } + + @Override + public ToolType getToolType() { + return ToolType.FUNCTION; + } + + @Override + public ToolResponse call(ToolParameters parameters) { + return ToolResponse.success(null); + } + } +} diff --git a/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelSetupTest.java b/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelSetupTest.java new file mode 100644 index 000000000..8cd126ab3 --- /dev/null +++ b/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelSetupTest.java @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.agents.integrations.chatmodels.anthropic; + +import org.apache.flink.agents.api.resource.ResourceContext; +import org.apache.flink.agents.api.resource.ResourceDescriptor; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; + +import java.util.Map; + +import static org.assertj.core.api.Assertions.assertThat; + +/** Tests for {@link AnthropicChatModelSetup}. */ +class AnthropicChatModelSetupTest { + + private static final ResourceContext NOOP = ResourceContext.fromGetResource((a, b) -> null); + + private static ResourceDescriptor.Builder base() { + return ResourceDescriptor.Builder.newBuilder(AnthropicChatModelSetup.class.getName()) + .addInitialArgument("connection", "conn"); + } + + @Test + @DisplayName("getParameters applies the documented defaults") + void testGetParametersDefaults() { + Map params = + new AnthropicChatModelSetup(base().build(), NOOP).getParameters(); + + assertThat(params).containsEntry("model", "claude-sonnet-4-20250514"); + assertThat(params).containsEntry("temperature", 0.1d); + assertThat(params).containsEntry("max_tokens", 1024L); + assertThat(params).containsEntry("strict_tools", false); + // json_prefill is opt-in: it steers the model with a technique several models reject + // outright, so a setup that does not ask for it must not send it. + assertThat(params).containsEntry("json_prefill", false); + } + + @Test + @DisplayName("getParameters honors an explicit json_prefill") + void testGetParametersHonorsExplicitJsonPrefill() { + // Pins that the argument is read rather than the default being emitted unconditionally. + Map params = + new AnthropicChatModelSetup( + base().addInitialArgument("json_prefill", true).build(), NOOP) + .getParameters(); + + assertThat(params).containsEntry("json_prefill", true); + } +} diff --git a/integrations/pom.xml b/integrations/pom.xml index bbd51c55f..8c7b4cbae 100644 --- a/integrations/pom.xml +++ b/integrations/pom.xml @@ -35,7 +35,7 @@ under the License. 8.19.0 2.6.18 4.8.0 - 2.11.1 + 2.12.0 2.32.16 1.56.0 diff --git a/plan/src/main/java/org/apache/flink/agents/plan/tools/SchemaUtils.java b/plan/src/main/java/org/apache/flink/agents/plan/tools/SchemaUtils.java index 87ee67d30..b44760f8a 100644 --- a/plan/src/main/java/org/apache/flink/agents/plan/tools/SchemaUtils.java +++ b/plan/src/main/java/org/apache/flink/agents/plan/tools/SchemaUtils.java @@ -102,9 +102,19 @@ private static Map getParamSchema(Parameter param) { if (paramType == String.class) { paramSchema.put("type", "string"); - } else if (paramType == int.class || paramType == Integer.class) { + } else if (paramType == int.class + || paramType == Integer.class + || paramType == long.class + || paramType == Long.class + || paramType == short.class + || paramType == Short.class + || paramType == byte.class + || paramType == Byte.class) { paramSchema.put("type", "integer"); - } else if (paramType == double.class || paramType == Double.class) { + } else if (paramType == double.class + || paramType == Double.class + || paramType == float.class + || paramType == Float.class) { paramSchema.put("type", "number"); } else if (paramType == boolean.class || paramType == Boolean.class) { paramSchema.put("type", "boolean"); diff --git a/plan/src/test/java/org/apache/flink/agents/plan/tools/SchemaUtilsTest.java b/plan/src/test/java/org/apache/flink/agents/plan/tools/SchemaUtilsTest.java index 1792a12df..a511e405a 100644 --- a/plan/src/test/java/org/apache/flink/agents/plan/tools/SchemaUtilsTest.java +++ b/plan/src/test/java/org/apache/flink/agents/plan/tools/SchemaUtilsTest.java @@ -44,6 +44,15 @@ public void methodWithBasicTypes( public void methodWithoutAnnotations(String param1, int param2) {} + public void methodWithWideNumericTypes( + @ToolParam(name = "longParam", description = "A long parameter") long longParam, + @ToolParam(name = "boxedLongParam", description = "A boxed long") Long boxedLong, + @ToolParam(name = "floatParam", description = "A float parameter") float floatParam, + @ToolParam(name = "boxedFloatParam", description = "A boxed float") + Float boxedFloat, + @ToolParam(name = "shortParam", description = "A short parameter") short shortParam, + @ToolParam(name = "byteParam", description = "A byte parameter") byte byteParam) {} + public void methodWithCustomObject( @ToolParam(name = "objectParam", description = "A custom object parameter") Object customObject) {} @@ -55,6 +64,34 @@ public void methodWithExternallyInjectedRequiredParam( private final ObjectMapper mapper = new ObjectMapper(); + @Test + void testGenerateSchemaWithWideNumericTypes() throws Exception { + Method method = + TestClass.class.getMethod( + "methodWithWideNumericTypes", + long.class, + Long.class, + float.class, + Float.class, + short.class, + byte.class); + String schema = SchemaUtils.generateSchema(method); + final JsonNode jsonNode = mapper.readTree(schema); + JsonNode properties = jsonNode.get("properties"); + + // Integral types map to "integer" (#1015): previously these all fell back to "object", + // which models either follow (producing object-shaped arguments) or treat as an + // uncallable tool. + assertEquals("integer", properties.get("longParam").get("type").asText()); + assertEquals("integer", properties.get("boxedLongParam").get("type").asText()); + assertEquals("integer", properties.get("shortParam").get("type").asText()); + assertEquals("integer", properties.get("byteParam").get("type").asText()); + + // Floating-point types map to "number". + assertEquals("number", properties.get("floatParam").get("type").asText()); + assertEquals("number", properties.get("boxedFloatParam").get("type").asText()); + } + @Test void testGenerateSchemaWithBasicTypes() throws Exception { Method method = diff --git a/python/flink_agents/integrations/chat_models/anthropic/anthropic_chat_model.py b/python/flink_agents/integrations/chat_models/anthropic/anthropic_chat_model.py index f816a2890..2cf1d8da1 100644 --- a/python/flink_agents/integrations/chat_models/anthropic/anthropic_chat_model.py +++ b/python/flink_agents/integrations/chat_models/anthropic/anthropic_chat_model.py @@ -18,10 +18,10 @@ import uuid from typing import Any, Dict, List, Sequence -from anthropic import Anthropic +from anthropic import Anthropic, transform_schema from anthropic._types import NOT_GIVEN from anthropic.types import MessageParam, TextBlockParam, ToolParam -from pydantic import Field, PrivateAttr +from pydantic import BaseModel, Field, PrivateAttr from typing_extensions import override from flink_agents.api.agents.types import OutputSchema @@ -111,6 +111,113 @@ def convert_to_anthropic_system_prompts( ] +# Models Anthropic documents native structured-output support for. Source of truth: +# https://platform.claude.com/docs/en/build-with-claude/structured-outputs +# +# The documented rule is generational rather than a per-snapshot list: structured +# outputs are generally available for Claude 4.5 and later models, and for Claude Mythos +# Preview. Names from the 4.6 generation onward carry no date and are pinned, so the +# name is itself the snapshot and is matched exactly. +# +# The three 4.5-generation names are aliases that front a dated snapshot, so a request +# may carry either the alias or the snapshot behind it and both have to match. Those +# match the alias itself or a name continuing with a "-" separator, which covers +# claude-sonnet-4-5-20250929. A name that extends the alias without that separator is +# a different minor version and is capable only if the exact set names it. The alias +# also has to retain the minor version: "claude-opus-4" would capture +# claude-opus-4-1-20250805, which predates the cutoff and is not capable. +# +# A name outside both sets reports not-capable and degrades to the prompt-engineering +# fallback rather than failing at the provider. +_NATIVE_STRUCTURED_OUTPUT_MODELS = frozenset( + { + "claude-opus-4-6", + "claude-opus-4-7", + "claude-opus-4-8", + "claude-opus-5", + "claude-sonnet-4-6", + "claude-sonnet-5", + "claude-fable-5", + "claude-mythos-5", + "claude-mythos-preview", + } +) + +_NATIVE_STRUCTURED_OUTPUT_ALIAS_PREFIXES = ( + "claude-opus-4-5", + "claude-sonnet-4-5", + "claude-haiku-4-5", +) + + +# Models Anthropic documents as rejecting assistant-message prefilling. Source of truth: +# https://platform.claude.com/docs/en/build-with-claude/working-with-messages#putting-words-in-claudes-mouth +# +# Prefilling is not supported from the Claude 4.6 generation onward, nor on Claude +# Mythos Preview, Claude Fable 5 or Claude Mythos 5; a request that prefills one of +# them is answered with a 400 rather than a completion. Anthropic publishes no +# programmatic signal for prefill support the way it does for structured outputs, so +# the rule has to be a maintained list of names. Those names carry no date and are +# pinned, so the name is itself the snapshot and is matched exactly, and a name outside +# the list is treated as accepting the prefill. +# +# Kept in its own storage rather than derived from the structured-output allowlists +# above, whose contents it currently coincides with. The two encode different +# documented boundaries: structured output starts at the 4.5 generation while prefill +# rejection starts at 4.6, so the three 4.5-generation names are structured-output +# capable and still accept a prefill. Sharing one list would hold only until a model +# moves one boundary without moving the other. +_PREFILL_UNSUPPORTED_MODELS = frozenset( + { + "claude-opus-4-6", + "claude-opus-4-7", + "claude-opus-4-8", + "claude-opus-5", + "claude-sonnet-4-6", + "claude-sonnet-5", + "claude-fable-5", + "claude-mythos-5", + "claude-mythos-preview", + } +) + + +def _supports_json_prefill(effective_model: str | None) -> bool: + """Whether ``effective_model`` accepts the prefilled assistant ``"{"`` message. + + See the list above for the source of truth and for why it is matched exactly and + kept apart from the structured-output allowlists. An unrecognized name reports + ``True``, which matches the documented rule: prefilling is the long-standing + behaviour and only the listed names withdraw it. The cost of that default runs the + opposite way to ``supports_native_structured_output``: a rejecting model this list + has not caught up with is prefilled and answered with a 400, where an unrecognized + name on the structured-output path degrades silently to the prompt-engineering + fallback instead. + """ + return effective_model not in _PREFILL_UNSUPPORTED_MODELS + + +def _native_output_config(output_schema: Any) -> Dict[str, Any] | None: + """Build the Anthropic ``output_config`` for a native structured-output request. + + Returns ``None`` (leaving the request unchanged) unless the schema is a + ``BaseModel`` subclass. A ``RowTypeInfo`` schema is skipped so it keeps the + prompt-engineering fallback. + + Anthropic's format object carries only the schema and its type, so it shares no + shape with the providers that nest the schema under a named, strict + ``json_schema`` object and is built here rather than in a shared helper. + """ + if output_schema is None: + return None + model = ( + output_schema.output_schema if isinstance(output_schema, OutputSchema) else None + ) + if not (isinstance(model, type) and issubclass(model, BaseModel)): + return None + return {"format": {"type": "json_schema", "schema": transform_schema(model)}} + + class AnthropicChatModelConnection(BaseChatModelConnection): """Manages the connection to the Anthropic AI models for chat interactions. @@ -165,6 +272,25 @@ def client(self) -> Anthropic: ) return self._client + @override + def supports_native_structured_output(self, effective_model: str | None) -> bool: + """Whether Anthropic documents structured output for ``effective_model``. + + See the module-level allowlists for the source of truth and for why a + 4.5-generation alias also matches the dated snapshot behind it while every other + name is matched exactly. A name outside both reports ``False`` so it degrades to + the prompt-engineering fallback rather than failing at the provider. + + Reads no instance state, so capability stays answerable independently of how + the connection was configured. + """ + if not effective_model: + return False + return effective_model in _NATIVE_STRUCTURED_OUTPUT_MODELS or any( + effective_model == prefix or effective_model.startswith(prefix + "-") + for prefix in _NATIVE_STRUCTURED_OUTPUT_ALIAS_PREFIXES + ) + def chat( self, messages: Sequence[ChatMessage], @@ -174,12 +300,29 @@ def chat( ) -> ChatMessage: """Direct communication with Anthropic model service for chat conversation. - A non-``None`` ``output_schema`` is rejected: this connection has no native - structured-output translation, so callers stay on the prompt-engineering - fallback. Declaring the parameter keeps a caller-supplied schema out of - ``**kwargs``, which is forwarded to the provider SDK. + Parameters + ---------- + messages : Sequence[ChatMessage] + Input message sequence + tools : Optional[List] + List of tools that can be called by the model + output_schema : OutputSchema | None + The schema the response should conform to, or ``None`` for an unconstrained + response. Native structured output is applied only for a ``BaseModel`` + schema on a model the provider documents as capable, and only when the + caller has not already supplied ``output_config``. Any other combination + sends no derived schema and keeps the prompt-engineering fallback. + **kwargs : Any + Additional parameters passed to the model service (e.g., temperature, + max_tokens, etc.). ``json_prefill`` is consumed here rather than + forwarded: it selects the prefilled assistant ``"{"`` message described + below and is not a request field the provider accepts. + + Returns: + ------- + ChatMessage + Model response message """ - self._reject_unsupported_output_schema(output_schema) anthropic_tools = None if tools is not None: anthropic_tools = [ @@ -189,6 +332,55 @@ def chat( anthropic_system = convert_to_anthropic_system_prompts(messages) anthropic_messages = convert_to_anthropic_messages(messages) + # Removed from kwargs unconditionally: it is a framework parameter, and leaving + # it in place would reach messages.create as an unknown request field. + json_prefill = kwargs.pop("json_prefill", False) + + # TODO(#912): the requested strategy is not visible here, so this check + # cannot tell an explicit NATIVE request apart from one that merely + # resolved to native. A caller asking for NATIVE on a model this + # predicate rejects therefore degrades silently to the prompt-engineering + # fallback instead of getting an error. Once strategy resolution is wired + # up, NATIVE must either bypass this capability check or fail explicitly. + if output_schema is not None and self.supports_native_structured_output( + kwargs.get("model") + ): + output_config = _native_output_config(output_schema) + # An output_config already in kwargs is the caller being explicit about the + # exact parameter this branch writes, so it is left alone and the schema + # keeps the prompt-engineering fallback. Writing over it would drop the + # caller's value with no error and no other trace. + if output_config is not None and "output_config" not in kwargs: + kwargs["output_config"] = output_config + + # JSON prefill appends a prefilled assistant "{" message to steer the model + # into emitting a JSON document. It applies only when the request carries none + # of three features: + # - tool use, because the prefill forces JSON text instead of native tool_use + # blocks; + # - structured outputs, which Anthropic documents as incompatible with message + # prefilling - output_config already has the provider enforcing the very + # document the prefill exists to coax out of the model; + # - a model that rejects prefilling outright, which answers with a 400 rather + # than a completion. + # Evaluated after the block above so the output_config test covers both ways one + # can reach the request: derived from output_schema there, or supplied by the + # caller. It keys on what the request ends up carrying rather than on what was + # supplied, so a schema that could not be sent natively keeps the prefill its + # prompt-engineering fallback depends on - unless the caller supplied an + # output_config of its own. + prefill_applied = ( + json_prefill is True + and not anthropic_tools + and "output_config" not in kwargs + and _supports_json_prefill(kwargs.get("model")) + ) + if prefill_applied: + anthropic_messages = [ + *anthropic_messages, + {"role": MessageRole.ASSISTANT.value, "content": "{"}, + ] + message = self.client.messages.create( messages=anthropic_messages, tools=anthropic_tools or NOT_GIVEN, @@ -211,6 +403,13 @@ def chat( (block.text for block in message.content if block.type == "text"), "" ) + # The response continues the prefilled "{" rather than repeating it, so the + # document is only complete once it is put back. Keyed on the decision actually + # applied above: reconstructing on any other signal either prepends a stray "{" + # or drops a required one, and the response itself gives no sign of either. + if prefill_applied: + text = "{" + text + if message.stop_reason == "tool_use": tool_calls = [ { @@ -254,6 +453,7 @@ def close(self) -> None: DEFAULT_ANTHROPIC_MODEL = "claude-sonnet-4-20250514" DEFAULT_MAX_TOKENS = 1024 DEFAULT_TEMPERATURE = 0.1 +DEFAULT_JSON_PREFILL = False class AnthropicChatModelSetup(BaseChatModelSetup): @@ -274,6 +474,12 @@ class AnthropicChatModelSetup(BaseChatModelSetup): The maximum number of tokens to generate before stopping. Defaults to 1024. temperature : float Amount of randomness injected into the response. + json_prefill : bool + When True, prefills the assistant response with "{" to enforce JSON output. + Applies only on models Anthropic documents as accepting assistant-message + prefilling, and is automatically disabled when tools are passed, or when the + request carries an output_config, whether that was derived from an output + schema or supplied by the caller. Defaults to False. """ max_tokens: int = Field( @@ -287,6 +493,13 @@ class AnthropicChatModelSetup(BaseChatModelSetup): ge=0.0, le=1.0, ) + json_prefill: bool = Field( + default=DEFAULT_JSON_PREFILL, + description=( + 'When True, prefills the assistant response with "{" to enforce JSON ' + "output. Defaults to False." + ), + ) def __init__( self, @@ -294,6 +507,8 @@ def __init__( model: str = DEFAULT_ANTHROPIC_MODEL, max_tokens: int = DEFAULT_MAX_TOKENS, temperature: float = DEFAULT_TEMPERATURE, + *, + json_prefill: bool = DEFAULT_JSON_PREFILL, **kwargs: Any, ) -> None: """Init method.""" @@ -302,6 +517,7 @@ def __init__( model=model, max_tokens=max_tokens, temperature=temperature, + json_prefill=json_prefill, **kwargs, ) @@ -312,4 +528,5 @@ def model_kwargs(self) -> Dict[str, Any]: "model": self.model, "max_tokens": self.max_tokens, "temperature": self.temperature, + "json_prefill": self.json_prefill, } diff --git a/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py b/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py index f11889e4e..90d3be815 100644 --- a/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py +++ b/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py @@ -15,18 +15,30 @@ # See the License for the specific language governing permissions and # limitations under the License. ################################################################################# +from typing import Any, Dict from unittest.mock import MagicMock +import pytest from anthropic.types import Message, TextBlock, ToolUseBlock, Usage +from pydantic import BaseModel +from pyflink.common.typeinfo import Types +from flink_agents.api.agents.types import OutputSchema from flink_agents.api.chat_message import ChatMessage, MessageRole +from flink_agents.api.tools.tool import Tool, ToolMetadata, ToolType from flink_agents.integrations.chat_models.anthropic.anthropic_chat_model import ( AnthropicChatModelConnection, + AnthropicChatModelSetup, + _supports_json_prefill, ) +def _connection() -> AnthropicChatModelConnection: + return AnthropicChatModelConnection(name="test", api_key="dummy") + + def _connection_returning(message: Message) -> AnthropicChatModelConnection: - connection = AnthropicChatModelConnection(name="test", api_key="dummy") + connection = _connection() client = MagicMock() client.messages.create.return_value = message connection._client = client @@ -137,3 +149,370 @@ def test_tool_use_response_keeps_token_usage() -> None: ) assert response.extra_args["promptTokens"] == 7 assert response.extra_args["completionTokens"] == 3 + + +# --------------------------------------------------------------------------------- +# Native structured output +# --------------------------------------------------------------------------------- + + +class _Answer(BaseModel): + """A representative BaseModel output schema.""" + + verdict: str + + +# A model the provider documents native structured-output support for. +# +# Deliberately a 4.5-generation name, which is the only generation that is both +# structured-output capable and still accepts a JSON prefill. The prefill tests below +# assert that an output_config suppresses the prefill; on a 4.6-or-later name the +# prefill capability guard would suppress it as well, so those assertions would hold +# even with the output_config suppression removed. +_CAPABLE_MODEL = "claude-sonnet-4-5" + +# The default model this integration ships with, which predates the cutoff. +_INCAPABLE_MODEL = "claude-sonnet-4-20250514" + +# The models the provider documents native structured-output support for, in the order +# the connection lists them: the exact-matched names first, then the prefix-matched +# aliases. The names are written out here rather than read from the connection, so that +# a name mistyped there is a disagreement between two lists rather than a value both +# sides share. +_CAPABLE_MODELS = [ + "claude-opus-4-6", + "claude-opus-4-7", + "claude-opus-4-8", + "claude-opus-5", + "claude-sonnet-4-6", + "claude-sonnet-5", + "claude-fable-5", + "claude-mythos-5", + "claude-mythos-preview", + "claude-opus-4-5", + "claude-sonnet-4-5", + "claude-haiku-4-5", +] + +# Names that must not be treated as capable. claude-opus-4-1-20250805 and claude-opus-4 +# are the reason the alias prefixes retain their minor version: truncating +# claude-opus-4-5 to claude-opus-4 would admit both. +_INCAPABLE_MODELS = [ + "claude-opus-4-1-20250805", + "claude-opus-4", + "claude-sonnet-4-20250514", + "claude-3-5-sonnet-latest", + "", + None, +] + + +def _request_kwargs(**chat_kwargs: Any) -> Dict[str, Any]: + """The keyword arguments the connection passed to ``messages.create``.""" + message = Message( + id="m", + model="claude", + role="assistant", + type="message", + stop_reason="end_turn", + content=[TextBlock(type="text", text='{"verdict": "ok"}')], + usage=_usage(), + ) + connection = _connection_returning(message) + connection.chat([ChatMessage(role=MessageRole.USER, content="hi")], **chat_kwargs) + return connection.client.messages.create.call_args.kwargs + + +@pytest.mark.parametrize("model", ["claude-sonnet-4-5", "claude-opus-4-6"]) +def test_native_output_config_applied_on_capable_model(model) -> None: + # One name from each way the capability check can match: a 4.5-generation alias + # reached by prefix, and a 4.6 name reached by exact match. The chat path consults + # the check as a whole, so covering only one branch would let it be narrowed to + # that branch while silently dropping native structured output for every model on + # the other. + output_config = _request_kwargs( + model=model, output_schema=OutputSchema(output_schema=_Answer) + )["output_config"] + + # Asserting the property name rather than mere presence: a config derived from the + # wrong schema, or from an empty placeholder, would also be present. + assert output_config["format"]["type"] == "json_schema" + assert set(output_config["format"]["schema"]["properties"]) == {"verdict"} + + +def test_native_output_config_not_applied_on_incapable_model() -> None: + assert "output_config" not in _request_kwargs( + model=_INCAPABLE_MODEL, output_schema=OutputSchema(output_schema=_Answer) + ) + + +def test_native_output_config_not_applied_without_schema() -> None: + assert "output_config" not in _request_kwargs( + model=_CAPABLE_MODEL, output_schema=None + ) + + +def test_native_output_config_not_applied_for_row_type_info() -> None: + # A RowTypeInfo schema has no native translation and must keep the + # prompt-engineering fallback rather than failing. + row_type = Types.ROW_NAMED(["verdict"], [Types.STRING()]) + assert "output_config" not in _request_kwargs( + model=_CAPABLE_MODEL, output_schema=OutputSchema(output_schema=row_type) + ) + + +def test_caller_output_config_wins_over_schema() -> None: + # Only one channel carries output_config into the request, so a derived config + # would replace the caller's outright and report nothing. The caller's value is + # kept and the schema stays on the prompt-engineering fallback. + caller_config = {"format": {"type": "json_schema", "schema": {"type": "object"}}} + + sent = _request_kwargs( + model=_CAPABLE_MODEL, + output_schema=OutputSchema(output_schema=_Answer), + output_config=caller_config, + )["output_config"] + + assert sent == caller_config + + +@pytest.mark.parametrize("model", _CAPABLE_MODELS) +def test_capability_predicate_accepts_capable_models(model) -> None: + assert _connection().supports_native_structured_output(model) is True + + +@pytest.mark.parametrize("model", _INCAPABLE_MODELS) +def test_capability_predicate_rejects_incapable_models(model) -> None: + assert _connection().supports_native_structured_output(model) is False + + +def test_alias_prefix_matches_dated_snapshot() -> None: + # The three 4.5-generation names are aliases, so a request may carry the dated + # snapshot instead. Turning the prefixes into exact matches would still satisfy + # the capable-models test above. + predicate = _connection().supports_native_structured_output + assert predicate("claude-sonnet-4-5-20250929") is True + + +def test_alias_prefix_does_not_match_longer_minor_version() -> None: + # A dated snapshot continues the alias with a "-" separator. A name that extends + # the alias without one is a different minor version, whose capability is not the + # alias's to answer for. + predicate = _connection().supports_native_structured_output + assert predicate("claude-sonnet-4-50") is False + + +def test_capability_reads_no_instance_state() -> None: + # __new__ skips __init__, so no field is set and no client exists. A predicate + # reading instance state would raise here instead of answering for its argument. + bare = AnthropicChatModelConnection.__new__(AnthropicChatModelConnection) + + assert bare.supports_native_structured_output(_CAPABLE_MODEL) is True + assert bare.supports_native_structured_output(_INCAPABLE_MODEL) is False + + +# --------------------------------------------------------------------------------- +# JSON prefill +# --------------------------------------------------------------------------------- + +# The continuation an assistant returns after a "{" prefill, and the document it +# completes. +_CONTINUATION = '"verdict": "ok"}' +_COMPLETED = "{" + _CONTINUATION + +# The models the provider documents as rejecting assistant-message prefilling, in the +# order the connection lists them. Mirroring that order keeps the two lists comparable +# side by side, so a name added to one and not the other stands out. +_PREFILL_UNSUPPORTED = [ + "claude-opus-4-6", + "claude-opus-4-7", + "claude-opus-4-8", + "claude-opus-5", + "claude-sonnet-4-6", + "claude-sonnet-5", + "claude-fable-5", + "claude-mythos-5", + "claude-mythos-preview", +] + +# Names that accept a prefill. The three 4.5-generation names are the load-bearing +# ones: they are documented as structured-output capable, so folding the two rules onto +# one list would silently withdraw the prefill from exactly these models. +# claude-sonnet-4-5-20250929 is the dated snapshot behind one of those aliases, and +# claude-3-5-sonnet-latest stands for every name the list does not mention, which keeps +# the prefill because only the listed names withdraw it. +_PREFILL_SUPPORTED = [ + "claude-opus-4-5", + "claude-sonnet-4-5", + "claude-haiku-4-5", + "claude-sonnet-4-5-20250929", + "claude-sonnet-4-20250514", + "claude-3-5-sonnet-latest", + "", + None, +] + + +class _AddArgs(BaseModel): + a: int + + +class _StubTool(Tool): + """Minimal tool stub; only its presence in the tools list matters.""" + + @classmethod + def tool_type(cls) -> ToolType: + return ToolType.FUNCTION + + def call(self, *args: Any, **kwargs: Any) -> None: + return None + + +def _prefill_outcome(**chat_kwargs: Any) -> tuple: + """Whether the request carried the prefill, and the content the response yielded. + + The two have to agree: a response reconstructed on any other signal than the + decision the request was built with either prepends a stray "{" or drops a + required one, and the response itself gives no sign of either. + """ + message = Message( + id="m", + model="claude", + role="assistant", + type="message", + stop_reason="end_turn", + content=[TextBlock(type="text", text=_CONTINUATION)], + usage=_usage(), + ) + connection = _connection_returning(message) + response = connection.chat( + [ChatMessage(role=MessageRole.USER, content="hi")], **chat_kwargs + ) + sent = connection.client.messages.create.call_args.kwargs["messages"] + return sent[-1] == {"role": "assistant", "content": "{"}, response.content + + +def test_json_prefill_not_applied_by_default() -> None: + # The parameter is opt-in: it steers the model with a technique several models + # reject outright, so a caller that does not ask for it must not get it. + assert _prefill_outcome(model=_INCAPABLE_MODEL) == (False, _CONTINUATION) + + +def test_json_prefill_not_applied_when_explicitly_false() -> None: + # The setup emits json_prefill on every call, so the key is always present and only + # its value separates opt-in from opt-out. Detecting the parameter by presence + # rather than by value would re-enable the prefill for every request. + assert _prefill_outcome(model=_INCAPABLE_MODEL, json_prefill=False) == ( + False, + _CONTINUATION, + ) + + +def test_json_prefill_applied_when_requested() -> None: + # An empty tools list, rather than no tools argument: a request configured with no + # tools still reaches the decision carrying a list, and only an emptiness test + # rather than a None test lets the prefill through there. + assert _prefill_outcome(model=_INCAPABLE_MODEL, json_prefill=True, tools=[]) == ( + True, + _COMPLETED, + ) + + +def test_json_prefill_suppressed_by_tools() -> None: + # The prefill forces JSON text where the model would otherwise emit tool_use blocks. + tool = _StubTool( + name="add", + metadata=ToolMetadata(name="add", description="adds", args_schema=_AddArgs), + ) + + assert _prefill_outcome( + model=_INCAPABLE_MODEL, json_prefill=True, tools=[tool] + ) == (False, _CONTINUATION) + + +def test_json_prefill_suppressed_by_caller_output_config() -> None: + assert _prefill_outcome( + model=_INCAPABLE_MODEL, + json_prefill=True, + output_config={"format": {"type": "json_schema", "schema": {"type": "object"}}}, + ) == (False, _CONTINUATION) + + +def test_json_prefill_suppressed_by_derived_output_config() -> None: + # The schema reaches the request as an output_config of the framework's own making, + # which the provider documents as incompatible with prefilling just the same. The + # model accepts prefilling, so the output_config is the only thing suppressing it. + assert _prefill_outcome( + model=_CAPABLE_MODEL, + json_prefill=True, + output_schema=OutputSchema(output_schema=_Answer), + ) == (False, _CONTINUATION) + + +def test_json_prefill_applied_when_schema_falls_back() -> None: + # Suppression keys on whether the schema reached the request, not on whether one was + # supplied. Keying it on the schema would strip the prefill the prompt-engineering + # fallback depends on, which is the case the prefill mainly exists for. + assert _prefill_outcome( + model=_INCAPABLE_MODEL, + json_prefill=True, + output_schema=OutputSchema(output_schema=_Answer), + ) == (True, _COMPLETED) + + +def test_json_prefill_suppressed_on_prefill_unsupported_model() -> None: + assert _prefill_outcome(model="claude-opus-4-6", json_prefill=True) == ( + False, + _CONTINUATION, + ) + + +def test_json_prefill_applied_on_structured_output_capable_model() -> None: + # The two capability rules draw different lines, and this model sits between them: + # the provider documents structured-output support from the 4.5 generation on but + # withdraws prefilling only from 4.6 on. Deriving the prefill rule from the + # structured-output allowlists would strip the prefill here, where the provider + # still accepts it. + assert _connection().supports_native_structured_output("claude-sonnet-4-5") is True + + assert _prefill_outcome(model="claude-sonnet-4-5", json_prefill=True) == ( + True, + _COMPLETED, + ) + + +@pytest.mark.parametrize("json_prefill", [True, False]) +def test_json_prefill_is_not_forwarded_to_the_provider(json_prefill) -> None: + # It is a framework parameter, so the SDK would reject it as an unknown request + # field whichever way the decision went. + assert "json_prefill" not in _request_kwargs( + model=_INCAPABLE_MODEL, json_prefill=json_prefill + ) + + +@pytest.mark.parametrize("model", _PREFILL_UNSUPPORTED) +def test_prefill_predicate_rejects_unsupported_models(model) -> None: + assert _supports_json_prefill(model) is False + + +@pytest.mark.parametrize("model", _PREFILL_SUPPORTED) +def test_prefill_predicate_accepts_every_other_model(model) -> None: + assert _supports_json_prefill(model) is True + + +# --------------------------------------------------------------------------------- +# Setup parameters +# --------------------------------------------------------------------------------- + + +def test_setup_defaults_json_prefill_to_false() -> None: + assert ( + AnthropicChatModelSetup(connection="conn").model_kwargs["json_prefill"] is False + ) + + +def test_setup_honors_explicit_json_prefill() -> None: + # Pins that the argument is read rather than the default being emitted + # unconditionally. + setup = AnthropicChatModelSetup(connection="conn", json_prefill=True) + assert setup.model_kwargs["json_prefill"] is True diff --git a/python/pyproject.toml b/python/pyproject.toml index 244ea3466..a9660d433 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -55,7 +55,7 @@ dependencies = [ "ollama==0.6.1", "dashscope~=1.24.2", "openai>=1.66.3", - "anthropic>=0.64.0", + "anthropic>=0.77.0", "chromadb==1.0.21", "mem0ai>=0.1.43,<2.0.0", "onnxruntime<1.24.1;python_version<'3.11'", diff --git a/review-guides/dist-dependency.md b/review-guides/dist-dependency.md new file mode 100644 index 000000000..facd8a8d3 --- /dev/null +++ b/review-guides/dist-dependency.md @@ -0,0 +1,62 @@ +# Review Guide: dist and Dependency + +Load this guide when a PR changes what the project ships or what it bundles: +the `dist` modules and their shade configuration, LICENSE or NOTICE, or a +dependency's coordinates, version, or scope in any module's `pom.xml`. It +narrows the full passes in `code_review.md` to the ones that matter most for +this area; the general passes still apply. + +## Focused checklist + +- Check that a module the change adds appears in `dist/pom.xml`. The shipped + set is a hand-written list of dependencies rather than anything derived + from the reactor, so a module missing from it can compile, test, and + release green while shipping none of its classes. +- Re-derive the NOTICE from the dependency tree the change resolves to + rather than from the dependencies it declares. A transitive addition + changes what is bundled without appearing in the diff, and nothing in the + build validates a NOTICE, so an entry that is absent or stale ships. +- Confirm a newly bundled dependency's version is declared rather than left + to Maven to resolve. Nothing is relocated, so every bundled dependency + shares one namespace, and the version one integration pulls in becomes the + version the others get. +- Treat a version change, or a move into or out of provided or test scope, + as a change to what ships, including when it arrives as cleanup unrelated + to the stated goal. Shading bundles compile and runtime alike, so moving + between those two changes nothing, while provided and test move a + dependency out of the artifact or back into it, which carries both a + NOTICE consequence and a deployment one. +- Read a shade filter for what it matches rather than for what it was + written to match. An exclude placed inside a filter whose artifact pattern + matches everything applies to every artifact being shaded, so a class + meant to be dropped from one conflicting dependency takes the + distribution's own copy with it. +- Compare a new file's license header against a complete one. The automated + check keys on the Apache License URL, so a header that keeps that URL + passes with prose lines dropped or altered, and the configuration that + would compare header text against a template runs in no workflow. + +## Validation + +No job compares what an artifact contains against what the NOTICE claims, +and the license check reads for a header's presence rather than its text, so +a green build says nothing about either. Build the artifact and read it. + +- Build one distribution: `mvn --batch-mode package -pl dist/flink- + -am -DskipTests`. Use `package` rather than `install`, which writes + distribution jars into your local repository. +- List what shipped: `jar tf dist/flink-/target/.jar`. The + fat jar carries the bundled dependencies and the NOTICE, while the `-thin` + variant carries this project's own classes alone and holds no NOTICE. +- See which version won: `mvn --batch-mode dependency:tree -Dverbose -pl + dist/common`. Without `-Dverbose` the output shows the resolved tree with + no indication of what was mediated away. +- License headers: `./tools/check-license.sh`. It reports presence, not + correctness. + +## Examples from past reviews + +| Case | Pass it exercises | Review | +|---|---|---| +| A new shade execution produced a thin jar for the Python wheel and placed a single-class exclude inside a filter whose artifact pattern matched every artifact. One distribution supplies its own version of that class, so it would have shipped without it. | Whether a shade filter matches only the artifact it was written for. | [#557](https://github.com/apache/flink-agents/pull/557#discussion_r2875661184) | +| After a rebase the NOTICE no longer matched the tree the distribution resolved: a bundled version was declared at an older number than the one that resolved, and three transitive dependencies were absent. The review also asked that the resolved version be managed explicitly rather than left to dependency mediation, since it was winning over the versions two other integrations requested. | Whether the NOTICE matches the resolved dependency tree, and whether a bundled version is declared rather than implicitly chosen. | [#821](https://github.com/apache/flink-agents/pull/821#discussion_r3719020558) |