Skip to content

Feature/openclaw lm#843

Open
lm041520 wants to merge 6 commits intodevelopfrom
feature/openclaw_lm
Open

Feature/openclaw lm#843
lm041520 wants to merge 6 commits intodevelopfrom
feature/openclaw_lm

Conversation

@lm041520
Copy link
Copy Markdown
Collaborator

@lm041520 lm041520 commented Apr 9, 2026

This pull request introduces the new OpenClawTool, integrating a remote OpenClaw Agent service as a built-in tool with multimodal (text and image) capabilities. It adds configuration, parameter handling, and runtime context injection for OpenClaw, and ensures the tool is properly registered and initialized for tenants. Several core services are updated to support runtime context injection for tools that require it, and the overall tool configuration and discovery mechanisms are extended for OpenClaw.

The most important changes are:

OpenClawTool Implementation and Configuration

  • Added OpenClawTool in openclaw_tool.py, supporting multimodal (text/image) input, runtime context injection, connection testing, and robust execution with error handling. The tool can process user-uploaded images and interact with the OpenClaw Agent via HTTP.
  • Registered OpenClawTool in the tool service import map and created its configuration file (openclaw_tool.json), as well as its entry in builtin_tools.json for system-level discovery and initialization.

Parameter Handling and Operation Support

  • Extended the operation tool logic to support OpenClaw-specific operations and parameters, including custom parameter sets for each operation type (e.g., print_task, device_query, image_understand).
  • Updated the Langchain adapter to recognize openclaw_tool as a multi-operation tool.

Runtime Context Injection

  • Updated chat and draft run services to inject runtime context (user ID, conversation ID, uploaded files) into tools that support set_runtime_context, ensuring OpenClaw and similar tools receive necessary data during execution.

Tool Initialization and Discovery

  • Improved the tenant tool initialization logic to support incremental addition of new built-in tools, using a new method to query existing tool classes and avoid duplicates.

Custom Tool Schema Parsing

  • Fixed schema parsing in the custom tool base class to handle stringified JSON schemas, improving robustness for custom tool definitions.

Summary by Sourcery

将 OpenClaw 远程 Agent 集成为内置多模态工具,并改进在聊天和草稿服务中的工具处理、初始化和上下文注入。

新功能:

  • 添加支持文本和图像的内置工具 OpenClawTool,并在工具服务和内置配置中注册该工具。
  • 在工具服务和 operation_tool 包装器中引入 OpenClaw 专用的操作参数处理逻辑,并在 Langchain 适配器中将其识别为多操作工具。

缺陷修复:

  • 修复自定义工具模式在模式内容以 JSON 字符串而非对象形式提供时的解析问题。

增强优化:

  • 允许按租户增量初始化内置工具:跳过已注册的工具类,仅在新增工具时提交变更。
  • 在聊天和草稿运行期间,将运行时上下文(用户、会话、已上传文件)注入到支持该能力的工具中,以更好地支持像 OpenClaw 这样的上下文感知工具。
  • 改进自定义工具基础模式解析,对字符串化的 JSON 模式进行更健壮的处理。
Original summary in English

Summary by Sourcery

Integrate the OpenClaw remote Agent as a built-in multimodal tool and enhance tool handling, initialization, and context injection across chat and draft services.

New Features:

  • Add the OpenClawTool builtin tool with text and image support and register it in the tool service and builtin configuration.
  • Introduce OpenClaw-specific operation parameter handling in both the tool service and operation_tool wrapper, and recognize it as a multi-operation tool in the Langchain adapter.

Bug Fixes:

  • Fix custom tool schema parsing when the schema content is provided as a JSON string instead of an object.

Enhancements:

  • Allow incremental initialization of builtin tools per tenant by skipping already-registered tool classes and committing only when new tools are added.
  • Inject runtime context (user, conversation, uploaded files) into tools that support it during chat and draft runs to better support context-aware tools like OpenClaw.
  • Improve custom tool base schema parsing to handle stringified JSON schemas more robustly.

lm041520 added 3 commits April 9, 2026 18:14
- Detect x-openclaw flag in OpenAPI schema and init dedicated config
- Implement multimodal input/output (image download, compress, base64)
- Add OpenClaw connection test and status validation in tool service
- Fix auth_config token check to support both api_key and bearer_token
- Inject runtime context (user_id, conversation_id, files) in chat services
…ibility

- Use safe .get() for server URL to avoid KeyError
- Support both api_key and token in connection test auth
- Add OpenAI/Volcano image format (image_url) support
- Add aiohttp import in _test_openclaw_connection
Create OpenClawTool class inheriting BuiltinTool with dedicated config
Remove all x-openclaw special handling from CustomTool (~270 lines)
Add multi-operation support (print_task, device_query, image_understand, general)
Change ensure_builtin_tools_initialized to incremental mode for auto-provisioning
Fix OperationTool and LangchainAdapter to support OpenClaw operation routing
Copilot AI review requested due to automatic review settings April 9, 2026 11:11
@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai bot commented Apr 9, 2026

Reviewer's Guide

实现了一个新的多模态 OpenClaw 内置工具,将其接入到工具发现/初始化流程和 LangChain 操作处理当中,为支持该能力的工具添加了运行时上下文注入机制,并增强了自定义/内置工具基础设施(增量式内置工具初始化和更健壮的 schema 解析)。

使用 OpenClawTool 和运行时上下文注入的聊天流程时序图

sequenceDiagram
    actor User
    participant AppChatService
    participant MultimodalService
    participant Agent
    participant OperationTool
    participant OpenClawTool
    participant OpenClawAgentHTTP

    User->>AppChatService: send message, files, selected tools

    alt has_files
        AppChatService->>MultimodalService: process_files(files)
        MultimodalService-->>AppChatService: processed_files
    else no_files
        AppChatService-->>AppChatService: processed_files = []
    end

    loop for each tool in tools
        AppChatService->>OpenClawTool: set_runtime_context(user_id, conversation_id, uploaded_files)
        note over AppChatService,OpenClawTool: Only called for tools with set_runtime_context
    end

    AppChatService->>Agent: chat(message, tools, processed_files)

    Agent->>OperationTool: execute(operation, message, image_url)
    OperationTool->>OpenClawTool: execute(operation, message, image_url)

    OpenClawTool->>OpenClawTool: _extract_image_from_uploads()
    OpenClawTool->>OpenClawTool: _build_input(message, image_url)

    OpenClawTool->>OpenClawAgentHTTP: POST /v1/responses
    OpenClawAgentHTTP-->>OpenClawTool: JSON response

    OpenClawTool->>OpenClawTool: _extract_response(response_data)
    OpenClawTool-->>OperationTool: ToolResult.success_result(data)
    OperationTool-->>Agent: ToolResult
    Agent-->>AppChatService: chat result
    AppChatService-->>User: final answer with OpenClaw result
Loading

OpenClawTool 及相关工具基础设施的类图

classDiagram
    class BuiltinTool {
    }

    class OpenClawTool {
        - string _server_url
        - string _api_key
        - string _agent_id
        - string _model
        - string _session_strategy
        - int _timeout
        - string _user_id
        - string _conversation_id
        - list _uploaded_files
        + OpenClawTool(tool_id, config)
        + name str
        + description str
        + parameters List~ToolParameter~
        + set_runtime_context(user_id, conversation_id, uploaded_files) void
        + test_connection() Dict~str, Any~
        + execute(**kwargs) ToolResult
        - _extract_image_from_uploads() str
        - _download_and_encode_image(image_url) str
        - _build_input(message, image_url) Any
        - _extract_response(response_data) str
        - _format_result(text) str
    }

    class OperationTool {
        - BaseTool base_tool
        - string operation
        + parameters List~ToolParameter~
        - _get_openclaw_params() List~ToolParameter~
        + execute(**kwargs) ToolResult
    }

    class ToolService {
        + ensure_builtin_tools_initialized(tenant_id) void
        + _get_operation_specific_params(tool_instance, operation) List~Dict[str, Any]~
        - _get_openclaw_tool_params(operation) List~Dict[str, Any]~
    }

    class BuiltinToolConfig {
        + uuid id
        + string tool_class
        + string display_name
        + bool requires_config
    }

    class ToolConfig {
        + uuid id
        + string name
        + string tool_type
        + uuid tenant_id
    }

    class BuiltinToolRepository {
        + get_existing_tool_classes(db, tenant_id) set
    }

    class CustomToolBase {
        - Any schema_content
        - dict _parsed_operations
        + __init__(tool_id, config)
        + _parse_openapi_schema() Any
    }

    class LangchainAdapter {
        + convert_tool(tool, operation) LangchainTool
        - _tool_supports_operations(tool) bool
    }

    class ToolParameter {
        + string name
        + ParameterType type
        + string description
        + bool required
        + list enum
    }

    class ToolResult {
        + Any data
        + float execution_time
        + string error
        + string error_code
        + success_result(data, execution_time) ToolResult
        + error_result(error, error_code, execution_time) ToolResult
    }

    BuiltinTool <|-- OpenClawTool
    BaseTool <|-- BuiltinTool

    OperationTool --> BaseTool
    OperationTool --> ToolParameter
    OperationTool --> ToolResult

    ToolService --> BuiltinToolRepository
    ToolService --> ToolConfig
    ToolService --> BuiltinToolConfig
    ToolService --> OpenClawTool
    ToolService --> ToolParameter

    BuiltinToolRepository --> BuiltinToolConfig
    BuiltinToolRepository --> ToolConfig

    CustomToolBase --> ToolConfig

    LangchainAdapter --> BaseTool
    LangchainAdapter --> OperationTool
    LangchainAdapter --> OpenClawTool
Loading

文件级变更

Change Details Files
在整个工具栈中添加支持多模态的 OpenClawTool 内置工具及其 OpenClaw 专用参数化。
  • 引入 OpenClawTool 内置类,支持文本/图像处理、运行时上下文存储、HTTP 调用、连接测试以及响应提取/格式化。
  • 定义 OpenClawTool 的参数和操作(print_task、device_query、image_understand、general),并通过 BuiltinTool.parameters 和基于 operation_tool 的参数构建器对外暴露。
  • 在内置工具导入映射、内置工具配置中注册 OpenClawTool,并在 LangChain 适配器中将其注册为多操作工具,以便 agent 栈能够发现并调用它。
api/app/core/tools/builtin/openclaw_tool.py
api/app/core/tools/builtin/operation_tool.py
api/app/core/tools/langchain_adapter.py
api/app/services/tool_service.py
api/app/core/tools/configs/builtin/openclaw_tool.json
api/app/core/tools/configs/builtin_tools.json
在聊天和草稿运行过程中,将运行时上下文(用户、会话、上传文件)注入到支持该能力的工具中。
  • 在多模态文件预处理之后,对工具进行迭代,当 tool_instance 暴露 set_runtime_context 时,在普通和流式聊天流程中使用 user_id、conversation_id 和 processed_files 调用它。
  • 在草稿运行和草稿运行流式流程中应用相同的运行时上下文注入模式,使 OpenClaw 和类似工具能够看到上传文件和用户元数据。
api/app/services/app_chat_service.py
api/app/services/draft_run_service.py
支持按租户增量初始化内置工具,并暴露已有内置工具类以便去重。
  • 用新的逻辑替换之前“只要存在任一内置工具就短路返回”的实现:始终加载内置工具配置,查询该租户已有的 builtin tool_class 值,只插入缺失的那部分。
  • 跟踪是否有新的内置工具被添加,仅在增量初始化实际改变状态时才提交和记录日志。
  • 添加仓储辅助方法,通过 BuiltinToolConfig 和 ToolConfig 的关联查询某个租户已有内置工具的 tool_class 值。
api/app/services/tool_service.py
api/app/repositories/tool_repository.py
扩展 tool_service 的操作参数解析,以处理 OpenClaw 专用操作。
  • 在 _get_operation_specific_params 中添加 openclaw_tool 分支,并委托给新的 _get_openclaw_tool_params 助手方法。
  • 实现 _get_openclaw_tool_params,用于返回 print_task、device_query、image_understand 以及通用回退 general 的操作专用参数集合,其语义与 OpenClawTool 保持一致。
api/app/services/tool_service.py
加强自定义工具 schema 处理,以支持字符串化的 JSON schema。
  • 在初始化自定义工具时,如果检测到 schema_content 为字符串,则尝试执行 json.loads 并用解析后的对象替换 schema_content;如果 JSON 解码失败,则回退为空字典。
  • 确保 _parse_openapi_schema 始终接收到一个类似 dict 的 schema,从而避免使用存储配置中字符串 schema 时的失败。
api/app/core/tools/custom/base.py

Tips and commands

Interacting with Sourcery

  • 触发新的评审: 在 pull request 中评论 @sourcery-ai review
  • 继续讨论: 直接回复 Sourcery 的评审评论。
  • 从评审评论生成 GitHub issue: 在某条评审评论下回复,要求 Sourcery 从该评论创建 issue。你也可以直接在该评论下回复 @sourcery-ai issue 来从中创建 issue。
  • 生成 pull request 标题: 在 pull request 标题的任意位置写上 @sourcery-ai,即可随时生成标题。你也可以在 pull request 中评论 @sourcery-ai title 来(重新)生成标题。
  • 生成 pull request 摘要: 在 pull request 正文任意位置写上 @sourcery-ai summary,即可在指定位置生成 PR 摘要。你也可以在 pull request 中评论 @sourcery-ai summary 来(重新)生成摘要。
  • 生成评审者指南: 在 pull request 中评论 @sourcery-ai guide,即可随时(重新)生成评审者指南。
  • 批量解决所有 Sourcery 评论: 在 pull request 中评论 @sourcery-ai resolve,将标记所有 Sourcery 评论为已解决。如果你已经处理完所有评论且不想再看到它们,这会很有用。
  • 撤销所有 Sourcery 评审: 在 pull request 中评论 @sourcery-ai dismiss,可撤销所有现有的 Sourcery 评审。若你希望从一次全新的评审开始,这尤其有用——别忘了再评论 @sourcery-ai review 来触发新的评审!

Customizing Your Experience

访问你的 dashboard 来:

  • 启用或禁用评审功能,例如 Sourcery 自动生成的 pull request 摘要、评审者指南等。
  • 更改评审语言。
  • 添加、删除或编辑自定义评审指令。
  • 调整其他评审设置。

Getting Help

Original review guide in English

Reviewer's Guide

Implements a new multimodal OpenClaw built‑in tool, wires it into tool discovery/initialization and LangChain operation handling, adds runtime context injection for tools that support it, and hardens custom/builtin tool infrastructure (incremental builtin init and robust schema parsing).

Sequence diagram for chat flow with OpenClawTool and runtime context injection

sequenceDiagram
    actor User
    participant AppChatService
    participant MultimodalService
    participant Agent
    participant OperationTool
    participant OpenClawTool
    participant OpenClawAgentHTTP

    User->>AppChatService: send message, files, selected tools

    alt has_files
        AppChatService->>MultimodalService: process_files(files)
        MultimodalService-->>AppChatService: processed_files
    else no_files
        AppChatService-->>AppChatService: processed_files = []
    end

    loop for each tool in tools
        AppChatService->>OpenClawTool: set_runtime_context(user_id, conversation_id, uploaded_files)
        note over AppChatService,OpenClawTool: Only called for tools with set_runtime_context
    end

    AppChatService->>Agent: chat(message, tools, processed_files)

    Agent->>OperationTool: execute(operation, message, image_url)
    OperationTool->>OpenClawTool: execute(operation, message, image_url)

    OpenClawTool->>OpenClawTool: _extract_image_from_uploads()
    OpenClawTool->>OpenClawTool: _build_input(message, image_url)

    OpenClawTool->>OpenClawAgentHTTP: POST /v1/responses
    OpenClawAgentHTTP-->>OpenClawTool: JSON response

    OpenClawTool->>OpenClawTool: _extract_response(response_data)
    OpenClawTool-->>OperationTool: ToolResult.success_result(data)
    OperationTool-->>Agent: ToolResult
    Agent-->>AppChatService: chat result
    AppChatService-->>User: final answer with OpenClaw result
Loading

Class diagram for OpenClawTool and related tool infrastructure

classDiagram
    class BuiltinTool {
    }

    class OpenClawTool {
        - string _server_url
        - string _api_key
        - string _agent_id
        - string _model
        - string _session_strategy
        - int _timeout
        - string _user_id
        - string _conversation_id
        - list _uploaded_files
        + OpenClawTool(tool_id, config)
        + name str
        + description str
        + parameters List~ToolParameter~
        + set_runtime_context(user_id, conversation_id, uploaded_files) void
        + test_connection() Dict~str, Any~
        + execute(**kwargs) ToolResult
        - _extract_image_from_uploads() str
        - _download_and_encode_image(image_url) str
        - _build_input(message, image_url) Any
        - _extract_response(response_data) str
        - _format_result(text) str
    }

    class OperationTool {
        - BaseTool base_tool
        - string operation
        + parameters List~ToolParameter~
        - _get_openclaw_params() List~ToolParameter~
        + execute(**kwargs) ToolResult
    }

    class ToolService {
        + ensure_builtin_tools_initialized(tenant_id) void
        + _get_operation_specific_params(tool_instance, operation) List~Dict[str, Any]~
        - _get_openclaw_tool_params(operation) List~Dict[str, Any]~
    }

    class BuiltinToolConfig {
        + uuid id
        + string tool_class
        + string display_name
        + bool requires_config
    }

    class ToolConfig {
        + uuid id
        + string name
        + string tool_type
        + uuid tenant_id
    }

    class BuiltinToolRepository {
        + get_existing_tool_classes(db, tenant_id) set
    }

    class CustomToolBase {
        - Any schema_content
        - dict _parsed_operations
        + __init__(tool_id, config)
        + _parse_openapi_schema() Any
    }

    class LangchainAdapter {
        + convert_tool(tool, operation) LangchainTool
        - _tool_supports_operations(tool) bool
    }

    class ToolParameter {
        + string name
        + ParameterType type
        + string description
        + bool required
        + list enum
    }

    class ToolResult {
        + Any data
        + float execution_time
        + string error
        + string error_code
        + success_result(data, execution_time) ToolResult
        + error_result(error, error_code, execution_time) ToolResult
    }

    BuiltinTool <|-- OpenClawTool
    BaseTool <|-- BuiltinTool

    OperationTool --> BaseTool
    OperationTool --> ToolParameter
    OperationTool --> ToolResult

    ToolService --> BuiltinToolRepository
    ToolService --> ToolConfig
    ToolService --> BuiltinToolConfig
    ToolService --> OpenClawTool
    ToolService --> ToolParameter

    BuiltinToolRepository --> BuiltinToolConfig
    BuiltinToolRepository --> ToolConfig

    CustomToolBase --> ToolConfig

    LangchainAdapter --> BaseTool
    LangchainAdapter --> OperationTool
    LangchainAdapter --> OpenClawTool
Loading

File-Level Changes

Change Details Files
Add OpenClawTool builtin with multimodal support and OpenClaw-specific parameterization across the tool stack.
  • Introduce OpenClawTool builtin class with text/image handling, runtime context storage, HTTP invocation, connection testing, and response extraction/formatting.
  • Define OpenClawTool parameters and operations (print_task, device_query, image_understand, general) and expose them via both BuiltinTool.parameters and operation_tool-specific parameter builders.
  • Register OpenClawTool in the builtin tool import map, builtin tool configs, and as a multi-operation tool in the LangChain adapter so the agent stack can discover and call it.
api/app/core/tools/builtin/openclaw_tool.py
api/app/core/tools/builtin/operation_tool.py
api/app/core/tools/langchain_adapter.py
api/app/services/tool_service.py
api/app/core/tools/configs/builtin/openclaw_tool.json
api/app/core/tools/configs/builtin_tools.json
Inject runtime context (user, conversation, uploaded files) into tools that support it during chat and draft runs.
  • After multimodal file preprocessing, iterate over tools and, when a tool_instance exposes set_runtime_context, call it with user_id, conversation_id, and processed_files in both normal and streaming chat flows.
  • Apply the same runtime context injection pattern in draft run and draft run streaming flows so OpenClaw and similar tools see uploaded files and user metadata.
api/app/services/app_chat_service.py
api/app/services/draft_run_service.py
Support incremental initialization of builtin tools per tenant and expose existing builtin tool classes for de-duplication.
  • Replace the previous short-circuit on any existing builtin tool with logic that always loads builtin config, queries existing builtin tool_class values for the tenant, and only inserts missing ones.
  • Track whether any new builtin tools were added, committing and logging only when incremental initialization actually changed state.
  • Add repository helper to query existing builtin tool_class values for a tenant’s builtin tools via a join on BuiltinToolConfig and ToolConfig.
api/app/services/tool_service.py
api/app/repositories/tool_repository.py
Extend tool_service operation parameter resolution to handle OpenClaw-specific operations.
  • Add openclaw_tool branch in _get_operation_specific_params that delegates to a new _get_openclaw_tool_params helper.
  • Implement _get_openclaw_tool_params to return operation-specific parameter sets for print_task, device_query, image_understand, and a general fallback, mirroring the semantics of OpenClawTool.
api/app/services/tool_service.py
Harden custom tool schema handling to support stringified JSON schemas.
  • When initializing a custom tool, detect if schema_content is a string; if so, attempt json.loads and replace schema_content with the parsed object, falling back to an empty dict on JSON decode error.
  • Ensure _parse_openapi_schema always receives a dict-like schema, avoiding failures on string schemas from stored configuration.
api/app/core/tools/custom/base.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - 我发现了 5 个问题,并留下了一些高层次的反馈:

  • app_chat_servicedraft_run_service 中的运行时上下文注入循环里,当没有传入文件时,processed_files 可能在赋值前被引用;请在 if files/if file_ids 分支之前先初始化 processed_files = [],以避免出现 NameError
  • CustomToolBase.__init__ 中,当 json.loads 失败时,你将 schema 设为 {},但仍然将 self.schema_content 保持为原始(无效)字符串;请在成功和异常分支中都将 self.schema_content = schema,以确保 _parse_openapi_schema 始终接收到解析后的对象或空字典。
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
-`app_chat_service``draft_run_service` 中的运行时上下文注入循环里,当没有传入文件时,`processed_files` 可能在赋值前被引用;请在 `if files`/`if file_ids` 分支之前先初始化 `processed_files = []`,以避免出现 `NameError`-`CustomToolBase.__init__` 中,当 `json.loads` 失败时,你将 `schema` 设为 `{}`,但仍然将 `self.schema_content` 保持为原始(无效)字符串;请在成功和异常分支中都将 `self.schema_content = schema`,以确保 `_parse_openapi_schema` 始终接收到解析后的对象或空字典。

## Individual Comments

### Comment 1
<location path="api/app/services/app_chat_service.py" line_range="168" />
<code_context>
         if files:
</code_context>
<issue_to_address>
**issue (bug_risk):** 当没有提供文件时,processed_files 可能在赋值前被引用

由于 `processed_files` 只在 `if files:` 块内部定义,而后续的 `processed_files or []` 用法在 `files` 为 falsy 时会抛出 `UnboundLocalError`。请在 `if` 之前初始化 `processed_files`(例如设为 `None``[]`),以确保它始终已定义。
</issue_to_address>

### Comment 2
<location path="api/app/services/app_chat_service.py" line_range="171" />
<code_context>
             if files:
</code_context>
<issue_to_address>
**issue (bug_risk):** 在流式聊天路径中也存在同样的 processed_files 未初始化问题

这里同样,`processed_files` 只在 `if files:` 中定义,但在 `uploaded_files = processed_files or []` 中被无条件使用,这会在 `files` 为 falsy 时抛出异常。请和非流式路径保持一致,在条件判断前初始化 `processed_files`。
</issue_to_address>

### Comment 3
<location path="api/app/services/draft_run_service.py" line_range="643" />
<code_context>
             if files:
</code_context>
<issue_to_address>
**issue (bug_risk):** draft_run 非流式路径也有在赋值前使用 processed_files 的风险

`processed_files` 只会在 `files` 为 truthy 时设置,但在上下文注入循环中会始终被使用。请在 `if files:` 块之前初始化 `processed_files`,以避免在未提供文件时出现 `UnboundLocalError`。
</issue_to_address>

### Comment 4
<location path="api/app/services/draft_run_service.py" line_range="643" />
<code_context>
             if files:
</code_context>
<issue_to_address>
**issue (bug_risk):** 在 run_stream 中也存在相同的 processed_files 初始化问题

当 `files` 为 falsy 时,`processed_files` 没有被初始化,但在注入运行时上下文时仍然会被使用。请将 `processed_files` 的初始化放在 `if files:` 块之外(如其它路径一样),以确保它始终已定义。
</issue_to_address>

### Comment 5
<location path="api/app/core/tools/custom/base.py" line_range="35-40" />
<code_context>
-        # 解析schema
+
+        # 解析 schema
+        schema = self.schema_content
+        if isinstance(schema, str):
+            try:
+                schema = json.loads(schema)
+                self.schema_content = schema
+            except json.JSONDecodeError:
+                schema = {}
         self._parsed_operations = self._parse_openapi_schema()
</code_context>
<issue_to_address>
**issue (bug_risk):** 在 JSON 解码失败时,schema_content 保留了无效的字符串,而没有使用回退字典

如果 `schema_content` 是字符串并且 `json.loads` 失败,`schema` 会被设为 `{}`,但 `self.schema_content` 仍然是无效字符串,因此 `_parse_openapi_schema()` 依然会收到错误的值。请在 `except` 块中同样赋值 `self.schema_content = {}`(或 `schema`),以确保下游代码始终看到的是一个字典。
</issue_to_address>

Sourcery 对开源项目是免费的——如果你觉得我们的评审有帮助,请考虑分享它们 ✨
帮我变得更有用!请在每条评论上点选 👍 或 👎,你的反馈会用于改进后续的评审。
Original comment in English

Hey - I've found 5 issues, and left some high level feedback:

  • In the runtime context injection loops in app_chat_service and draft_run_service, processed_files can be referenced before assignment when no files are provided; initialize processed_files = [] before the if files/if file_ids branches to avoid a NameError.
  • In CustomToolBase.__init__, when json.loads fails you set schema = {} but leave self.schema_content as the original (invalid) string; assign self.schema_content = schema in both the success and exception paths so _parse_openapi_schema always receives a parsed object or empty dict.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In the runtime context injection loops in `app_chat_service` and `draft_run_service`, `processed_files` can be referenced before assignment when no files are provided; initialize `processed_files = []` before the `if files`/`if file_ids` branches to avoid a `NameError`.
- In `CustomToolBase.__init__`, when `json.loads` fails you set `schema = {}` but leave `self.schema_content` as the original (invalid) string; assign `self.schema_content = schema` in both the success and exception paths so `_parse_openapi_schema` always receives a parsed object or empty dict.

## Individual Comments

### Comment 1
<location path="api/app/services/app_chat_service.py" line_range="168" />
<code_context>
         if files:
</code_context>
<issue_to_address>
**issue (bug_risk):** processed_files may be referenced before assignment when no files are provided

Because `processed_files` is only defined inside the `if files:` block, the later `processed_files or []` usage will raise `UnboundLocalError` when `files` is falsy. Initialize `processed_files` (e.g. to `None` or `[]`) before the `if` so it’s always defined.
</issue_to_address>

### Comment 2
<location path="api/app/services/app_chat_service.py" line_range="171" />
<code_context>
             if files:
</code_context>
<issue_to_address>
**issue (bug_risk):** Same uninitialized processed_files problem in the streaming chat path

Here too, `processed_files` is only defined inside `if files:` but then used unconditionally in `uploaded_files = processed_files or []`, which will raise when `files` is falsy. Mirror the non-streaming path by initializing `processed_files` before the conditional.
</issue_to_address>

### Comment 3
<location path="api/app/services/draft_run_service.py" line_range="643" />
<code_context>
             if files:
</code_context>
<issue_to_address>
**issue (bug_risk):** draft_run non-stream path also risks using processed_files before assignment

`processed_files` is only set when `files` is truthy but is always used in the context-injection loop. Please initialize `processed_files` before the `if files:` block to avoid an `UnboundLocalError` when no files are supplied.
</issue_to_address>

### Comment 4
<location path="api/app/services/draft_run_service.py" line_range="643" />
<code_context>
             if files:
</code_context>
<issue_to_address>
**issue (bug_risk):** Same processed_files initialization issue in run_stream

When `files` is falsy, `processed_files` isn’t initialized but is still used when injecting runtime context. Please initialize `processed_files` outside the `if files:` block (as in the other path) so it’s always defined.
</issue_to_address>

### Comment 5
<location path="api/app/core/tools/custom/base.py" line_range="35-40" />
<code_context>
-        # 解析schema
+
+        # 解析 schema
+        schema = self.schema_content
+        if isinstance(schema, str):
+            try:
+                schema = json.loads(schema)
+                self.schema_content = schema
+            except json.JSONDecodeError:
+                schema = {}
         self._parsed_operations = self._parse_openapi_schema()
</code_context>
<issue_to_address>
**issue (bug_risk):** On JSON decode failure, schema_content keeps the invalid string instead of the fallback dict

If `schema_content` is a string and `json.loads` fails, `schema` is set to `{}`, but `self.schema_content` remains the invalid string, so `_parse_openapi_schema()` still receives the bad value. In the `except` block, also assign `self.schema_content = {}` (or `schema`) so downstream code always sees a dict.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new built-in multimodal tool (OpenClawTool) that calls a remote OpenClaw Agent service, and extends the tool system to support operation-specific parameters and runtime context injection (user/conversation/uploaded files). It also adjusts tenant built-in tool initialization to support incremental additions, and improves robustness of custom tool schema parsing.

Changes:

  • Introduce OpenClawTool (text + image), register it as a built-in tool, and add corresponding built-in configs.
  • Extend operation-tool plumbing (ToolService method discovery + LangChain adapter + OperationTool) to treat openclaw_tool as a multi-operation tool with per-operation parameter sets.
  • Inject runtime context into tools during chat/draft runs and make built-in tool initialization incremental.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
api/app/services/tool_service.py Registers OpenClawTool, supports incremental built-in init, and adds OpenClaw operation-specific parameter metadata.
api/app/services/draft_run_service.py Injects runtime context (user/conversation/uploaded files) into tools before agent execution.
api/app/services/app_chat_service.py Injects runtime context into tools for both non-streaming and streaming chat paths.
api/app/repositories/tool_repository.py Adds query to fetch existing built-in tool classes for incremental initialization.
api/app/core/tools/langchain_adapter.py Treats openclaw_tool as an operation-capable built-in tool.
api/app/core/tools/custom/base.py Attempts to parse stringified OpenAPI schema content during custom tool initialization.
api/app/core/tools/configs/builtin/openclaw_tool.json Adds per-tool built-in config file for OpenClaw.
api/app/core/tools/configs/builtin_tools.json Adds OpenClaw entry to system-level built-in tool discovery/config schema.
api/app/core/tools/builtin/operation_tool.py Adds OpenClaw operation-specific parameter sets for operation-wrapped execution.
api/app/core/tools/builtin/openclaw_tool.py Implements the new OpenClaw built-in tool (HTTP execution, multimodal input building, image handling, response extraction).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +722 to +736
elif operation == "image_understand":
return [
{
"name": "message",
"type": "string",
"description": "发送给 OpenClaw 的图片理解任务,应描述需要对图片做什么(如描述内容、提取文字、分析信息)",
"required": True
},
{
"name": "image_url",
"type": "string",
"description": "必须提供,要分析的图片 URL 或 base64 data URI",
"required": True
}
]
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_get_openclaw_tool_params() 对 image_understand 也把 image_url 标记为必填,但 OpenClawTool 实现支持从 uploaded_files 提取图片;两者会造成能力/参数声明不一致(尤其是方法发现/前端表单/模型工具选择阶段会认为必须显式传 image_url)。建议把必填改为“有上传图片时可省略”,并在执行阶段做兜底校验与报错。

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同“image_understand 操作这里把 image_url 设为必填会与 OpenClawTool 的“优先从 uploaded_files 提取图片”的实现冲突:如果用户上传了图片但模型未显式传 image_url,参数校验会在执行前失败。建议将 image_url 设为非必填,并在 OpenClawTool.execute 中在“上传文件/参数均无图片”时再返回明确错误。”问题

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改

lm041520 added 2 commits April 9, 2026 19:33
…schema parsing

The _parse_openapi_schema() method already handles string-to-dict conversion internally, so the duplicate json.loads in __init__ was unnecessary.
…_tool

OperationTool wraps builtin tools for multi-operation support but did not
forward set_runtime_context, causing OpenClawTool to miss uploaded_files
and conversation_id when used with operation routing.
Copilot AI review requested due to automatic review settings April 9, 2026 12:01
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 10 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +722 to +736
elif operation == "image_understand":
return [
{
"name": "message",
"type": "string",
"description": "发送给 OpenClaw 的图片理解任务,应描述需要对图片做什么(如描述内容、提取文字、分析信息)",
"required": True
},
{
"name": "image_url",
"type": "string",
"description": "必须提供,要分析的图片 URL 或 base64 data URI",
"required": True
}
]
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同“image_understand 操作这里把 image_url 设为必填会与 OpenClawTool 的“优先从 uploaded_files 提取图片”的实现冲突:如果用户上传了图片但模型未显式传 image_url,参数校验会在执行前失败。建议将 image_url 设为非必填,并在 OpenClawTool.execute 中在“上传文件/参数均无图片”时再返回明确错误。”问题

"Authorization": f"Bearer {self._api_key}",
"Content-Type": "application/json",
"x-openclaw-agent-id": self._agent_id
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

明确抛错的原因

Copy link
Copy Markdown
Collaborator Author

@lm041520 lm041520 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

完成评论

…d operation variable

Change image_url from required to optional in both operation_tool.py and
tool_service.py for image_understand operation, avoiding parameter validation
conflict with uploaded_files priority logic.
Remove unused operation variable from OpenClawTool.execute().
@lanceyq lanceyq requested review from keeees and lanceyq April 10, 2026 05:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants