Skip to content

fix(tongyi): use DashScope keepalive session pool - #3870

Merged
crazywoola merged 1 commit into
langgenius:mainfrom
Jinx2000:fix/tongyi-use-dashscope-keepalive
Sep 11, 2026
Merged

crazywoola merged 1 commit into
langgenius:mainfrom
Jinx2000:fix/tongyi-use-dashscope-keepalive

Conversation

@Jinx2000

@Jinx2000 Jinx2000 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #3815 and #3817.

DashScope confirmed that its process-wide shared synchronous session could
reuse silently stale pooled connections, causing requests to block until the
default 300-second read timeout. The underlying issue was fixed in
dashscope/dashscope-sdk-python#187 and released in
DashScope SDK 1.27.4.

Tongyi 0.2.19 already locks DashScope 1.27.4, but the embedding implementation
still creates a separate requests.Session for each invocation and passes it
explicitly to the SDK. This bypasses the SDK-managed shared session and its new
keepalive-enabled HTTP adapter.

This change removes explicit session creation and injection from:

  • Text embedding
  • Multimodal embedding
  • Embedding credential validation

Embedding requests now use the shared session managed by DashScope 1.27.4.
Existing batching, token accounting, connection-error retries, timeout
handling, and HTTP 429 handling remain unchanged.

The LLM and rerank implementations are not changed.

Release Notes

Use DashScope's keepalive-enabled shared connection pool for Tongyi text and
multimodal embedding requests, reducing the risk of long stalls caused by stale
idle connections.

Change Type

  • Documentation / non-plugin change
  • Non-LLM plugin (tools, extensions, datasource, etc.)
  • LLM plugin

Screenshots / Videos

This is a backend connection-management change and has no user-interface
changes.

Before After
Embedding requests could intermittently take approximately 301 seconds after an idle connection became stale. The approximately 301-second stall was not reproduced with the validation build using DashScope's keepalive-enabled shared session.

LLM Plugin Checklist

Areas affected by this change (check all that apply)
  • Message flow (system messages, user ↔ assistant turn-taking)
  • Tool interaction flow (multi-round usage, Agent App and Agent Node)
  • Multimodal input (images, PDFs, audio, video, etc.)
  • Multimodal output (images, audio, video, etc.)
  • Structured output (JSON, XML, etc.)
  • Token consumption metrics
  • Other LLM functionality (text and multimodal embedding connection management)
  • New models / model parameter fixes

Version

  • Bumped top-level version in manifest.yaml from 0.2.19 to 0.2.20
  • dify_plugin>=0.3.0,<0.6.0 is declared in pyproject.toml and locked in uv.lock

Testing

  • Local deployment — Dify version: 1.13.3 (self-hosted local build)
  • SaaS (cloud.dify.ai)

Validation performed:

  • Packaged the plugin from a clean Git archive using the Dify Plugin CLI.
  • Installed frozen dependencies under Python 3.12.
  • Confirmed that uv.lock resolves dashscope==1.27.4.
  • Passed the official serverless plugin installation validator.
  • Ran the complete test suite from the unpacked .difypkg.
  • Test result: 167 passed, 83 skipped.
  • Deployed the 0.2.20-dev1 validation package to two self-hosted
    plugin_daemon nodes.
  • Text embedding and knowledge retrieval remained responsive after more than
    two hours of idle time, without reproducing the approximately 301-second stall.

@crazywoola
crazywoola deployed to models/tongyi September 11, 2026 14:24 — with GitHub Actions Active
@crazywoola
crazywoola merged commit 8863b79 into langgenius:main Sep 11, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants