Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions models/orcarouter/.difyignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
.venv/
__pycache__/
*.py[cod]
.pytest_cache/
.ruff_cache/
.env
*.difypkg
3 changes: 3 additions & 0 deletions models/orcarouter/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
INSTALL_METHOD=remote
REMOTE_INSTALL_URL=debug-plugin.dify.dev:5003
REMOTE_INSTALL_KEY=********-****-****-****-************
1 change: 1 addition & 0 deletions models/orcarouter/.python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
54 changes: 54 additions & 0 deletions models/orcarouter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Overview

OrcaRouter is an **OpenAI-compatible LLM gateway** that routes requests across 40+ upstream providers (OpenAI, Anthropic, Google, DeepSeek, Qwen, Grok, and more) to the cheapest or fastest path that can serve the model you asked for. Users pay **below list price**, and the dashboard tracks how much they save in real time.

# Configuration

After installation, sign up at [orcarouter.ai](https://www.orcarouter.ai), grab an API key from the [console](https://www.orcarouter.ai/console), and set it up under **Settings → Model Provider** in Dify.

# Adaptive routing — `orcarouter/auto`

The `orcarouter/auto` model is a virtual router that picks the best upstream per request. Configure its strategy from the [routing console](https://www.orcarouter.ai/console/routing). Available strategies:

| Strategy | Behavior |
|---|---|
| `cheapest` | Lowest-priced upstream that can serve the request (default) |
| `balanced` | Trades off price vs latency vs quality |
| `quality` | Highest-quality upstream |
| `adaptive` | Linear contextual bandit picks among candidates based on per-request features (prompt length, code/math/JSON density, declared `max_tokens` budget tier, MinHash-LSH similarity to recent traffic) |
| `gated_adaptive` | Layers a task-difficulty score on top of `adaptive` — mundane prompts restricted to a "weak" model pool, hard prompts to a "strong" pool |

**Why this matters**

- **Self-tuning** — adaptive strategies learn from your own traffic; performance shifts when workload changes
- **Microsecond overhead** — feature extraction and bandit are closed-form math; routing adds no measurable latency
- **Workload-aware in one call** — same `orcarouter/auto` endpoint serves cheap summarization and premium code-refactor requests with no client-side dispatch logic
- **Cost & reliability guardrails by construction** — reward explicitly penalizes cost, latency, rate-limit, and format failures
- **Admin-tunable without redeploys** — strategies, pools, thresholds, and reward weights are changed from console, not client code

# Fallback routing (`extra_body`)

OrcaRouter supports an OpenAI-compatible extension to specify per-request fallback models. Use the **`Fallback models`** and **`Routing mode`** parameters in the model node:

| Parameter | Example | Effect |
|---|---|---|
| `Fallback models` (string, JSON array) | `["openai/gpt-4o-mini", "openai/gpt-4o"]` | If the primary upstream fails, try these in order |
| `Routing mode` | `fallback` | Activate the fallback list above |

These are translated to the request body's `extra_body: {models, route}` key — see the [API reference](https://docs.orcarouter.ai).

# Reasoning models

Some models (OpenAI `o1`/`o3`/`gpt-5`, Anthropic `claude-opus-4.7`, DeepSeek `deepseek-reasoner`/`deepseek-r1`) expose reasoning controls:

- **OpenAI o-style**: `reasoning_effort` (high / medium / low / minimal), `verbosity`, `exclude_reasoning_tokens`
- **Anthropic thinking**: `enable_thinking`, `reasoning_budget` (token budget), `exclude_reasoning_tokens`
- **DeepSeek r-style**: `exclude_reasoning_tokens` (the model reasons by default)

These map onto the upstream provider's native reasoning protocol — OrcaRouter handles the translation.

Reasoning models do not accept `temperature`. The Dify UI will show `temperature` only for non-reasoning models.

# Pricing

All prices are configured at the per-model level in this plugin and reflect the **effective price you pay through OrcaRouter** (which may be below the upstream's list price). See the live [models page](https://www.orcarouter.ai/models) or `GET https://www.orcarouter.ai/api/pricing` for the authoritative source.
2 changes: 2 additions & 0 deletions models/orcarouter/_assets/orcarouter.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions models/orcarouter/_assets/orcarouter_square.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 6 additions & 0 deletions models/orcarouter/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
from dify_plugin import Plugin, DifyPluginEnv

plugin = Plugin(DifyPluginEnv(MAX_REQUEST_TIMEOUT=120))

if __name__ == "__main__":
plugin.run()
30 changes: 30 additions & 0 deletions models/orcarouter/manifest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
meta:
arch:
- amd64
- arm64
runner:
entrypoint: main
language: python
version: "3.12"
version: 0.0.1
name: orcarouter
author: langgenius
icon: orcarouter_square.svg
description:
en_US: OrcaRouter is an OpenAI-compatible LLM gateway that routes requests across 40+ upstream providers (OpenAI, Anthropic, Google, DeepSeek, Qwen, Grok, and more) to the cheapest or fastest path, with adaptive routing, fallback chains, and pay-below-list-price billing.
zh_Hans: OrcaRouter 是兼容 OpenAI 协议的 LLM 网关,跨 40+ 上游 provider(OpenAI、Anthropic、Google、DeepSeek、Qwen、Grok 等)按最便宜或最快路径分发请求,内置自适应路由、降级链与低于官方价计费。
ja_JP: OrcaRouter は OpenAI 互換の LLM ゲートウェイで、40 以上の上流プロバイダー(OpenAI、Anthropic、Google、DeepSeek、Qwen、Grok など)への最安または最速のパスへリクエストをルーティングします。アダプティブルーティング、フォールバックチェーン、表示価格以下の課金を内蔵。
pt_BR: OrcaRouter é um gateway LLM compatível com OpenAI que roteia solicitações entre mais de 40 provedores upstream (OpenAI, Anthropic, Google, DeepSeek, Qwen, Grok e outros) pelo caminho mais barato ou mais rápido, com roteamento adaptativo, cadeias de fallback e preços abaixo da tabela.
label:
en_US: OrcaRouter
resource:
memory: 268435456
permission:
model:
enabled: false
type: plugin
plugins:
models:
- provider/orcarouter.yaml
version: 0.0.1
created_at: 2026-05-14T19:20:00.000000000-04:00
13 changes: 13 additions & 0 deletions models/orcarouter/models/_endpoint_utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
DEFAULT_ENDPOINT_URL = "https://api.orcarouter.ai/v1"


def normalize_endpoint_url(credentials: dict) -> str:
"""Resolve and normalize the API endpoint URL from credentials.

Strips leading/trailing whitespace and trailing slashes from the URL.
Falls back to DEFAULT_ENDPOINT_URL when the value is missing or blank.
"""
endpoint_url = (
(credentials.get("endpoint_url") or DEFAULT_ENDPOINT_URL).strip().rstrip("/")
)
return endpoint_url or DEFAULT_ENDPOINT_URL
Empty file.
107 changes: 107 additions & 0 deletions models/orcarouter/models/llm/_position.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
- orcarouter/auto
- openai/gpt-5.5-2026-04-23
- openai/gpt-3.5-turbo-1106
- openai/gpt-4.1-mini-2025-04-14
- qwen/qwen3-vl-235b-a22b-thinking
- qwen/qwen3.6-plus
- z-ai/glm-5
- openai/gpt-3.5-turbo-0125
- openai/gpt-4o-mini-search-preview-2025-03-11
- openai/gpt-5-chat-latest
- openai/gpt-5.4-mini-2026-03-17
- z-ai/glm-4.5
- kimi/kimi-k2.6
- openai/gpt-5.2-2025-12-11
- qwen/qwen3-vl-235b-a22b-instruct
- openai/gpt-4-0613
- openai/gpt-5.4-mini
- openai/gpt-5.4-nano-2026-03-17
- z-ai/glm-4.6
- minimax/minimax-m2.7-highspeed
- openai/gpt-4-turbo-2024-04-09
- openai/gpt-4.1
- google/gemini-3-pro-preview
- openai/gpt-4o-2024-05-13
- openai/gpt-5.2-chat-latest
- qwen/qwen3.5-27b
- anthropic/claude-opus-4.7
- openai/gpt-4.1-nano
- openai/gpt-5.1
- google/gemini-2.5-flash-lite
- google/gemini-2.5-pro
- google/gemini-flash-latest
- openai/gpt-3.5-turbo
- openai/gpt-5-nano
- qwen/qwen3.6-flash
- openai/gpt-4o-2024-11-20
- openai/gpt-5
- google/gemini-3.1-flash-lite-preview
- minimax/minimax-m2.7
- anthropic/claude-opus-4.5
- google/gemma-4-31b-it
- openai/gpt-4o-mini-search-preview
- openai/gpt-5-2025-08-07
- openai/gpt-5.1-chat-latest
- qwen/qwen3.5-plus-2026-02-15
- google/gemini-robotics-er-1.6-preview
- openai/gpt-3.5-turbo-16k
- openai/gpt-4o-mini-2024-07-18
- openai/gpt-5.4-nano
- openai/gpt-4o-2024-08-06
- openai/gpt-5.2
- qwen/qwen3-max
- anthropic/claude-opus-4
- google/gemini-pro-latest
- openai/gpt-4-turbo
- qwen/qwen3-vl-8b-instruct
- qwen/qwen3.5-flash
- z-ai/glm-5.1
- openai/gpt-4.1-2025-04-14
- openai/gpt-4.1-mini
- openai/gpt-5-mini
- qwen/qwen3-max-preview
- anthropic/claude-opus-4.1
- anthropic/claude-sonnet-4.5
- deepseek/deepseek-v4-flash
- qwen/qwen3.6-plus-2026-04-02
- anthropic/claude-sonnet-4.6
- minimax/minimax-m2.5
- openai/gpt-5-search-api-2025-10-14
- qwen/qwen3.5-plus
- qwen/qwen3.6-35b-a3b
- deepseek/deepseek-chat
- google/gemini-flash-lite-latest
- google/gemini-2.5-flash
- google/gemini-3-flash-preview
- openai/gpt-4.1-nano-2025-04-14
- qwen/qwen3.6-flash-2026-04-16
- minimax/minimax-m2.5-highspeed
- qwen/qwen3-vl-8b-thinking
- anthropic/claude-haiku-4.5
- kimi/kimi-k2.5
- openai/gpt-5.5
- qwen/qwen3.5-397b-a17b
- anthropic/claude-sonnet-4
- google/gemini-3.1-pro-preview-customtools
- openai/gpt-5.4-2026-03-05
- deepseek/deepseek-reasoner
- deepseek/deepseek-v4-pro
- openai/gpt-4o-search-preview
- openai/gpt-5.4
- z-ai/glm-4.7
- openai/gpt-4o-search-preview-2025-03-11
- qwen/qwen3.5-122b-a10b
- google/gemma-4-26b-a4b-it
- qwen/qwen3.5-35b-a3b
- z-ai/glm-4.5-air
- google/gemini-3.1-pro-preview
- openai/gpt-5.1-2025-11-13
- anthropic/claude-opus-4.6
- openai/gpt-4
- openai/gpt-5-mini-2025-08-07
- openai/gpt-5-nano-2025-08-07
- openai/gpt-5-search-api
- openai/gpt-5.3-chat-latest
- openai/gpt-4o-mini
- qwen/qwen3.5-flash-2026-02-23
- openai/gpt-4o
97 changes: 97 additions & 0 deletions models/orcarouter/models/llm/anthropic-claude-haiku-4.5.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# https://www.orcarouter.ai/models/anthropic/claude-haiku-4.5
model: anthropic/claude-haiku-4.5
label:
en_US: claude-haiku-4.5
model_type: llm
features:
- agent-thought
- tool-call
- multi-tool-call
- stream-tool-call
- vision
- document
model_properties:
mode: chat
context_size: 200000
parameter_rules:
- name: temperature
use_template: temperature
- name: top_p
use_template: top_p
- name: top_k
label:
zh_Hans: 取样数量
en_US: Top k
type: int
required: false
help:
zh_Hans: 仅从每个后续标记的前 K 个选项中采样。
en_US: Only sample from the top K options for each subsequent token.
- name: max_tokens
use_template: max_tokens
default: 1024
min: 1
max: 64000
- name: response_format
label:
zh_Hans: 回复格式
en_US: Response Format
type: string
required: false
options:
- text
- json_object
- name: enable_thinking
type: boolean
required: false
default: false
label:
zh_Hans: 推理模式
en_US: Thinking Mode
help:
zh_Hans: 启用 Anthropic native thinking 模式;启用时输出会包含思考过程。
en_US: Enable Anthropic native thinking. When enabled, the output includes reasoning content.
- name: reasoning_budget
type: int
required: false
default: 2000
min: 1024
max: 64000
label:
en_US: Thinking Budget
zh_Hans: 推理预算
help:
en_US: Token budget for thinking (min 1024, must be less than max_tokens). Only valid when thinking is enabled.
zh_Hans: 推理 token 预算(最小 1024,必须小于 max_tokens)。仅 thinking 模式生效。
- name: exclude_reasoning_tokens
type: boolean
required: false
default: false
label:
zh_Hans: 隐藏思考过程
en_US: Hide the thought process
- name: orcarouter_fallback_models
type: string
required: false
label:
en_US: Fallback models (JSON array)
zh_Hans: 降级模型(JSON 数组)
help:
en_US: 'Upstream models to try if the primary fails. Example: ["openai/gpt-4o-mini","openai/gpt-4o"]'
zh_Hans: 主上游失败时按顺序尝试这些模型。例:["openai/gpt-4o-mini","openai/gpt-4o"]
- name: orcarouter_route
type: string
required: false
options:
- fallback
label:
en_US: Routing mode
zh_Hans: 路由模式
help:
en_US: Activate fallback list. Set to 'fallback' to use orcarouter_fallback_models in order.
zh_Hans: 启用降级列表。设为 'fallback' 后按 orcarouter_fallback_models 顺序尝试。
pricing:
input: "1.0"
output: "5.0"
unit: "0.000001"
currency: USD
Loading
Loading