cliproxy-embeddings-rerank-forward

A native plugin for CLIProxyAPI that forwards OpenAI-compatible /v1/embeddings, Cohere-style rerank, and Jev decision requests (OpenRouter Decisions API and TypeSafe System One API) to upstream providers. Supports multiple providers per route with key failover.

How It Works

The plugin registers four Management API routes:

Route	Module	Purpose	Default upstream path
/v0/management/embeddings	embeddings	OpenAI embeddings	/embeddings
/v0/management/rerank	rerank	Cohere-style rerank	/rerank
/v0/management/decisions	decisions	Jev via the OpenRouter Decisions API	/alpha/decisions
/v0/management/v1/systemone	systemone	Jev via the TypeSafe System One API	/v1/systemone

Each route is an independent module. Enable any subset. Within a module, configure one or more providers. A request is routed by matching body.model against provider model mappings; the first provider with a matching model is tried first, and if its keys all return 429/5xx, the next matching provider is tried.

The upstream API key is read from plugin configuration — it is never relayed from the inbound Authorization header (which carries the CLIProxyAPI management key).

Features

• Four independent modules: embeddings, rerank, decisions, systemone — configure any subset.
• Jev forwarding: decisions and systemone requests and responses are forwarded unchanged (only model is rewritten when you map an alias). TypeSafe-Danger-Zone-* request headers (images, long context) are passed to the upstream.
• Multiple providers per module: same model across providers for failover.
• Multiple keys per provider: first key priority, 429/5xx switches to next key.
• Model alias: client-visible alias mapped to upstream real model name (body rewritten on forward).
• Catch-all provider: a provider with no models list accepts any model (passthrough, no rewrite).
• Legacy config compat: v0.1/v0.2 single-provider config still works — auto-migrated to the new schema.

Installation

Build the shared library:

go build -buildmode=c-shared -o embeddings-forward.dylib .

Place the .dylib (macOS) / .so (Linux) / .dll (Windows) file in your CLIProxyAPI plugins directory.

Note on naming: The repo and Go module are cliproxy-embeddings-rerank-forward, but the artifact filename and plugin config key remain embeddings-forward for CLIProxyAPI store-upgrade compatibility. CPA derives the plugin ID from the dylib filename, so keeping embeddings-forward.dylib means existing plugins.configs.embeddings-forward: configs keep working after upgrade with zero changes. The plugin's internal metadata Name (embeddings-rerank-forward) is display-only and does not affect config matching.

Configuration

In the CPA management UI, each config field renders as a form input. Here's what you'll see and what to fill:

Field	Type	What to fill	Example
upstream_base_url	string	Embeddings upstream base URL	https://api.openai.com/v1
upstream_api_key	string	Embeddings API key	sk-xxx
upstream_path	string	Embeddings path (optional, default /embeddings)	/embeddings
upstream_models	string	Embeddings models, comma-separated. alias=name or just name. Empty = accept any	small=text-embedding-3-small, large
rerank_base_url	string	Rerank upstream base URL	https://api.openai.com/v1
rerank_api_key	string	Rerank API key	rk-xxx
rerank_path	string	Rerank path (optional, default /rerank)	/rerank
rerank_models	string	Rerank models, comma-separated. alias=name or just name. Empty = accept any	fast=rerank-8b, rerank-0.6b
embeddings	object	Advanced: multi-provider YAML. Paste module content (see below). Overrides flat fields	(see example below)
rerank	object	Advanced: multi-provider YAML. Paste module content (see below). Overrides flat fields	(see example below)

Common case: just fill the flat string fields above. Leave embeddings/rerank object fields empty.

Advanced (multi-provider): leave flat fields empty, fill the embeddings or rerank object field with the module YAML content:

Multi-provider (recommended)

plugins:
  enabled: true
  dir: plugins
  configs:
    embeddings-forward:
      enabled: true
      embeddings:
        enabled: true
        retries: 0                          # optional: extra attempts when a request ends without a 2xx (default 0)
        providers:
          - name: openai
            base_url: "https://api.openai-compatible.example.com/v1"
            path: "/embeddings"            # optional, defaults to /embeddings
            api_keys: ["sk-key1", "sk-key2"]
            strategy: round_robin           # optional: failover (default) | round_robin
            cooldown_seconds: 60            # optional: skip a key for N s after it returns 429
            models:
              - name: "text-embedding-3-small"
                alias: "emb-small"          # optional; client uses this alias
              - name: "text-embedding-3-large"
          - name: backup
            base_url: "https://backup.example.com/v1"
            api_keys: ["sk-backup"]
            models:
              - name: "text-embedding-3-small"  # same model, failover provider
      rerank:
        enabled: true
        providers:
          - name: primary
            base_url: "https://api.openai-compatible.example.com/v1"
            path: "/rerank"               # optional, defaults to /rerank
            api_keys: ["key1"]
            models:
              - name: "rerank-model-8b"
                alias: "rerank-large"
          - name: secondary
            base_url: "https://secondary.example.com/v1"
            api_keys: ["key2"]
            # no models → catch-all: accepts any model, forwards as-is

What to fill in the embeddings / rerank object fields

In the CPA management UI, the embeddings and rerank fields are object type — paste the module content (without the top-level key). For example, for embeddings:

enabled: true
providers:
  - name: openai
    base_url: "https://api.openai-compatible.example.com/v1"
    api_keys: ["sk-key1", "sk-key2"]
    models:
      - name: "text-embedding-3-small"
        alias: "emb-small"
  - name: backup
    base_url: "https://backup.example.com/v1"
    api_keys: ["sk-backup"]
    models:
      - name: "text-embedding-3-small"

For rerank:

enabled: true
providers:
  - name: primary
    base_url: "https://api.openai-compatible.example.com/v1"
    path: "/rerank"
    api_keys: ["key1"]
    models:
      - name: "rerank-model-8b"
        alias: "rerank-large"

Or skip the object fields entirely and use the flat fields (upstream_base_url, rerank_base_url, etc.) for single-provider setups.

Flat single-provider (simple, UI-friendly)

For the common case of one embeddings provider and/or one rerank provider, use flat fields — these map directly to CPA's config form:

plugins:
  enabled: true
  dir: plugins
  configs:
    embeddings-forward:
      enabled: true
      # Embeddings (optional — configure if you need /v1/embeddings)
      upstream_base_url: "https://api.openai-compatible.example.com/v1"
      upstream_api_key: "sk-your-key"
      upstream_path: "/embeddings"                              # optional, defaults to /embeddings
      upstream_models: "small=text-embedding-3-small, large"   # optional, comma-separated; empty = passthrough
      # Rerank (optional — configure if you need /v1/rerank)
      rerank_base_url: "https://api.openai-compatible.example.com/v1"
      rerank_api_key: "rk-your-key"
      rerank_path: "/rerank"                                    # optional, defaults to /rerank
      rerank_models: "fast=rerank-8b, rerank-0.6b"              # optional, comma-separated; empty = passthrough
      # Jev via OpenRouter Decisions API (optional — POST /v0/management/decisions)
      decisions_base_url: "https://openrouter.ai/api"
      decisions_api_key: "sk-or-your-key"
      decisions_models: "jev=typesafe/jev-1.14"                 # optional; empty = passthrough
      # Jev via TypeSafe System One API (optional — POST /v0/management/v1/systemone)
      systemone_base_url: "https://api.typesafe.ai"             # or https://openrouter.ai/api with an OpenRouter key
      systemone_api_key: "ts-your-key"
      systemone_retries: 3                                      # optional; same for upstream_/rerank_/decisions_retries, default 0

Each module is independent — configure any subset. Flat fields are migrated to a single legacy provider per module. The *_models format is comma-separated entries of either name (alias defaults to name) or alias=name. Empty *_models means catch-all (accept any model, passthrough).

Legacy single-provider (v0.1, still supported)

plugins:
  enabled: true
  dir: plugins
  configs:
    embeddings-forward:
      enabled: true
      upstream_base_url: "https://api.openai-compatible.example.com/v1"
      upstream_api_key: "sk-your-key"
      upstream_path: "/embeddings"          # optional, defaults to /embeddings

Legacy v0.1 config (embeddings only, no models) is auto-migrated: embeddings module enabled with a single legacy provider, catch-all passthrough. Rerank requires flat rerank_* fields or the modular schema.

Provider/model routing

1. Request arrives at /v0/management/embeddings or /v0/management/rerank.
2. Module must be enabled, else 404.
3. body.model is matched against providers in config order:
   • Provider with no models → catch-all, matches any model.
   • Else, match by alias (defaults to name when empty) or name exactly.
4. Keys of the first matching provider are tried in order. On 429/5xx, next key; if all keys fail, next matching provider.
   • strategy: failover (default) always starts at api_keys[0].
   • strategy: round_robin rotates the starting key on every request, spreading load across keys (useful when the upstream rate-limits per key).
   • cooldown_seconds: N — a key that returned 429 is moved to the end of the try order for N seconds, so the same limited key is not hit first again. Cooling-down keys are still tried last rather than skipped, so a request never fails only because every key is cooling down. Default 0 (off).
   • Rotation and cooldown state are per provider name and in-memory (reset on plugin reload).
5. If alias was used and differs from name, body.model is rewritten to the upstream name before forwarding.
6. No provider matches → 404.
7. Module retries: N (default 0): steps 4–5 form one round. When a round ends without a 2xx response (including a 4xx such as 404), the whole round runs again, up to N more times. The last response is returned.

Upstream URL construction

The final upstream URL is provider.base_url + provider.path:

base_url	path	Final URL
https://api.example.com/v1	(default) /embeddings	https://api.example.com/v1/embeddings
https://api.example.com/v1	(default) /rerank	https://api.example.com/v1/rerank
https://openrouter.ai/api	(default) /alpha/decisions	https://openrouter.ai/api/alpha/decisions
https://api.typesafe.ai	(default) /v1/systemone	https://api.typesafe.ai/v1/systemone
https://api.example.com/v1	/custom/path	https://api.example.com/v1/custom/path

Client Usage

Set your SDK's base_url to the CLIProxyAPI host with the /v0/management prefix.

Embeddings

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8317/v0/management",
    api_key="<MANAGEMENT_PASSWORD>",  # used as management key
)

# Use alias "emb-small" or upstream name "text-embedding-3-small"
response = client.embeddings.create(
    model="emb-small",
    input="hello world",
)

Rerank

import requests

resp = requests.post(
    "http://localhost:8317/v0/management/rerank",
    headers={
        "Authorization": "Bearer <MANAGEMENT_PASSWORD>",
        "Content-Type": "application/json",
    },
    json={
        "model": "rerank-large",  # alias or upstream name
        "query": "apple",
        "documents": ["apple is a fruit", "banana is a fruit"],
    },
)
print(resp.json())
# {"object":"list","data":[{"index":0,"relevance_score":0.95},{"index":1,"relevance_score":0.03}]}

Jev (System One API)

The route keeps the TypeSafe SDK path, so the official SDK works by changing the base URL:

from typesafe_sdk import TypeSafeClient  # pip install typesafe-sdk

client = TypeSafeClient(
    base_url="http://localhost:8317/v0/management",
    api_key="<MANAGEMENT_PASSWORD>",  # used as management key
    compression=False,  # the plugin needs plain JSON; the SDK zstd-compresses bodies >= 1 KiB by default
)

Or with plain HTTP:

curl http://localhost:8317/v0/management/v1/systemone \
  -H "Authorization: Bearer <MANAGEMENT_PASSWORD>" \
  -H "Content-Type: application/json" \
  -d '{"model":"jev-latest","state":"I was charged twice this month.","questions":{"billing":{"type":"noul","instructions":"Is this message about billing?"}}}'

Jev (OpenRouter Decisions API)

curl http://localhost:8317/v0/management/decisions \
  -H "Authorization: Bearer <MANAGEMENT_PASSWORD>" \
  -H "Content-Type: application/json" \
  -d '{"model":"typesafe/jev-1.14","state":{"ticket":"Checkout shows a blank screen after I click Pay."},"questions":{"is_bug":{"type":"noul","instructions":"Is the customer reporting a software defect?"}}}'

Both return the upstream body unchanged, e.g. {"answers":{"billing":{"type":"noul","noul":0.97}}, ...}.

Limitations

This is a pure plugin — no server-side modifications required. The tradeoffs:

Limitation	Detail
Path	Routes live under /v0/management/..., not /v1/...
Compressed bodies	Request bodies must be plain JSON; Content-Encoding (e.g. the TypeSafe SDK's default zstd) is rejected with 400
CPA version	Requires a CPA with plugin schema v6 (2026-09-08 or later), which returns plugin JSON responses without HTML escaping
Jev multipart	System One multipart/form-data image uploads are not supported; send images as assets with data_base64 in JSON
Auth	Client must send the management key (Authorization: Bearer <MANAGEMENT_PASSWORD> or X-Management-Key)
Response sanitization	CLIProxyAPI's ServeManagementHTTP applies htmlsanitize to JSON-looking bodies. Numeric embedding vectors are preserved (Go's json.Number retains precision).
No AuthManager	Provider keys are managed in plugin config, not via CLIProxyAPI's auth pool

License

MIT