feat: optional bearer auth for local providers (mlx, llamacpp, lmstudio, ollama) - #9
Open
vinayvobbili wants to merge 1 commit into
Open
Conversation
…io, ollama) The four local-provider kinds all support OpenAI-compatible bearer auth in their server config, but the CLI's buildProviderApiKey() only honored OPENROUTER_API_KEY — meaning any locally-served model deployed behind --api-key (or a reverse-proxy with bearer auth) was unreachable from ToolCall-15 without a code change. Concrete deployments this unblocks: - vllm-mlx with `--api-key` (recommended prod config for the OpenAI- compatible endpoint on a shared LAN) - llama.cpp server with `--api-key` - LM Studio with the bearer-auth toggle enabled - Ollama fronted by an authenticating reverse proxy (we run one of these internally; the pattern is common enough to deserve a first-class env var) Each provider now reads an optional `<PROVIDER>_API_KEY` env var. When unset, the request goes out without an Authorization header — exact behavior the CLI has today, so this is purely additive for the default unauthenticated local-serving path. Symmetric with the desktop app's `api_key` / `api_key_env` provider fields documented in CONFIG_SCHEMA_V1.md. Validation: ran the full 15-scenario bench against a vllm-mlx server hosting Qwen3-Coder-30B-A3B-Instruct-8bit with --api-key set; MLX_API_KEY correctly attaches the bearer header and all scenarios that were expected to reach the model did.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds optional
<PROVIDER>_API_KEYenv-var support to the four local provider kinds (mlx,llamacpp,lmstudio,ollama) so models served behind bearer auth are reachable from the CLI.Today
buildProviderApiKey()only readsOPENROUTER_API_KEY; any locally-served OpenAI-compatible endpoint deployed with--api-key(or behind an authenticating reverse proxy) silently goes out without anAuthorizationheader and gets rejected.Concrete deployments this unblocks
--api-key— recommended prod config for serving a single MLX model on a shared LAN--api-keyChange shape
lib/models.ts—buildProviderApiKeynow also readsMLX_API_KEY,LLAMACPP_API_KEY,LMSTUDIO_API_KEY,OLLAMA_API_KEY(in addition toOPENROUTER_API_KEY). The local-provider keys are optional — when unset, the request goes out exactly as it does today (noAuthorizationheader), so the default unauthenticated local-serving path is unchanged..env.example— documents the new vars next to the existing host vars.Symmetric with the desktop app's
api_key/api_key_envprovider fields inCONFIG_SCHEMA_V1.md— the desktop UI has supported per-provider api keys for a while; this brings the CLI in line.Validation
Ran the full 15-scenario bench against a vllm-mlx server hosting Qwen3-Coder-30B-A3B-Instruct-8bit with
--api-keyset:Result:
83 / 100 — ★★★★ Good(12 pass / 1 partial / 2 fail in ~39 s wall-clock). Before this patch, every request returnedInvalid API key.Notes
vllmprovider kind). Different scope — that PR adds a new provider; this PR makes the existing local providers usable with bearer-protected servers. They don't conflict onlib/models.tsrebases other than thebuildProviderApiKeyfunction body.