Repository navigation
feat(agent): add OrcaRouter as a first-class provider with API key and OAuth 2.0 + PKCE auth - #41
beilsteinernest215-cmd wants to merge 1 commit into
Conversation
…d OAuth 2.0 + PKCE auth Signed-off-by: beilsteinernest215-cmd <beilsteinernest215-cmd@users.noreply.github.com>
|
|
|
Hi @beilsteinernest215-cmd, thanks for the PR and for the detailed write-up and test coverage. We appreciate the effort, and thanks also for disclosing your affiliation up front. We're going to close this one, though. We'd like VulnHunter to stay provider-neutral: no vendor-specific auth modes, credential stores, login flows, or model catalogs in the codebase. At ~3.7k lines this also adds more maintenance surface than we can take on for a single gateway. Our current approach for non-Anthropic backends is documented in #11, in
Could you take a look at that doc and let us know whether the approach is sufficient for OrcaRouter? From your PR description ( Thanks again! |
|
Thanks for the clear decision — I'll leave the provider-neutral call to you and won't push the vendor-specific version. On your question: yes, the generic path in #11 is sufficient for OrcaRouter, with one base-URL detail. I checked it live against the gateway today (2026-10-08):
One real gap, and it's vendor-neutral rather than OrcaRouter-specific: on the headless side, Thanks again for the review. |
What
Adds OrcaRouter as a model provider for the runtime agent, plus a one-click
connect flow so a user can either paste an existing API key or authorize with
their own OrcaRouter account.
OrcaRouter is an OpenAI-compatible AI gateway that
routes many providers behind one endpoint. It is added the way this repo already
onboards a backend — as a new value of
[anthropic].auth_mode("orcarouter")with its own
[orcarouter]config block — so it appears as a named provider inthe config, the docs, and the CLI, not as a custom base URL on an existing mode.
anthropic.auth_mode = "orcarouter"—vulnhunter-agent/agent/config.py[orcarouter]—vulnhunter-agent/agent/config.example.tomlagent/orcarouter/connect.py(ConnectCredentialProvider)~/.vulnhunter/orcarouter.json(0o600), override with[orcarouter].credentials_filehttps://api.orcarouter.ai/v1([orcarouter].api_base_url)https://www.orcarouter.ai([orcarouter].auth_base_url)agent/orcarouter/models.pyharness/local_harness/orcarouter_env.pyAffiliation disclosure: I'm an engineer on the OrcaRouter team. This change
is made on OrcaRouter's behalf.
How the credential works
Two explicit, independent entries both yield one ordinary OrcaRouter API key
(
sk-orca-…). They are two adapters over a single small interface,ConnectCredentialProvider, which implements the project's existingTokenProviderprotocol — sorunner.py,verify_runner.py,_llm.py, and theissues stages keep working unchanged and never learn which entry produced the key:
The key belongs to the user, not to this project: it is billed to their
OrcaRouter account, listed at
/console/token, and revocable bythem at any time. No client secret is involved.
Connect flow — Flow A (loopback redirect) by default, Flow B (out-of-band
code) selectable via
[orcarouter].flow. Chosen because the agent runs on amachine where
127.0.0.1:<port>is bindable, so the ordinary case can be oneclick — but the same binary must also work on locked-down or remote hosts that
cannot listen on loopback, which is what Flow B exists for. S256 is always
sent (OrcaRouter lets the user choose "show me a code" even on Flow A, so the
weaker
plainis never safe). Flow C (device grant) is the optional flow I didnot implement — the spec requires PKCE and only permits C as an extra.
Lifecycle. A PKCE-issued key is durable, not a refresh token. It is
reused across restarts rather than re-authorized (OrcaRouter caps PKCE issuance
at 10 keys per user per 24 hours), and no refresh grant is ever attempted. A
relay
401/403marks the exact stored credential generation forreauthentication (
agent/runner.py::_mark_credential_rejected) and the oldsecret is not deleted before a new login succeeds; because the mark is
keyed on
(api_key, generation), a late failure from a superseded request cannever invalidate a credential that was just reauthorized.
Origins are kept apart. Authentication stays on
https://www.orcarouter.ai(authorize/auth, exchange/api/v1/auth/keys—never the relay's
/v1/auth/keys, which 404s) and inference/catalog onhttps://api.orcarouter.ai/v1. Neither is derived from the other by swapping ahostname or appending
/v1. A shared self-hostedORCA_BASE_URLplus explicitORCA_AUTH_BASE_URL/ORCA_API_BASE_URLoverrides are supported, with envwinning over TOML and non-loopback origins required to be HTTPS.
AI entry points covered
Every model call in the repo funnels through two chokepoints —
make_token_managerand
build_claude_settings— which is where the provider is wired:agent/runner.py(via the Claude Agent SDK).agent/verify_runner.py.agent/_llm.py(
call_json/call_json_with_fallback), used byissues_extract.py,issues_dedup.py,verify_refs.py.claudesubprocesses —harness/local_harness/{scan.py,benchmark/judge.py,benchmark/analyze_misses.py}via the new env bridge.
vulnhunter-fixmakes no model call of its own — it prompts the host Claude Codesession — so nothing there needs changing; it inherits OrcaRouter like any other
model backend. The prompt-only skills (
vulnhunt/,vulnhunt-fix-verify/) haveno provider code.
Model discovery and capability filtering
The model list is the gateway's real catalog, fetched from
GET {api_base}/modelswith Bearer auth — not a hand-written example list.--mode=orcarouter modelsand the tests use the live catalog; when discoveryfails, a small verified seed
(
openai/gpt-5.5,anthropic/claude-opus-4.8,google/gemini-3.5-flash,deepseek/deepseek-v4-pro,orcarouter/auto) is used and the result isreported as degraded rather than degrading to free text. A live result is
authoritative and the seed is never mixed into it.
Filtering is metadata-driven and fail-closed (nothing is guessed from a name):
supported_endpoint_typesincludes one ofopenai/openai-response/anthropic/gemini, and excludesimage-generation/openai-video/jina-rerank/embeddings;imageinarchitecture.input_modalities; a model with no declared modalities isexcluded;
The capability set is a parameter of the single discovery path, so a selector
built for one entry point is rebuilt for another, and an incompatible current
selection is dropped and reported (
--orcarouter-action=models --capability image_input --model <text-only>exits non-zero naming the model) instead ofbeing sent blind.
Testing
Run against this tree (
python -m pytest), Python 3.12, with the livecredential only for the live file:
vulnhunter-agent(withORCAROUTER_API_KEY)vulnhunter-agent(no credential; live file skips)harnessvulnhunter-fixNew focused tests:
tests/test_orcarouter_connect.py(43) — API-key save/read/clear/mask and0o600 perms; verifier/challenge/state shape; authorize URL; the exact exchange
path and body; success persistence; denial; Flow A state mismatch (code never
redeemed); timeout; 403 code reuse/expiry; 400 downgrade; 429; network
failure; scope downgrade; corrupt-key terminal classification; exact-generation
401 reconnect; and that the verifier and key never appear in logs or errors.
tests/test_orcarouter_models.py(24) — catalog parsing with vendor/modelnamespaces and metadata; every capability filter; multimodal fail-closed; live
discovery is authoritative; degraded fallback keeps the seed intact; a
selection that becomes incompatible is cleared.
tests/test_orcarouter_integration.py(16) — the provider as a first-classauth_mode; both adapters return the same credential to the same downstream;auth only ever hits the auth origin and inference/catalog only the API origin;
the CLI settings JSON carries
ANTHROPIC_BASE_URL=https://api.orcarouter.ai(no trailing
/v1) with the key; a self-hosted origin flows through; thegeneration-safe terminal rejection.
tests/test_orcarouter_cli.py(10) — both entries discoverable and separatelyexercised; status masks the secret; clear works.
tests/test_orcarouter_live.py(6, gated) — live catalog + filters, and a realrequest through the repo's own SDK path.
harness/tests/test_orcarouter_env.py(8) — the env bridge injects theinference origin/key, and is a no-op (
env=None, inherit) when unconfigured.Live evidence (verification date 2026-10-05):
GET https://api.orcarouter.ai/v1/models→ HTTP 200, 16 models; the chatfilter excludes the non-text endpoint types;
deepseek/deepseek-v4.1-flashanddeepseek/deepseek-v4-flash-vision-expare the models declaringimageinput.agent.auth.make_token_manager→
ConnectCredentialProvider→build_claude_settings→ the SDK, returning{"ok": true}from the gateway (the CLI'sANTHROPIC_BASE_URLhandshake wasalso confirmed by hand: without
/v1it routes, with/v1it 404s).POST https://www.orcarouter.ai/api/v1/auth/keyswith a bogus code → HTTP 403{"error":"Invalid code or code_verifier"}; the wrong relay pathhttps://api.orcarouter.ai/v1/auth/keys→ HTTP 301, as documented.to approve), so PKCE is proven by the fake-auth-server path above, not by a
recorded real consent.
grepof the suite confirms no real key literal iscommitted — only obviously fake
sk-orca-…fixtures.UI
No screenshot: this repository has no renderable interface (no HTML/CSS/JS/TSX/
Vue/Svelte, no
package.json, no web server or desktop shell). The OrcaRouterentries are CLI commands, and the visible behavior is asserted in
tests/test_orcarouter_cli.py.Provider evidence
Authorization: Bearer):https://api.orcarouter.ai/v1GET https://api.orcarouter.ai/v1/models(public; richer with a key)https://www.orcarouter.ai/.well-known/openid-configuration(
authorization_endpoint/auth,token_endpoint/api/v1/auth/keys)https://www.orcarouter.ai/console/token,https://www.orcarouter.ai/console/authorized-appshttps://www.orcarouter.ai/pricingfrom multiple upstream providers behind one OpenAI-compatible endpoint; it
operates under its own entity and terms.
repository review by
@capitalone/vulnhunter-admin-teamperCODEOWNERS.Not done
A/B) is the required path.
secrets,hashlib,base64) and discovery reuses the existinghttpx.