Skip to content

feat(agent): add OrcaRouter as a first-class provider with API key and OAuth 2.0 + PKCE auth - #41

Closed
beilsteinernest215-cmd wants to merge 1 commit into
capitalone:mainfrom
beilsteinernest215-cmd:orcarouter/task-8246
Closed

beilsteinernest215-cmd wants to merge 1 commit into
capitalone:mainfrom
beilsteinernest215-cmd:orcarouter/task-8246

Conversation

@beilsteinernest215-cmd

Copy link
Copy Markdown

What

Adds OrcaRouter as a model provider for the runtime agent, plus a one-click
connect flow so a user can either paste an existing API key or authorize with
their own OrcaRouter account.

OrcaRouter is an OpenAI-compatible AI gateway that
routes many providers behind one endpoint. It is added the way this repo already
onboards a backend — as a new value of [anthropic].auth_mode ("orcarouter")
with its own [orcarouter] config block — so it appears as a named provider in
the config, the docs, and the CLI, not as a custom base URL on an existing mode.

Piece Where
Provider selection anthropic.auth_mode = "orcarouter" — vulnhunter-agent/agent/config.py
Provider config [orcarouter] — vulnhunter-agent/agent/config.example.toml
Credential seam agent/orcarouter/connect.py (ConnectCredentialProvider)
Credential store ~/.vulnhunter/orcarouter.json (0o600), override with [orcarouter].credentials_file
Inference base https://api.orcarouter.ai/v1 ([orcarouter].api_base_url)
Auth base https://www.orcarouter.ai ([orcarouter].auth_base_url)
Model discovery agent/orcarouter/models.py
Harness bridge harness/local_harness/orcarouter_env.py

Affiliation disclosure: I'm an engineer on the OrcaRouter team. This change
is made on OrcaRouter's behalf.

How the credential works

Two explicit, independent entries both yield one ordinary OrcaRouter API key
(sk-orca-…). They are two adapters over a single small interface,
ConnectCredentialProvider, which implements the project's existing
TokenProvider protocol — so runner.py, verify_runner.py, _llm.py, and the
issues stages keep working unchanged and never learn which entry produced the key:

python -m agent --mode=orcarouter --orcarouter-action=store-api-key   # paste a key
python -m agent --mode=orcarouter --orcarouter-action=login           # OAuth 2.0 + PKCE
python -m agent --mode=orcarouter --orcarouter-action=status
python -m agent --mode=orcarouter --orcarouter-action=clear
python -m agent --mode=orcarouter --orcarouter-action=models --capability chat

The key belongs to the user, not to this project: it is billed to their
OrcaRouter account, listed at
/console/token, and revocable by
them at any time. No client secret is involved.

Connect flow — Flow A (loopback redirect) by default, Flow B (out-of-band
code) selectable
via [orcarouter].flow. Chosen because the agent runs on a
machine where 127.0.0.1:<port> is bindable, so the ordinary case can be one
click — but the same binary must also work on locked-down or remote hosts that
cannot listen on loopback, which is what Flow B exists for. S256 is always
sent
(OrcaRouter lets the user choose "show me a code" even on Flow A, so the
weaker plain is never safe). Flow C (device grant) is the optional flow I did
not implement — the spec requires PKCE and only permits C as an extra.

Lifecycle. A PKCE-issued key is durable, not a refresh token. It is
reused across restarts rather than re-authorized (OrcaRouter caps PKCE issuance
at 10 keys per user per 24 hours), and no refresh grant is ever attempted. A
relay 401/403 marks the exact stored credential generation for
reauthentication (agent/runner.py::_mark_credential_rejected) and the old
secret is not deleted before a new login succeeds; because the mark is
keyed on (api_key, generation), a late failure from a superseded request can
never invalidate a credential that was just reauthorized.

Origins are kept apart. Authentication stays on
https://www.orcarouter.ai (authorize /auth, exchange /api/v1/auth/keys —
never the relay's /v1/auth/keys, which 404s) and inference/catalog on
https://api.orcarouter.ai/v1. Neither is derived from the other by swapping a
hostname or appending /v1. A shared self-hosted ORCA_BASE_URL plus explicit
ORCA_AUTH_BASE_URL / ORCA_API_BASE_URL overrides are supported, with env
winning over TOML and non-loopback origins required to be HTTPS.

AI entry points covered

Every model call in the repo funnels through two chokepoints — make_token_manager
and build_claude_settings — which is where the provider is wired:

  • Scan session — agent/runner.py (via the Claude Agent SDK).
  • Verify session — agent/verify_runner.py.
  • Issues extraction + semantic dedup one-shot calls — agent/_llm.py
    (call_json / call_json_with_fallback), used by issues_extract.py,
    issues_dedup.py, verify_refs.py.
  • Harness scan, LLM-judge, and miss-diagnostic claude subprocesses —
    harness/local_harness/{scan.py,benchmark/judge.py,benchmark/analyze_misses.py}
    via the new env bridge.

vulnhunter-fix makes no model call of its own — it prompts the host Claude Code
session — so nothing there needs changing; it inherits OrcaRouter like any other
model backend. The prompt-only skills (vulnhunt/, vulnhunt-fix-verify/) have
no provider code.

Model discovery and capability filtering

The model list is the gateway's real catalog, fetched from
GET {api_base}/models with Bearer auth — not a hand-written example list.
--mode=orcarouter models and the tests use the live catalog; when discovery
fails, a small verified seed
(openai/gpt-5.5, anthropic/claude-opus-4.8, google/gemini-3.5-flash,
deepseek/deepseek-v4-pro, orcarouter/auto) is used and the result is
reported as degraded rather than degrading to free text. A live result is
authoritative and the seed is never mixed into it.

Filtering is metadata-driven and fail-closed (nothing is guessed from a name):

  • text chat/agent — supported_endpoint_types includes one of
    openai / openai-response / anthropic / gemini, and excludes
    image-generation / openai-video / jina-rerank / embeddings;
  • image input — must be a chat model and declare image in
    architecture.input_modalities; a model with no declared modalities is
    excluded;
  • embedding / image generation / video / rerank — strict endpoint match.

The capability set is a parameter of the single discovery path, so a selector
built for one entry point is rebuilt for another, and an incompatible current
selection is dropped and reported (--orcarouter-action=models --capability image_input --model <text-only> exits non-zero naming the model) instead of
being sent blind.

Testing

Run against this tree (python -m pytest), Python 3.12, with the live
credential only for the live file:

Suite Result
vulnhunter-agent (with ORCAROUTER_API_KEY) 1239 passed
vulnhunter-agent (no credential; live file skips) 1233 passed, 6 skipped
harness 176 passed
vulnhunter-fix 556 passed, 10 skipped

New focused tests:

  • tests/test_orcarouter_connect.py (43) — API-key save/read/clear/mask and
    0o600 perms; verifier/challenge/state shape; authorize URL; the exact exchange
    path and body; success persistence; denial; Flow A state mismatch (code never
    redeemed); timeout; 403 code reuse/expiry; 400 downgrade; 429; network
    failure; scope downgrade; corrupt-key terminal classification; exact-generation
    401 reconnect; and that the verifier and key never appear in logs or errors.
  • tests/test_orcarouter_models.py (24) — catalog parsing with vendor/model
    namespaces and metadata; every capability filter; multimodal fail-closed; live
    discovery is authoritative; degraded fallback keeps the seed intact; a
    selection that becomes incompatible is cleared.
  • tests/test_orcarouter_integration.py (16) — the provider as a first-class
    auth_mode; both adapters return the same credential to the same downstream;
    auth only ever hits the auth origin and inference/catalog only the API origin;
    the CLI settings JSON carries ANTHROPIC_BASE_URL=https://api.orcarouter.ai
    (no trailing /v1) with the key; a self-hosted origin flows through; the
    generation-safe terminal rejection.
  • tests/test_orcarouter_cli.py (10) — both entries discoverable and separately
    exercised; status masks the secret; clear works.
  • tests/test_orcarouter_live.py (6, gated) — live catalog + filters, and a real
    request through the repo's own SDK path.
  • harness/tests/test_orcarouter_env.py (8) — the env bridge injects the
    inference origin/key, and is a no-op (env=None, inherit) when unconfigured.

Live evidence (verification date 2026-10-05):

  • GET https://api.orcarouter.ai/v1/models → HTTP 200, 16 models; the chat
    filter excludes the non-text endpoint types; deepseek/deepseek-v4.1-flash and
    deepseek/deepseek-v4-flash-vision-exp are the models declaring image input.
  • A real request was sent through this PR's code: agent.auth.make_token_manager
    → ConnectCredentialProvider → build_claude_settings → the SDK, returning
    {"ok": true} from the gateway (the CLI's ANTHROPIC_BASE_URL handshake was
    also confirmed by hand: without /v1 it routes, with /v1 it 404s).
  • POST https://www.orcarouter.ai/api/v1/auth/keys with a bogus code → HTTP 403
    {"error":"Invalid code or code_verifier"}; the wrong relay path
    https://api.orcarouter.ai/v1/auth/keys → HTTP 301, as documented.
  • A consent-requiring browser login was not performed here (it needs a human
    to approve), so PKCE is proven by the fake-auth-server path above, not by a
    recorded real consent. grep of the suite confirms no real key literal is
    committed — only obviously fake sk-orca-… fixtures.

UI

No screenshot: this repository has no renderable interface (no HTML/CSS/JS/TSX/
Vue/Svelte, no package.json, no web server or desktop shell). The OrcaRouter
entries are CLI commands, and the visible behavior is asserted in
tests/test_orcarouter_cli.py.

Provider evidence

  • Inference (OpenAI-compatible, Authorization: Bearer): https://api.orcarouter.ai/v1
  • Model list: GET https://api.orcarouter.ai/v1/models (public; richer with a key)
  • Authorization / token / device endpoints: https://www.orcarouter.ai/.well-known/openid-configuration
    (authorization_endpoint /auth, token_endpoint /api/v1/auth/keys)
  • Key management / revocation: https://www.orcarouter.ai/console/token,
    https://www.orcarouter.ai/console/authorized-apps
  • Pricing: https://www.orcarouter.ai/pricing
  • Routing/resale: OrcaRouter is an aggregator/reseller that routes hosted models
    from multiple upstream providers behind one OpenAI-compatible endpoint; it
    operates under its own entity and terms.
  • Maintenance owner for this provider entry: the OrcaRouter team (this PR);
    repository review by @capitalone/vulnhunter-admin-team per CODEOWNERS.
  • Verification date: 2026-10-05.

Not done

  • Flow C (device grant). Optional per the integration spec; PKCE (this PR's Flow
    A/B) is the required path.
  • No dependency was added: PKCE uses only the standard library (secrets,
    hashlib, base64) and discovery reuses the existing httpx.

…d OAuth 2.0 + PKCE auth

Signed-off-by: beilsteinernest215-cmd <beilsteinernest215-cmd@users.noreply.github.com>
@beilsteinernest215-cmd
beilsteinernest215-cmd requested a review from a team as a code owner October 5, 2026 16:43
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@schenksj

schenksj commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Hi @beilsteinernest215-cmd, thanks for the PR and for the detailed write-up and test coverage. We appreciate the effort, and thanks also for disclosing your affiliation up front.

We're going to close this one, though. We'd like VulnHunter to stay provider-neutral: no vendor-specific auth modes, credential stores, login flows, or model catalogs in the codebase. At ~3.7k lines this also adds more maintenance surface than we can take on for a single gateway.

Our current approach for non-Anthropic backends is documented in #11, in docs/working-with-non-anthropic-models.md (see How it works and Quick start). It treats any gateway that exposes an Anthropic-compatible inference endpoint the same way:

  • point Claude Code at the gateway by overriding the API path with ANTHROPIC_BASE_URL (e.g. https://openrouter.ai/api),
  • supply the gateway key via ANTHROPIC_AUTH_TOKEN (with ANTHROPIC_API_KEY explicitly empty),
  • select the model by the gateway's model slug via ANTHROPIC_MODEL, and pin the ANTHROPIC_DEFAULT_*_MODEL aliases and CLAUDE_CODE_SUBAGENT_MODEL to that same slug.

Could you take a look at that doc and let us know whether the approach is sufficient for OrcaRouter? From your PR description (ANTHROPIC_BASE_URL=https://api.orcarouter.ai routes correctly), it looks like it may work with no code changes. If you find a real gap in that generic path, we'd welcome a small, vendor-neutral issue or PR that addresses it for Anthropic-compatible endpoints in general.

Thanks again!

@schenksj schenksj closed this Oct 7, 2026
@beilsteinernest215-cmd

Copy link
Copy Markdown
Author

Thanks for the clear decision — I'll leave the provider-neutral call to you and won't push the vendor-specific version.

On your question: yes, the generic path in #11 is sufficient for OrcaRouter, with one base-URL detail. I checked it live against the gateway today (2026-10-08):

  • POST https://api.orcarouter.ai/v1/messages with an Anthropic Messages body returns HTTP 200, and accepts either x-api-key or Authorization: Bearer. Streaming works too.
  • So ANTHROPIC_BASE_URL should be https://api.orcarouter.ai without /v1 — the bare host 301s on /messages, and Claude Code appends /v1/messages itself, same shape as the OpenRouter example.
  • ANTHROPIC_AUTH_TOKEN carries the sk-orca-… key with ANTHROPIC_API_KEY empty, and ANTHROPIC_MODEL plus the ANTHROPIC_DEFAULT_* aliases and CLAUDE_CODE_SUBAGENT_MODEL take a vendor/model slug. GET https://api.orcarouter.ai/v1/models returns those same IDs, so slug pinning is straightforward.

One real gap, and it's vendor-neutral rather than OrcaRouter-specific: on the headless side, vulnhunter-agent has no config field for ANTHROPIC_BASE_URL, and in api_key mode it derives the sandbox network allow-list from api.anthropic.com (agent/build_settings.py:58, _anthropic_host). With the sandbox enabled (the load_config default), any Anthropic-compatible gateway — OpenRouter, OrcaRouter, or a self-hosted proxy — is therefore not allow-listed. If you'd like that closed, I can prepare a small vendor-neutral issue or PR that lets api_key mode target an arbitrary Anthropic-compatible base URL and allow-lists that host. Your call; otherwise I'll leave it here.

Thanks again for the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants