feat(anthropic): multi-account load balancing with quota-aware rotation#70
Merged
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
이번 풀 리퀘스트는 Anthropic 멀티 계정 지원 및 로드 밸런싱 기능(claude_oauth 인증 모드)을 추가하여 세션 스티키 및 할당량 인지형 계정 풀(AccountPool)과 관련 CLI 명령어(shunt login claude)를 구현합니다. 코드 리뷰에서는 Windows 환경에서의 호환성 문제(실행 파일 확장자 누락 및 USERPROFILE 폴백 누락)와 비동기 컨텍스트 내에서 동기식 파일 I/O를 수행하여 Tokio 런타임을 블로킹할 수 있는 성능 문제를 지적하고 있으며, 이를 해결하기 위한 구체적인 개선 방안을 제시하고 있습니다.
Implements M8 across three phases: - Phase 1: auth = "claude_oauth" + [[providers.*.accounts]] config, an AccountPool (session-sticky/round-robin selection, cooldowns, 429 quota-vs-throttle classification, per-account refresh locks), the pooled failover loop in the Anthropic adapter (account_uuid body rewrite, oauth-2025-04-20 header, x-shunt-account), and a loopback carve-out on the claude_oauth host guard. - Phase 2: quota-aware proactive rotation that parses anthropic-ratelimit-unified-* headers into per-account QuotaState, a 0.98 switch threshold, model-governing weekly bucket, and soonest-reset preference. - Phase 3: `shunt login claude --name <n>` (imports a refreshable login) and `--long-lived` (stores a `claude setup-token`), a shunt-owned account store at ~/.shunt/accounts/claude/<name>.json, and name-only/empty-list store scanning. Behavior ported from KarpelesLab/teamclaude as the reference. Docs (README, docs/, site guide + config/CLI reference) updated alongside.
amondnet
force-pushed
the
amondnet/claude-multi-account
branch
from
July 12, 2026 18:20
ef86400 to
4b1ee84
Compare
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
- find_executable: search .exe/.cmd/.bat on Windows so "shunt login claude --long-lived" locates an npm-installed Claude CLI - default_accounts_dir / expand_tilde: fall back to USERPROFILE when HOME is unset (Windows), matching the existing auth path helpers - forward_claude_oauth: run scan_accounts() and account_is_static_store_token() on the blocking pool via spawn_blocking, matching the codebase's async file-I/O convention
amondnet
force-pushed
the
amondnet/claude-multi-account
branch
from
July 12, 2026 18:27
de6a0e6 to
922e4db
Compare
Contributor
Author
|
/gemini review |
…e-out Applies validated findings from an internal review pass: - Non-refreshable 401: when a static (token_env / long-lived setup-token) account returns 401 it cannot be refreshed, so it is cooled down and the pool rotates. Previously this branch cooled down silently — every sibling cooldown logs, so a revoked static token cycled invisibly. Now logs a warn. - RefreshRetry non-success: after a successful token refresh, a retry that came back non-401/non-2xx (transient 5xx, or a quota-rejected 429 unrelated to the auth failure) was relayed straight to the client, short-circuiting failover to remaining healthy accounts. Now it cools the account down and continues rotation, mirroring how an initial 5xx/429 rotates; the response still relays only if the pool is exhausted. - Docs: document the loopback carve-out in the claude_oauth base_url guard (plaintext + non-anthropic hosts are accepted for loopback so a local debugging proxy works) across the M8 spec, the Starlight guide, and the configuration reference — all three previously described the HTTPS + anthropic-host check as unconditional. - Test: add unauthorized_static_account_cools_down_and_rotates covering the previously-untested 401 rotation path for static accounts.
Contributor
Author
|
/gemini review |
There was a problem hiding this comment.
Code Review
본 풀 리퀘스트는 Claude 구독 OAuth 계정 풀을 관리하고 세션 스티키 및 모델 인지형 사전 쿼터 로테이션과 반응형 페일오버를 결합한 Anthropic 멀티 계정 로드 밸런싱(M8) 기능을 추가합니다. 코드 리뷰 결과, 비동기 컨텍스트 내 동기식 파일 I/O 수행으로 인한 런타임 워커 스레드 차단 문제, forward_claude_oauth 내 refresh_lock 획득 범위가 너무 넓어 동시 요청이 직렬화되는 병목 문제, RefreshRetry 분기에서 실패 시 조기 반환되어 페일오버 루프가 끊기는 문제, 그리고 취소 안전성(cancellation safety)을 위해 토큰 갱신 작업을 tokio::spawn으로 감싸야 하는 문제가 지적되었습니다.
There was a problem hiding this comment.
Code Review
본 풀 리퀘스트는 Claude 구독 OAuth 계정들을 풀링하여 세션 스티키 및 모델 인지형 사전 회전, 그리고 반응형 페일오버를 지원하는 Anthropic 멀티 계정 로드 밸런싱(M8) 기능을 추가합니다. 이를 위해 AccountPool 상태 관리 클래스와 shunt login claude CLI 명령어가 도입되었으며, 관련 설정 검증 및 테스트 코드가 작성되었습니다. 리뷰 피드백에서는 비동기 토큰 갱신 및 파일 쓰기 작업 시 클라이언트 연결 종료에 따른 취소 안전성(cancellation safety)을 확보하고 락 범위를 최소화하기 위해 tokio::spawn을 통한 독립 태스크 실행을 권장하고 있습니다. 또한, 디렉터리 생성 시 권한 노출 시간(window)을 제거하기 위해 처음부터 0700 권한으로 생성하는 "born-private" 패턴을 적용할 것을 제안합니다.
…etries Addresses review feedback on the claude_oauth failover loop: - refresh_lock scope: the per-account lock was held across the entire per-account attempt — credential resolution, the upstream POST, and the PauseSame back-off sleep (up to 300s) — even though it is only meant to serialize token refreshes. With session affinity this serialized concurrent same-account requests behind an unrelated retry-after wait. The lock is now held only around the two operations that can refresh + write the credential back (resolve_claude_account's refresh-on-read and the explicit force_refresh), and released across the POSTs and the sleep. - RefreshRetry non-success: route the refreshed retry response through accounts::classify() (like the initial response) instead of relaying any non-401/non-2xx status straight to the client. A 5xx or quota-rejected 429 now cools the account down and fails over to the remaining accounts; a non-429 4xx (client error) still relays without a wrongful cooldown. - resolve_claude_account: run the synchronous claude_store::account_uuid file read on the blocking pool via spawn_blocking, matching the other async file-I/O call sites.
Avoids the brief window where the account directory sat at the umask default between create_dir_all and the follow-up chmod on a multi-user host. Uses DirBuilderExt::mode so the directory is 0700 from creation, and drops the now-unused set_private_directory_permissions helper.
Adds integration coverage for the 401 -> force_refresh -> retry branch of forward_claude_oauth, exercised end-to-end through a mock OAuth refresh endpoint and a name-only store account (which also covers the spawn_blocking account_uuid read and the static-token classification): - refresh_retry_refreshes_then_succeeds_on_401: 401 triggers a refresh; the retry with the refreshed token succeeds and relays. - refresh_retry_non_success_rotates_to_next_account: a 5xx on the refreshed retry cools the account down and fails over to the next account. Adds a SHUNT_CLAUDE_TOKEN_URL override to ClaudeAuthStore::new so the refresh endpoint can be pointed at a mock in tests (unset in production).
Add four integration tests for the previously-untested forward_claude_oauth failover arms: - unresolvable_account_cools_down_and_rotates: an unset token_env fails to resolve, so the account is cooled down and the pool rotates. - server_error_rotates_and_cools_down_the_failing_account: a 5xx maps to Rotate via the non-429 fixed-cooldown sub-branch. - refresh_retry_still_unauthorized_cools_down_and_rotates: refresh succeeds but the refreshed token is still 401, so the account is cooled down and rotated instead of relaying the second 401. - all_accounts_unresolvable_returns_bad_gateway: when every account fails to resolve, the pool never reaches an upstream and surfaces a 502.
amondnet
marked this pull request as ready for review
July 13, 2026 01:24
Greptile SummaryThis PR adds M8 — Anthropic multi-account load balancing across three phases: a session-sticky / round-robin account pool with quota-aware rotation, per-account cooldowns and 429/5xx classification, and a
Confidence Score: 4/5The failover loop, quota rotation, and credential store are well-structured with good test coverage; the main concern is the test helper that uses a different hash than production, meaning session-stickiness cooldown-bypass assertions may pass without being exercised. The core multi-account logic — pooled failover, quota-aware rotation, credential provisioning, and file permissions — is sound and matches the spec. One security gap introduced by this PR (no validation on SHUNT_CLAUDE_TOKEN_URL) is minor because it requires environment access. The more notable concern is that the integration test helper computes session-to-account mappings with a different algorithm than production, so the cooldown-bypass paths those tests are meant to verify may be passing without actually being exercised. tests/multi_account.rs — the session_id_for_account helper uses a different hash function than stable_session_index in production, making the session-stickiness tests unreliable. src/auth/claude_auth.rs — SHUNT_CLAUDE_TOKEN_URL accepts any URL without host or scheme validation. Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant C as Client
participant G as Gateway (forward_claude_oauth)
participant P as AccountPool
participant SA as Anthropic (Account A)
participant SB as Anthropic (Account B)
C->>G: POST /v1/messages (x-claude-code-session-id: sess-1)
G->>P: select_order(provider, accounts, session_id, model)
note over P: SHA-256(sess-1) mod N → sticky to A<br/>Checks cooldown + near_quota
P-->>G: [A, B]
G->>G: refresh_lock(A).lock()
G->>G: resolve_claude_account(A)
G-->>G: Bearer A-token
G->>SA: POST /v1/messages (Bearer A)
SA-->>G: 429 + anthropic-ratelimit-unified-5h-status: rejected
G->>P: note_quota(A, headers)
G->>P: cooldown(A, retry_after or 60s)
note over G: FailoverAction::Rotate → continue loop
G->>G: refresh_lock(B).lock()
G->>G: resolve_claude_account(B)
G-->>G: Bearer B-token
G->>SB: POST /v1/messages (Bearer B)
SB-->>G: 200 OK
G->>P: mark_healthy(B)
G-->>C: 200 OK (x-shunt-account: B)
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant C as Client
participant G as Gateway (forward_claude_oauth)
participant P as AccountPool
participant SA as Anthropic (Account A)
participant SB as Anthropic (Account B)
C->>G: POST /v1/messages (x-claude-code-session-id: sess-1)
G->>P: select_order(provider, accounts, session_id, model)
note over P: SHA-256(sess-1) mod N → sticky to A<br/>Checks cooldown + near_quota
P-->>G: [A, B]
G->>G: refresh_lock(A).lock()
G->>G: resolve_claude_account(A)
G-->>G: Bearer A-token
G->>SA: POST /v1/messages (Bearer A)
SA-->>G: 429 + anthropic-ratelimit-unified-5h-status: rejected
G->>P: note_quota(A, headers)
G->>P: cooldown(A, retry_after or 60s)
note over G: FailoverAction::Rotate → continue loop
G->>G: refresh_lock(B).lock()
G->>G: resolve_claude_account(B)
G-->>G: Bearer B-token
G->>SB: POST /v1/messages (Bearer B)
SB-->>G: 200 OK
G->>P: mark_healthy(B)
G-->>C: 200 OK (x-shunt-account: B)
Reviews (5): Last reviewed commit: "chore(anthropic): apply AI review sugges..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 26 files
Architecture diagram
sequenceDiagram
participant Client as Claude Code Client
participant Proxy as Shunt Proxy (anthropic adapter)
participant Pool as AccountPool
participant Auth as ClaudeAuthStore
participant Store as Claude Account Store
participant Upstream as Anthropic API
Client->>Proxy: POST /v1/messages (Authorization, x-api-key, x-claude-code-session-id, model)
Note over Proxy: auth = "claude_oauth" → forward_claude_oauth
alt accounts list empty
Proxy->>Store: scan_accounts() [blocking pool]
Store-->>Proxy: list of AccountConfig (name, uuid)
end
Proxy->>Pool: select_order(provider, accounts, session_id, model)
Note over Pool: session-sticky hash or round-robin,<br/>quota/cooldown aware,<br/>never fails closed
loop for each account in order
Pool-->>Proxy: next account index
Proxy->>Proxy: resolve_claude_account(account)
alt token_env set
Proxy->>Proxy: read env var → access token
else credentials path set
Proxy->>Auth: get_valid_access_token()
Auth->>Auth: read file, refresh if near expiry
Auth-->>Proxy: access token
else name-only (store account)
Proxy->>Auth: get_valid_access_token() (from store file)
Auth-->>Proxy: access token
Proxy->>Store: account_uuid(name) [blocking pool]
Store-->>Proxy: UUID if present
end
alt resolution fails
Proxy->>Pool: cooldown(5 min)
Proxy->>Proxy: log warning, continue to next account
end
Proxy->>Proxy: build outbound headers
Note over Proxy: strip client auth/x-api-key<br/>set Authorization Bearer <token><br/>append anthropic-beta: oauth-2025-04-20<br/>preserve x-claude-code-session-id
Proxy->>Proxy: rewrite metadata.user_id.account_uuid if configured
Proxy->>Upstream: POST /v1/messages (outbound headers, body)
alt transport error
Proxy->>Pool: cooldown(30 sec)
Proxy->>Proxy: continue to next account
else response received
Proxy->>Pool: note_quota(headers) – parse 5h/7d/7d_oi headers
Proxy->>Proxy: classify(status, headers)
alt Relay (2xx)
Proxy->>Pool: mark_healthy()
Proxy-->>Client: 200 + response + x-shunt-account header
Note over Proxy: return immediately
else Rotate (quota-429 or 5xx)
Proxy->>Pool: cooldown(retry-after or 30s)
Proxy->>Proxy: store as last_response, continue
else PauseSame (plain 429)
Proxy->>Proxy: sleep(retry-after, max 300s)
Proxy->>Upstream: retry same account
alt retry OK
Proxy->>Pool: mark_healthy()
Proxy-->>Client: 200 + response
else retry fails
Proxy->>Pool: cooldown(30s)
Proxy->>Proxy: store last response, continue
end
else RefreshRetry (401)
alt token_env or long-lived setup token (static)
Proxy->>Pool: cooldown(5 min)
Proxy->>Proxy: log warn, store as last, continue
else credentials file or store-managed refreshable
Proxy->>Pool: refresh_lock(provider, account) – scope: only refresh
Proxy->>Auth: force_refresh()
Auth->>Auth: refresh token via OAuth endpoint<br/>(SHUNT_CLAUDE_TOKEN_URL)
Auth-->>Proxy: new access token
Proxy->>Upstream: retry same account with new token
alt retry 2xx
Proxy->>Pool: mark_healthy()
Proxy-->>Client: success + response
else retry 5xx/429
Proxy->>Pool: cooldown appropriately
Proxy->>Proxy: store last response, continue
else retry still 401
Proxy->>Pool: cooldown(5 min)
Proxy->>Proxy: store last, continue
end
end
end
end
end
alt any upstream responses obtained
Proxy-->>Client: last upstream status + body
else all accounts failed before request sent
Proxy-->>Client: 502 Bad Gateway (no upstream response)
end
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Apply internal review-loop findings on the claude_oauth failover path: - Add a tracing::warn! to the FailoverAction::Rotate arm of forward_claude_oauth. It is the most common failover trigger (quota-rejected 429 or any 5xx) yet was the only arm that cooled down and rotated with no operator-visible log, unlike every sibling arm. - Make the nested classify() match on the refreshed retry exhaustive (Rotate | PauseSame | RefreshRetry) instead of a `_` catch-all, so a new FailoverAction variant forces a decision here — without a panic arm. - Extract a shared SETUP_TOKEN_KIND const so the setup-token writer (claude_store::store_setup_token) and reader (account_is_static_store_token) cannot silently drift on the literal. Tests: - pause_same_retry_succeeds_and_relays_without_rotating: the PauseSame retry-success sub-path (429 → sleep → 200 relayed, no rotation). - static_setup_token_account_cools_down_without_refreshing: a store-based setup token 401s, cools down, and rotates without ever calling the refresh endpoint (exercises account_is_static_store_token's store path).
…t/claude-multi-account
…t/claude-multi-account
There was a problem hiding this comment.
All reported issues were addressed across 7 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds M8 — Anthropic multi-account load balancing with quota-aware rotation. Implements
auth = "claude_oauth"provider mode across three phases in a single commit:[[providers.*.accounts]]config and anAccountPoolwith session-sticky (x-claude-code-session-id) / round-robin selection, cooldowns, per-account refresh locks, and a 429 quota-vs-throttle classifier. The Anthropic adapter's pooled failover loop does per-account credential resolution, rewritesmetadata.user_id.account_uuidin the request body, injects theoauth-2025-04-20header, setsx-shunt-accounton the response, and relays the last upstream response on pool exhaustion. Also adds a loopback carve-out to theclaude_oauthbearer-leak host guard so local mocks/proxies work while remote hosts stay pinned tohttps+anthropic.com.anthropic-ratelimit-unified-{5h,7d,7d_oi}-{utilization,reset}and-statusheaders into per-accountQuotaState, with a 0.98 switch threshold and a model-governing weekly bucket (Fable →7d_oi, else7d), preferring the soonest-resetting weekly quota. Keeps session stickiness; only rotates off a near-quota or cooled account; never fails closed.shunt login claude --name <name>imports a refreshable~/.claude/.credentials.jsonlogin;--long-livedrunsclaude setup-tokenand stores the resulting token hidden. Both write to a shunt-owned account store at~/.shunt/accounts/claude/<name>.json(dir0700/ file0600), with name-only/empty-list store scanning.Behavior is ported from
KarpelesLab/teamclaudeas the reference implementation (no private mirror repos referenced).Docs updated alongside:
README.md,docs/m8-anthropic-multi-account.md,docs/comparison.md, and the docs site guide + config/CLI reference.Note for reviewers: Phase 3 (
shunt login claude/ account store, credential file permissions, hidden long-lived token storage) has not yet been reviewed by an approver — please give it a security + correctness pass. Storm-control (per-account concurrency ramp) and per-account metrics are intentionally out of scope here and tracked as follow-ups.Milestone / spec
M8 —
docs/m8-anthropic-multi-account.mdChecklist
cargo buildpassescargo testpasses (new behavior is covered; tests run without network/loopback where possible) — 237 tests greencargo clippy --all-targets -- -D warningscleancargo fmt --all --checkcleandocs/updated if this change deviates from itREADME.md/site/as applicable (wiki/is generated; don't hand-edit)Notes for reviewers
shunt login claudeaccount provisioning + account store) has been reviewed for security and correctness as part of this change, but has not yet been signed off by an approver — please double-check credential file permissions (0700/0600) and the hidden long-lived token storage path.Summary by cubic
Adds Anthropic multi-account load balancing with quota-aware rotation and a
shunt login claudeCLI for provisioning. Narrows refresh locks to refresh paths, hardens failover, adds a private on‑disk account store with tests, and capturesaccount_uuidduring login for accurate request metadata.New Features
auth = "claude_oauth"pool: session‑sticky onx-claude-code-session-id, round‑robin otherwise; cooldowns; refresh locks held only around refresh/refresh‑on‑read; loopback carve‑out in the host guard; account scans,account_uuidreads, and static‑token checks run on the blocking pool.5h/7d/7d_oiheaders and switches at 0.98; weekly bucket per model (Fable →7d_oi, else7d); prefers the soonest reset.metadata.user_id.account_uuid; sendsoauth-2025-04-20; setsx-shunt-account; after a successful refresh, non‑2xx/non‑401 retry responses are classified (5xx/429 cools and rotates) instead of relayed;Rotatelogs;PauseSameretry success relays without rotation. Theresponsesadapter accepts a Claude OAuth bearer.shunt login claude --name <name>imports a refreshable login and records itsaccount_uuid;--long-livedstores a hiddenclaude setup-token. Accounts live at~/.shunt/accounts/claude/<name>.jsonwith born‑private dir (0700) and files (0600); Windows findsclaudevia.exe/.cmd/.batand falls back toUSERPROFILE.SHUNT_CLAUDE_TOKEN_URLredirects refresh to a mock in tests.Migration
anthropicconfigs.auth = "claude_oauth"and add[[providers.<name>.accounts]]entries or runshunt login claude --name <name>; leaveaccountsempty to auto‑scan the store.--long-livedrequires theclaudeCLI on PATH.Written for commit 15feb25. Summary will update on new commits.