hackbrowser worker bypasses Copilot token exchange + required headers → crawl hangs on Copilot/GHE models

[badchars]

Summary

When hackbrowser crawls using a GitHub Copilot model (including GitHub Enterprise / GHE), the crawl hangs / finishes with no plan. The crawler runs as a separate worker subprocess with its own copy of the Copilot request handling, and that copy sends the raw GitHub OAuth token directly with missing Copilot headers — so every planner call to the Copilot API is rejected (403) and the crawl can never make progress.

This is the same bug class that was fixed for the main chat provider (Copilot token exchange + required headers + refresh + 403 retry), but the hackbrowser worker has a separate, unfixed duplicate of that logic.

Root cause

Two places talk to the Copilot API, and only one was fixed:

1. Main process / chat — packages/cyberstrike/src/plugin/copilot.ts. Already fixed: exchanges the GitHub OAuth token for a short-lived Copilot session token at copilot_internal/v2/token, sends it as Bearer, adds the headers Copilot validates (Copilot-Integration-Id: vscode-chat, Editor-Version, Editor-Plugin-Version, a GitHubCopilotChat User-Agent, X-GitHub-Api-Version), refreshes before expiry, and retries once on 403.

2. Hackbrowser worker (subprocess) — packages/cyberstrike/src/hackbrowser-subprocess/hackbrowser-worker.ts. Not fixed. Its Copilot branch:

   if (desc.npm.includes("github-copilot") && desc.copilotToken) {
     const token = desc.copilotToken
     ...
     headers.set("Authorization", `Bearer ${token}`)
     headers.set("User-Agent", `cyberstrike/${CYBERSTRIKE_VERSION}`)
     headers.set("Openai-Intent", "conversation-edits")
     ...
   }

   • desc.copilotToken is the raw GitHub OAuth token — set in provider.ts (getModelDescriptor) as copilotToken: auth.refresh. No exchange.
   • Missing the integration/editor headers the Copilot API validates (esp. Copilot-Integration-Id).
   • Wrong User-Agent.
   • No refresh, no 403 retry.

Result: api.githubcopilot.com rejects the request (403), the planner gets no usable response, and the crawl stalls / returns zero endpoints.

Why it happens even after the chat fix

The worker is a separate subprocess (Playwright isolation) and can't reuse the main process's live auth/fetch logic, so it carries its own copy of the Copilot request handling. The chat fix touched only copilot.ts; the worker's copy was never updated. This is a copy-drift bug — two implementations of the same thing, only one maintained.

Proposed fix

Don't just copy the fix again (that recreates the drift). Extract the Copilot session logic into a shared module and have both call sites use it:

1. New shared module (e.g. packages/cyberstrike/src/provider/copilot-session.ts):
   • constants (integration id, editor/plugin version, user-agent, api version)
   • exchangeCopilotToken(githubToken, apiBase) with per-token cache + refresh (60s before expires_at)
   • copilotHeaders(sessionToken, { vision }) helper
2. Refactor copilot.ts to use the shared module (no behavior change).
3. Update the worker (hackbrowser-worker.ts) to use the shared module: exchange the raw token, send the session token as Bearer with the correct headers, refresh on expiry (crawls can exceed the ~30 min token lifetime), and retry once on 403.
4. Pass the enterprise domain to the worker so the GHE exchange endpoint (https://api.<enterprise-domain>/copilot_internal/v2/token) can be derived; today only the raw token crosses the IPC boundary.

After this, a single Copilot session module is the source of truth, and future Copilot fixes apply to both chat and hackbrowser automatically.

Impact

• Hackbrowser is unusable with any GitHub Copilot model (github.com and GHE) — the crawl hangs or captures nothing.
• Affects users whose default/hackbrowser model is a Copilot model.

Related

• The main-provider Copilot token-exchange + 403-retry fix (chat) — this issue ports the same behavior to the crawler worker via a shared module.

[badchars]

Fixed and verified.

The hackbrowser worker now exchanges the GitHub OAuth token for a short-lived Copilot session token and sends the integration/editor headers Copilot validates (shared copilot-session.ts, used by both the chat provider and the crawler worker), with retry-on-403 and enterprise-domain threading.

Verification: full crawls completed over GitHub Copilot on both claude-opus-4.8 and gpt-5.6-luna — no more 403 "Forbidden" / "reauthenticate", the crawl reaches depth and completes naturally. The only remaining hackbrowser limitation is crawl depth on button/router-nav SPAs, tracked separately in #120.

On stage (commits 7ce104403 and the shared copilot-session.ts refactor); ships to latest with the stage → main sync.