Skip to content

feat: opt-in persistent browser profiles (CAMOFOX_PERSISTENT_PROFILES) - #6525

Closed
juniperbevensee wants to merge 4 commits into
jo-inc:masterfrom
NimbleCoOrg:feat/persistent-profiles
Closed

feat: opt-in persistent browser profiles (CAMOFOX_PERSISTENT_PROFILES)#6525
juniperbevensee wants to merge 4 commits into
jo-inc:masterfrom
NimbleCoOrg:feat/persistent-profiles

Conversation

@juniperbevensee

Copy link
Copy Markdown

Problem

Sessions are created with browser.newContext({ storageState }). storageState persists cookies + localStorage but not IndexedDB / Service Workers / Cache Storage. Sites that keep their auth in IndexedDB — Telegram Web (WebK) is a clear example — therefore can't be restored from a persisted session: after a session reaps and a new context is created, the user is logged out even though storage-state.json was reloaded.

Change

Opt-in persistent profiles. When CAMOFOX_PERSISTENT_PROFILES=1, a session launches a dedicated context via firefox.launchPersistentContext(userDataDir, …) instead of browser.newContext(…), keyed by sha256(userId) under CAMOFOX_USERDATA_DIR (default ~/.camofox/userdata). The full on-disk Firefox profile — IndexedDB included — then survives session reap/recreate (and, if the dir is on a volume, container recreate).

  • Reuses the shared Xvfb display and the same camoufox launchOptions as the pooled browser, so persistent contexts get identical stealth.
  • storageState injection is skipped for persistent sessions (the on-disk profile supersedes it).
  • Opt-in because each persistent identity is its own browser process; with the flag off, the default shared-browser / ephemeral-context model is completely unchanged.

Two new config keys (lib/config.js): persistentProfiles, userDataDir.

Verification

Built the image, ran two containers (flag on / flag off), and for each: opened a tab, wrote a key to IndexedDB, closed the session, recreated it for the same userId, and read the key back.

Mode IndexedDB value after session recreate
CAMOFOX_PERSISTENT_PROFILES=1 returned (survived)
default MISSING (lost)

Notes / trade-offs

  • Persistent contexts are one browser process per profile (heavier) — hence opt-in.
  • Per-session granularity (rather than a global env flag) would be a natural follow-up if there's interest.

juniperbevensee and others added 4 commits June 25, 2026 16:06
Sessions are created with browser.newContext({ storageState }), which persists
cookies + localStorage but NOT IndexedDB / Service Workers / Cache Storage. Sites
that keep auth in IndexedDB (e.g. Telegram Web) cannot be restored from
storageState, so their logins do not survive session recreation.

Add an opt-in mode: when CAMOFOX_PERSISTENT_PROFILES=1, a session launches a
dedicated context via firefox.launchPersistentContext(userDataDir, ...), keyed by
sha256(userId) under CAMOFOX_USERDATA_DIR. The full on-disk Firefox profile
(IndexedDB included) then survives session reap/recreate. Reuses the shared Xvfb
display and the same camoufox launchOptions as the pooled browser, so persistent
contexts get identical stealth. storageState injection is skipped for persistent
sessions (the on-disk profile supersedes it).

Opt-in because each persistent identity is its own browser process; the default
shared-browser / ephemeral-context model is unchanged when the flag is off.

Verified: with the flag on, a value written to IndexedDB survives closing the
session and recreating it (reloaded from userDataDir); with the flag off, the
same value is MISSING after recreate.
…able

Docker port-forwarding (-p 127.0.0.1:6081:6080) proxies to the container IP, not
container loopback — so binding websockify to 127.0.0.1 INSIDE the container made
the published port dead. Bind 0.0.0.0 inside; the host -p mapping remains the
security boundary (still loopback-only + SSH-tunnel access). Fixes the dead noVNC
that blocked the browser-login VNC handoff.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A closed tab left agents unable to recover: the hermes browser tool reopens a
fresh tab with 'open about:blank', but validateUrl rejected any non-http(s)
scheme, so tab-create 400'd ('Blocked URL scheme: about:') and the agent could
never get a working surface back. Observed live — an agent told to stop had its
tab closed, then spun retrying a dead tab / failing to reopen.

about:blank is the canonical empty page (a freshly created page already sits on
it, and the server itself does page.goto('about:blank') elsewhere); the scheme
guard is meant to block file:/chrome:/data:, not the inert blank page.

Moved validateUrl into lib/request-utils.js (where the sibling error classifier
already lives) so it is unit-testable, added the about:blank allowance, removed
the now-dead ALLOWED_URL_SCHEMES const from server.js. New unit test covers
about:blank/http/https allowed, file/chrome/data/about:config blocked, garbage
rejected. Verified all cases green.

Co-authored-by: Juniper Bevensee <juniperbevensee@users.noreply.github.com>
… + Dockerfile.simple

Two production fixes lived only as uncommitted changes on the Mini's build
checkout, at risk of being lost on any rebuild-from-git:

- persistentWindowSize(): pins the persistent browser window to the display
  size (CAMOFOX_WINDOW / VNC_RESOLUTION, default 1280x720) so pages don't
  render at the camoufox fingerprint's screen size (e.g. 2560x1440) and
  overflow the VNC view. Applied to both launch paths.
- Dockerfile.simple: the ARM (aarch64) build recipe that actually builds the
  Mini's camofox-browser:persistent image — copies a prebuilt dist/camoufox
  zip instead of downloading, for reproducible local builds. dist/ stays
  gitignored (build input, not source).

The fork's feat/persistent-profiles is now the single source of truth for the
deployed image: persistent-profiles + VNC_BIND + about:blank + window-size +
the build recipe. Reconciles the divergence that made rebuilds drop fork work.
@skyfallsin

Copy link
Copy Markdown
Contributor

thanks for putting this together. I’m thinking a little more about the persistent-profile approach and whether it’s the right boundary for this particular use case before committing either way. stay tuned.

@skyfallsin skyfallsin added the considering Under consideration; no decision yet label Jul 20, 2026
@skyfallsin

Copy link
Copy Markdown
Contributor

Thanks for the persistent-profile implementation and the clear verification work.

Since this PR was opened, the built-in persistence plugin gained opt-in IndexedDB storage-state support. This covers the stated login-persistence use case without adding one Firefox process and profile directory per user:

{
  "plugins": {
    "persistence": {
      "indexedDB": true
    }
  }
}

The branch is now conflicting and also includes unrelated VNC, URL-validation, and Docker changes. A persistent-profile mode would also need a redesigned reset/delete contract so DELETE /sessions/:userId/storage_state clears the on-disk Firefox profile, not only the storage-state file.

I am closing this PR as superseded by the current persistence path. Please open a new focused proposal if full Firefox-profile persistence is still needed for a case that IndexedDB storage state cannot support.

@skyfallsin skyfallsin closed this Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

considering Under consideration; no decision yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants