feat: add POST /tabs/:tabId/set_input_files endpoint - #4164
feat: add POST /tabs/:tabId/set_input_files endpoint#4164MrReasonable wants to merge 96 commits into
Conversation
- createTabState gets navigateAbort AbortController field - navigateCurrentPage races page.goto() with abort signal - DELETE /tabs/:tabId calls abort() before safePageClose, cancelling in-flight navigation immediately instead of waiting 30s timeout - Add camofox_snapshot_bytes histogram for snapshot size tracking - 6 new abort tests passing
…tion, fixes Backported from camofox-browser v1.5.2: - Lazy Prometheus metrics: off by default, enable with PROMETHEUS_ENABLED=1. Noop stubs when disabled — no prom-client import overhead. Fixes OpenClaw install blocking on optional dep. - Extract extractPageImages to lib/images.js (OpenClaw scanner isolation) - Extract actionFromReq + classifyError to lib/request-utils.js - buildRefs timeout leak: clearTimeout in both success and error paths - YT transcript session cleanup: close phantom __yt_transcript__ context when all tabs are gone - Session cleanup on empty tab groups: tab reaper now closes sessions with zero remaining tabs + triggers browser idle shutdown - Horizontal scroll: mouse.wheel now supports left/right directions - Delete tab/group: accept userId from query string or body - plugin.ts: proc.kill() on startup timeout, screenshot Content-Type guard (returns text error instead of base64-encoded JSON on non-image response) - 2 new test files: sessionCleanup.test.js, updated screenshotToolResult.test.js
Race 1 (YT transcript): browserTranscript cleanup counted tabGroups which was always 0 (YT pages aren't registered in tabGroups). First concurrent request to finish would close the context, killing other requests' pages. Fix: use context.pages() to check actual live pages before closing. Race 2 (tab reaper / session expiry): context.close() fired without await, then session deleted from map. A request that already got the session ref (between getSession return and newPage) would use a closing context. Fix: set session._closing = true before teardown; getSession() treats _closing sessions as dead and creates a new one. Both fixes also applied to the session expiry timer (same pattern). 16 unit tests added covering _closing flag, getSession skip, YT concurrent cleanup, and session expiry sentinel.
When a click/navigate/open_url times out (e.g., Cloudflare holding the connection for 30s), the Decodo proxy session becomes poisoned — all subsequent requests through the same BrowserContext hang. Previously, timeouts only tracked per-tab consecutive counts without destroying the session, so recovery tabs inherited the poisoned proxy. Now navigation-related timeouts (click, navigate, open_url) destroy the entire user session, so the next request gets a fresh BrowserContext with a fresh proxy session. Non-navigation timeouts (type, scroll) still use the per-tab consecutive timeout tracking. Root cause: WWDC cron clicked a MacRumors Cloudflare-protected link, the proxy session got poisoned, recovery tab's prewarm to google.com also timed out on the same session, browser agent gave up entirely.
…act, crash reporter, CI improvements New features from upstream: - OpenAPI spec auto-generated from JSDoc + swagger-stripey docs at /docs (jo-inc#78) - Opt-in session tracing via Playwright (jo-inc#68) - Structured extract endpoint with JSON Schema + x-ref hints (jo-inc#70) - session:destroying event for persistence checkpoint (jo-inc#75) - Anonymized crash/frustration reporter with GitHub App auth - GitHub Actions CI workflow for unit tests Also: compressed fox.png, added README badges, GitHub Sponsors funding link
Upstream refactor moves crash reporter credentials to camofox.config.json. Watchdog now includes active sessions/tabs/URLs in event-loop stall reports. jo-browser config uses its own GitHub App (Jo Browser Crash Reporter, app ID 3505356) targeting jo-inc/jo-browser.
Drifts >120s are OS sleep/hibernate, not real event-loop stalls. Skip them and suppress the next 5 ticks to avoid post-wake jitter false positives (like the 6s stall in #83). Fixes #82, #83
# Conflicts: # lib/reporter.js # openapi.json
Merge artifact from v1.7.4 sync introduced duplicate 'let lastHeapUsed' at lines 976 and 1004 in reporter.js. Node.js SyntaxError on startup caused all 3 Fly machines to crash-loop and exhaust retries. All machines have been stopped since ~05:11 UTC, breaking browser-dependent cron jobs and use_browser tool calls.
deploy.yml had zero dependency on CI — deployed immediately on push regardless of whether tests passed. This is how the duplicate let declaration shipped to prod and took down all 3 machines for 14+ hours. Now deploy requires: syntax check of all JS files + reporter unit tests.
- Install @sentry/node, init from SENTRY_DSN env var - lib/sentry.js: isolated module (no process.env, scanner-safe) - lib/config.js: add sentryDsn config key - server.js: capture 500s in sendError, uncaughtException, unhandledRejection - setupExpressErrorHandler after all routes, flush on graceful shutdown - fly.toml: add SENTRY_DSN for jo-browser project - StaleRefsError (422) filtered out via beforeSend
Root cause: .internal DNS bypassed Fly Proxy — all traffic pinned to one machine. 11 concurrent BrowserContexts × ~300MB = 3.3GB on 4GB → OOM. Changes: - MAX_SESSIONS=10 (was 50 default — way too high for 4GB) - concurrency type connections soft=25 (was requests soft=8 — wrong metric) - Memory admission control: 503 + fly-replay at >90% RAM - Memory pressure eviction: 30s loop, multi-evict until <80% RAM - Session overflow redirect: fly-replay when >ceil(MAX/3) sessions - tabNotFoundResponse: 410 Gone for local tabs after browser restart - SSL error handling: 502 recoverable:false for SEC_ERROR - Sentry: skip capture for intentional 503s
Dead context, timeouts, proxy errors, nav aborts, and tab lifecycle errors are expected operational behavior — not bugs. They were flooding Sentry with noise issues, hiding real problems. Added classifyError patterns for NS_ERROR_ABORT, tab deleted navigation abort, and page crash. Expanded beforeSend to drop operational errors.
…, timeout tuning Fix 1: Drain tab locks BEFORE closing browser context in closeSession(). Queued operations now get clean 'Tab destroyed' (410) instead of cascading 'Target page closed' (500) errors. Dead-context errors in handleRouteError now return 503 with session_expired code — the root cause (proxy/timeout) is still reported, but cascade errors no longer flood Sentry. Fix 2: Transparent retry on proxy/timeout errors during navigation. When page.goto fails with NS_ERROR_PROXY_FORBIDDEN or timeout, destroy the session, get a fresh proxy, and retry once before failing to caller. Applied to POST /tabs (create with URL), POST /tabs/:tabId/navigate, and POST /tabs/open. Uses existing context rotation pattern. Fix 3: Increase goBack timeout from 10s to 20s. Back navigation uses browser cache and shouldn't need aggressive timeouts. 10s was too short for complex SPA re-renders.
# Conflicts: # camofox.config.json # package-lock.json # server.js
…stration-names Fix OpenClaw factory tool registration names
|
Thanks for putting this together! I've been testing file uploads and found some edge cases that we should consider handling here to make this endpoint more robust. The Issue:
Proposed Improvement: We can enable file chooser interception and attempt to click the element to trigger the chooser event, falling back to direct Here is a conceptual example of how we can update the logic: // Inside withTabLock, before interacting:
await tabState.page.setFileChooserInterceptedBy(true);
const fileChooserPromise = tabState.page.waitForEvent('filechooser', { timeout: 3000 }).catch(() => null);
// Try clicking the visible dropzone wrapper or the input itself (not the hidden input if possible)
// (We could potentially auto-detect the nearest clickable ancestor or accept a dropzone selector)
await locator.click({ force: true }).catch(() => {});
const fileChooser = await fileChooserPromise;
if (fileChooser) {
// Intercept the filechooser and set the files via the chooser element.
// Since the server FileChooser might lack .accept(), use element().setInputFiles
await fileChooser.element().setInputFiles(resolved, { timeout: 10000 });
} else {
// Fallback: Direct target.setInputFiles for plain inputs if no chooser was triggered
await locator.setInputFiles(resolved, { timeout: 10000 });
}This ensures we intercept the trusted chooser event for modern upload widgets while preserving the direct fallback (Plan A) for simpler inputs. (Also referencing #3115 as this relates to similar input interaction logic.) Let me know what you think! |
|
Thanks @SSOURABH58 — this is a genuinely good catch. You're right that direct I've pushed a change that adds this as an opt-in second path rather than replacing the default (68ba18a):
A couple of implementation notes vs. the sketch:
Tests cover both dropzone paths (selector + ref) against a react-dropzone-style page, sandbox enforcement on the new path, and the no-chooser case. One thing on the #3115 reference: it already prototypes this file-chooser approach with the correct APIs ( |
…#5067) # Conflicts: # tests/unit/config.test.js
# Conflicts: # server.js
Adds a new route for programmatically attaching files to <input type="file"> elements via Playwright's setInputFiles. File paths are restricted to a configurable uploads directory (CAMOFOX_UPLOADS_DIR, default /home/node/cv-uploads) to prevent arbitrary file exfiltration. Supports both ref-based and CSS selector targeting, with auto-refresh of stale refs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers selector and ref-based file attachment, path-traversal rejection, missing ref/selector validation, and 404 for unknown tabs. Global setup now creates a temp uploads directory and passes CAMOFOX_UPLOADS_DIR to the test server so the sandbox restriction is exercisable in CI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds an opt-in file-chooser path to POST /tabs/:tabId/set_input_files for custom dropzone widgets (e.g. react-dropzone) that read the file from the trusted filechooser event rather than the hidden <input>. Direct setInputFiles on the input silently no-ops for these widgets. - New dropzoneRef / dropzoneSelector params target the visible dropzone element. The server clicks it, intercepts the resulting filechooser, and feeds it the same uploads-dir-sandboxed paths, so the security boundary is unchanged. - Direct ref/selector path is untouched and remains the default. - Returns `via: "input" | "filechooser"` so callers can see which mechanism ran; emits the same via/dropzone fields on the plugin event. - Returns 400 (not 500) when the dropzone click opens no chooser, with a message pointing plain-input callers back to ref/selector. - Regenerates openapi.json, which was missing the endpoint entirely (the original commit never re-ran generate-openapi). Tests: adds a react-dropzone-style test page and e2e coverage for the dropzone path via selector and ref, sandbox enforcement on that path, the no-chooser 400, and the via:input result. Full suite (10 tests) passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses an adversarial review of the endpoint: Security - Sandbox containment is now checked on the realpath of each path, not the pre-resolved string. A symlink planted inside the uploads dir can no longer smuggle an out-of-tree target (e.g. /etc/passwd) past the allowlist, on either the direct or the file-chooser sink. The allowlist root is itself realpath'd once (the mount may be a symlink). Functionality - Reject directories and other non-regular files (fs.access(R_OK) succeeds on a dir, which previously slipped through to a Playwright 500). - Dropzone chooser wait (15s) now exceeds the click timeout (10s) so a slow click that opens the chooser can't lose a race to an equal-length timer and report a false "no chooser". - Narrow the dropzone force-click fallback to blocked/timed-out clicks only; strict-mode violations, detached nodes and closed targets now surface with their real diagnostic instead of being retried with force. - Require userId (was silently returning 404 tab-not-found). Code quality - Extract timeout constants; move CAMOFOX_UPLOADS_DIR into lib/config.js; log client (4xx) failures at info level instead of error. Tests - Add coverage: symlink escape, directory rejection, missing userId, dropzone precedence over a direct target, and that a direct setInputFiles on the visible div fails (justifying the dropzone path). Harden the dropzoneRef test so it fails loudly instead of silently falling back to the selector (the dropzone now carries role=button so it appears in the snapshot). Full suite: 15/15 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update pinned Camoufox version (Firefox 152) across Makefile and both Dockerfiles. Upstream marked the FF135 build unsuitable for modern anti-bot systems. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Thanks for the upload work. This PR’s user-facing need is now covered by the production upload endpoint: It supports upload-root containment, symlink-safe path checks, direct file inputs, ref/selector triggers, native filechooser fallback, and post-upload ref refresh. The branch is now conflicting and contains many unrelated server, Docker, auth, and infrastructure changes. Maintaining a second upload route would duplicate the current API and increase drift risk. I am closing this PR as superseded. If there is a specific upload case that |
Why
AI agents using camofox-browser often need to apply for jobs or submit documents on behalf of users. Many job sites and portals use
<input type="file">elements for CV/resume uploads — but browsers intentionally block JavaScript from setting file inputs, so a standardtypeorclickaction cannot attach a file. This endpoint fills that gap by exposing Playwright's privilegedsetInputFileschannel via the REST API.What
Adds
POST /tabs/:tabId/set_input_filesto programmatically attach files to<input type="file">elements.ref(preferred, with automatic stale-ref refresh) or a CSSselectorto target the file inputCAMOFOX_UPLOADS_DIR, default/home/node/cv-uploads) to prevent the endpoint being used to read arbitrary files from the containertab:set_input_filesplugin event consistent with the rest of the interaction surfaceTest plan
refnorselectorwith 400npm run test:e2e)🤖 Generated with Claude Code