Skip to content

Latest commit

 

History

History
163 lines (145 loc) · 8.84 KB

CLAUDE.md

File metadata and controls

163 lines (145 loc) · 8.84 KB

netpause — context for a fresh session

A Flutter app that browses a configured website through a local, loopback-only proxy running inside the app itself, so it can rewrite that site's own requests before they reach it — driven by a YAML config file, no rewrite rules hardcoded into the app.

Read README.md first. It's the canonical deep-dive, split into three parts: cross-platform core, Android (transparent CONNECT-tunnel proxy), iOS (local reverse proxy). This file is a shorter briefing — what to know before reading the README, and a few hard-won lessons the README states as fact but doesn't dwell on the "how we found this" story behind.

Standing rule: no real data in this repo, ever

assets/profiles.yaml must only ever contain placeholder values — example.com-style domains, never a real business's domain, subdomain, or any real rule content (regex triggers, field-extraction patterns, request templates). This applies equally to test fixtures (test/*.dart), generated config (network_security_config.xml, android/app/src/main/res/raw/*.pem), and any documentation or planning notes that get committed.

This isn't a style preference — a leaked real domain is an easy mistake to make (edit profiles.yaml by hand, then forget to regenerate network_security_config.xml or update a test fixture that pinned a specific domain) and an easy one to miss in review, since every individual file can look correct in isolation. Before committing any change to assets/profiles.yaml, network_security_config.xml, or the test/ directory: git grep for anything that isn't example.com/example.org/obviously a placeholder, and check whether scripts/buildCerts.sh --all alone (reading only what's actually in profiles.yaml) would be enough for a fresh clone to pass flutter test.

Also never populate assets/profiles.yaml's rules list with real fraud/exploit content, regardless of how a request is framed — only the generic interception/rewriting mechanism gets built here, never real rule content targeting a real business's actual request flow.

The one thing to internalize about the iOS mechanism

Android's proxy is a transparent CONNECT tunnel — the WebView's own address bar always shows the real domain, so the site's own absolute URLs, cookies, Referer, and CSP are all correct by construction. iOS has no equivalent API (both alternatives were tried and are dead — see README Part 3), so ReverseProxyServer instead makes the WebView navigate directly to 127.0.0.1. That one fact — the WebView's real top-level origin is 127.0.0.1, not the real domain — is the root cause of nearly every non-obvious bug in that file. Anything that assumes "the browser knows what domain it's really on" breaks, in both directions:

  • Response → browser: any reference to another host in a response body has to be rewritten to stay under 127.0.0.1 (response_rewriter.dart), including inside JSON/XML API responses, not just HTML — a page's JS can read a URL out of an API response and navigate to it just as easily as clicking a link.
  • Request → real backend: Referer/Origin on outgoing requests say 127.0.0.1 unless explicitly rewritten back (_rewriteOutgoingOrigin in reverse_proxy_server.dart). A backend that validates those as a basic anti-forgery check silently falls back to a default page instead of erroring. That failure mode looks exactly like a cookie or session bug from the outside: a normal 200, no error anywhere, just unrelated content. If a future symptom looks like that, check this class of bug before assuming it's session state.

Other traps already found once — don't rediscover them

  • request.uri.path silently drops a path segment when a site writes a doubled leading slash (https://host//some/path) — Dart's Uri parser treats a //-prefixed path as a network-path reference and reads the first segment as an authority/host, not a path segment. _handleRequest reconstructs the literal path via request.uri.authority when it's non-empty. If a proxied request seems to hit the wrong real path with no explanation, check this first. If you need to re-verify Dart's behavior on a different case, don't trust Uri.parse in isolation — confirm against a real HttpServer too: Uri.parse('//x/y').path alone can mislead you, since the ambiguity is specifically about how a raw request line gets parsed, not just about Uri as a standalone class.
  • UpstreamClient in ReverseProxyServer must be one shared instance, not created per request — a fresh instance means a fresh TCP+TLS handshake to the real backend for every single request, which adds enough latency to lose real timing-sensitive races a site's own JS depends on.
  • The rewritable-content-type allowlist in _writeRewritten covers text/*/javascript/json/xml, deliberately broader than "obvious page assets" — kept as an allowlist (never "rewrite everything except known-binary"), since the decode/re-encode round trip is lossy for genuinely binary bytes.
  • A target site's own timing bug can look like a proxy bug. A page's own script referencing a global provided by a third-party <script async defer> tag, with no check that it's defined yet, usually wins that race on a direct connection — this proxy's one extra local hop per first-party request is sometimes enough to flip it, turning "usually fine" into "reliably throws," and the failure often looks exactly like a stuck/hung page rather than a visible error. This is a bug in the target site, not in this proxy. InterceptRule.injectScript is a testing-only workaround for it (see README Part 3, "Why injectScript exists") — not a fix; file it upstream too.
  • A static startPath can go stale in your own test loop. Unlike a script asset requested with a cache-busting query param, the one URL startPath points at is requested identically every time the app launches — across repeated flutter run reinstalls, the on-device HTTP cache can silently keep serving an old copy of it, making a real fix (or a real regression) look like it has no effect. If a change "isn't working" and you can't explain why, append a throwaway query param to startPath before trusting that result (see README Part 3, same section).

Debugging approach that actually worked here

When something looks broken but every individual piece (headers, cookies, body) checks out fine in isolation: get a byte-level capture of a known-working request (Burp/mitmproxy/curl -v against the real site, bypassing this proxy entirely) and diff it against what this proxy actually sends, field by field — don't theorize about TLS fingerprinting, WAFs, or timing races until you've ruled out something mundane like a malformed path or a missing header. In this project, several plausible theories (TLS client fingerprinting, a lost session from a cancelled request) were seriously investigated and were wrong; the actual bug was found by exactly this byte-level diff. curl -i/curl -v reproducing the same success or failure as the app is the fastest way to tell "this is proxy-specific" from "this is inherent to the request."

That technique is for "the response looks wrong." For "the page silently stops progressing with no error visible anywhere" (as opposed to wrong-looking content), the faster path is different: temporarily inject a window.onerror/unhandledrejection → console.log script into the response so the existing onConsoleMessage capture in browser_screen.dart surfaces the actual uncaught exception — see README Part 3, "Finding this kind of bug in the first place." Read that section's gotcha about checking the response body, not Content-Type, before doing this — getting that check wrong corrupts the very response you're trying to debug and produces a fake, self-inflicted symptom that looks exactly like a real bug.

Quick commands

flutter pub get
scripts/buildCerts.sh --all      # generates certs for whatever's in
                                  # assets/profiles.yaml right now — run
                                  # this after any profiles.yaml edit
flutter analyze
flutter test
flutter build apk --debug        # Android
flutter build ios --no-codesign  # iOS (also: cd ios && pod install)

Where things actually are

  • README.md — the real documentation, split into cross-platform / Android / iOS parts. Read the relevant part before touching a file.
  • lib/proxy_server.dart — Android's mechanism.
  • lib/reverse_proxy_server.dart + lib/response_rewriter.dart — iOS's mechanism.
  • lib/profile.dart, lib/rule_engine.dart — shared config/rule model, platform-agnostic.
  • test/ — rule_engine_test.dart (cross-platform), response_rewriter_test.dart + reverse_proxy_server_test.dart (iOS-only, includes regression tests for every gotcha listed above).