Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,3 @@ cmd/*/resource_windows_*.syso
.gochache/
.android-build-audit/
.tmp-android-engine-upstream/

# Local engineering audit (human-readable, intentionally not committed)
reports/cottendns-network-engineering-audit.html
26 changes: 6 additions & 20 deletions README.MD
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,7 @@ Recent work focused on staying usable on highly restrictive, lossy networks:

- **DNS-over-TLS and DNS-over-HTTPS.** `RESOLVER_TRANSPORT = dot | doh` encrypts the client→resolver hop, so on networks that fingerprint plaintext DNS on 53 the tunnel looks like a device using an encrypted DNS provider. **A public resolver needs no server change at all** — keep the resolver IPs you already use (`1.1.1.1`, `8.8.8.8`, `9.9.9.9`); the encryption covers exactly the hop that gets fingerprinted, and the resolver still reaches your server through the normal delegation. Strictly opt-in (`auto` never escalates into them) and never a one-way door: if the TLS port is blocked the client falls back to **UDP → TCP/53** by itself. Verify-by-default, with SPKI pinning (`RESOLVER_TLS_PIN`) for a self-signed server. Details: [Engineering Notes §17](docs/ENGINEERING_CHANGES.md).
- **Optional DoT/DoH server listeners that can share :443.** Only needed to point clients *directly* at this server. They reuse the same transport-agnostic packet handler as UDP/TCP, and TLS material resolves cert/key → ACME → self-signed so an enabled listener always comes up. `DOH_COEXIST_MODE` defaults to **never binding :443**, so a co-hosted panel (3x-ui, Hiddify, …) keeps the port and every inbound it supports — VMess/VLESS/Trojan, xhttp/gRPC/raw/ws/tls, CDN-fronted — keeps working untouched; taking the port is always an explicit choice. Both listeners off by default, and they draw from a capped connection sub-budget so flooding them can never starve the plain TCP/53 survival path.
- **Adaptive per-resolver transport.** The server serves UDP/53 and TCP/53 on the same port. With `RESOLVER_TRANSPORT = auto`, the client measures both paths for every resolver and independently uses the fastest healthy one, so a poisoned or slow UDP resolver can move to TCP without forcing healthy resolvers to follow it. DoT/DoH remain user opt-ins. Every response channel (TXT/CNAME/A/NULL/HTTPS) works across the transport set.
- **Packet-size-aware joint routing.** Resolver and transport are scored as one path for each packet. A fast low-MTU backup remains in the pool for ACK/control and small fragments instead of being discarded, while large fragments stay on paths whose measured upload MTU can carry them. Background scans can therefore add capacity without lowering the session MTU.
- **DNS-over-TCP/53 fallback.** The server serves both UDP/53 and TCP/53 on the same port, and the client (`RESOLVER_TRANSPORT = auto`) probes over UDP first, then transparently re-probes the **whole fleet over TCP/53** if UDP finds no resolvers — surviving networks that filter or truncate UDP/53. Zero cost when UDP works. Every response channel (TXT/CNAME/A/NULL/HTTPS) works over TCP too.
- **TCP survival-path guardrails.** TCP/53 is treated as a first-class fallback path: per-IP connection caps, optional per-connection query limits, read-idle timeouts, and write deadlines protect the listener while keeping persistent DNS-over-TCP useful.
- **Paired config presets.** Bundled client/server pairs (`speed`, `survival`, `tcp-survival`) tune both sides together through `CONFIG_PRESET`, while explicit TOML/CLI values still override the profile.
- **More honest MTU loss reporting.** Loss-aware MTU probing now reports failures against the configured sample budget, so scans can show intermediate loss percentages instead of collapsing early rejects into only `0%` or `100%`.
Expand Down Expand Up @@ -530,16 +529,15 @@ So the worst case is that you end up exactly where you were before.
| Key (client) | Default | Purpose |
| --- | --- | --- |
| `RESOLVER_TRANSPORT` | `auto` | `auto` \| `udp` \| `tcp` \| `dot` \| `doh` |
| `RESOLVER_TRANSPORT_PATHS` | `{}` | Optional per-resolver overrides, for example `{ "1.1.1.1" = "doh", "8.8.8.8" = "auto" }`. Keys may be resolver IPs, labels, or connection keys. |
| `RESOLVER_TRANSPORT_BACKGROUND_SCAN_INTERVAL_SECONDS` | `30` | Low-rate interval for remeasuring alternate paths one resolver at a time. |
| `RESOLVER_TLS_SERVER_NAME` | *(empty)* | SNI/cert name. **Leave empty for public resolvers**; set it only when pointing at your own DoT/DoH server. |
| `RESOLVER_TLS_PIN` | *(empty)* | Base64 SHA-256 of the server certificate's SubjectPublicKeyInfo. Replaces CA validation — the right way to trust a **self-signed** server. Survives certificate renewal. |
| `RESOLVER_TLS_INSECURE_SKIP_VERIFY` | `false` | Last resort. The payload stays AEAD-encrypted either way, but an unverified hop can be silently intercepted. |
| `RESOLVER_DOT_PORT` / `RESOLVER_DOH_PORT` / `RESOLVER_DOH_PATH` | `853` / `443` / `/dns-query` | Where the resolver IP is contacted. |

> `auto` never escalates into DoT/DoH. Add an explicit per-resolver override when
> an encrypted resolver hop is wanted. Port/path and TLS settings remain shared by
> resolvers using that encrypted transport.
> **Limits.** The transport and its port/path are **client-wide, not per-resolver**:
> you cannot run one resolver over DoH while another stays on UDP, and a provider
> using a different path needs its own profile. Resolver entries are IPs, not
> hostnames.

#### Optional: run your own DoT/DoH endpoint

Expand Down Expand Up @@ -636,7 +634,7 @@ The sample files are the source of truth for defaults and operational comments:
| 🪪 Tunnel identity/security | `DOMAINS`, `DATA_ENCRYPTION_METHOD`, `ENCRYPTION_KEY`, `QUERY_TYPES`, `DNS_RANDOMIZE_QUERY_ID`, `DNS_EDNS_COOKIE`, `DNS_QNAME_CASE_RANDOMIZATION`, `EDNS_UDP_SIZE`, `RESOLVER_IGNORE_INJECTED_NXDOMAIN` |
| 🧦 Local proxy | `PROTOCOL_TYPE`, `LISTEN_IP`, `LISTEN_PORT`, `SOCKS5_AUTH`, `SOCKS5_USER`, `SOCKS5_PASS` |
| 📛 Local DNS | `LOCAL_DNS_ENABLED`, `LOCAL_DNS_IP`, `LOCAL_DNS_PORT`, `LOCAL_DNS_CACHE_MAX_RECORDS`, `LOCAL_DNS_CACHE_TTL_SECONDS`, `LOCAL_DNS_PENDING_TIMEOUT_SECONDS`, `DNS_RESPONSE_FRAGMENT_TIMEOUT_SECONDS`, `LOCAL_DNS_CACHE_PERSIST_TO_FILE`, `LOCAL_DNS_CACHE_FLUSH_INTERVAL_SECONDS` |
| 📡 Resolver/loss handling | `RESOLVER_TRANSPORT`, `RESOLVER_TRANSPORT_PATHS`, `RESOLVER_TRANSPORT_BACKGROUND_SCAN_INTERVAL_SECONDS`, `RESOLVER_BALANCING_STRATEGY`, `RESOLVER_RATE_LIMIT_ENABLED`, `UPLOAD_PACKET_DUPLICATION_COUNT`, `DOWNLOAD_PACKET_DUPLICATION_COUNT`, `UPLOAD_SETUP_PACKET_DUPLICATION_COUNT`, `DOWNLOAD_SETUP_PACKET_DUPLICATION_COUNT`, `STREAM_RESOLVER_FAILOVER_RESEND_THRESHOLD`, `STREAM_RESOLVER_FAILOVER_COOLDOWN`, `RECHECK_INACTIVE_SERVERS_ENABLED`, `RECHECK_INACTIVE_INTERVAL_SECONDS`, `RECHECK_SERVER_INTERVAL_SECONDS`, `RECHECK_BATCH_SIZE`, `AUTO_DISABLE_TIMEOUT_SERVERS`, `AUTO_DISABLE_TIMEOUT_WINDOW_SECONDS`, `AUTO_DISABLE_MIN_OBSERVATIONS`, `AUTO_DISABLE_CHECK_INTERVAL_SECONDS`, `BASE_ENCODE_DATA`, `DUPLICATION_PREFER_DISTINCT_DOMAINS`, `ADAPTIVE_DUPLICATION`, `ADAPTIVE_DUPLICATION_TARGET_DELIVERY` |
| 📡 Resolver/loss handling | `RESOLVER_TRANSPORT`, `RESOLVER_BALANCING_STRATEGY`, `RESOLVER_RATE_LIMIT_ENABLED`, `UPLOAD_PACKET_DUPLICATION_COUNT`, `DOWNLOAD_PACKET_DUPLICATION_COUNT`, `UPLOAD_SETUP_PACKET_DUPLICATION_COUNT`, `DOWNLOAD_SETUP_PACKET_DUPLICATION_COUNT`, `STREAM_RESOLVER_FAILOVER_RESEND_THRESHOLD`, `STREAM_RESOLVER_FAILOVER_COOLDOWN`, `RECHECK_INACTIVE_SERVERS_ENABLED`, `RECHECK_INACTIVE_INTERVAL_SECONDS`, `RECHECK_SERVER_INTERVAL_SECONDS`, `RECHECK_BATCH_SIZE`, `AUTO_DISABLE_TIMEOUT_SERVERS`, `AUTO_DISABLE_TIMEOUT_WINDOW_SECONDS`, `AUTO_DISABLE_MIN_OBSERVATIONS`, `AUTO_DISABLE_CHECK_INTERVAL_SECONDS`, `BASE_ENCODE_DATA`, `DUPLICATION_PREFER_DISTINCT_DOMAINS`, `ADAPTIVE_DUPLICATION`, `ADAPTIVE_DUPLICATION_TARGET_DELIVERY` |
| 📦 Compression | `UPLOAD_COMPRESSION_TYPE`, `DOWNLOAD_COMPRESSION_TYPE`, `COMPRESSION_MIN_SIZE` |
| 📏 MTU discovery | `MIN_UPLOAD_MTU`, `MIN_DOWNLOAD_MTU`, `MAX_UPLOAD_MTU`, `MAX_DOWNLOAD_MTU`, `MTU_TEST_RETRIES_RESOLVERS`, `MTU_TEST_TIMEOUT_RESOLVERS`, `MTU_TEST_PARALLELISM_RESOLVERS`, `MTU_TEST_RETRIES_LOGS`, `MTU_TEST_TIMEOUT_LOGS`, `MTU_TEST_PARALLELISM_LOGS`, `MTU_PROBE_SAMPLES`, `MTU_MAX_LOSS`, `MTU_GROUP_GAP_RATIO`, `MTU_ADAPTIVE_GROUPING` |
| ⚙️ Workers/queues/timers | `RX_TX_WORKERS`, `TUNNEL_PROCESS_WORKERS`, `TUNNEL_PACKET_TIMEOUT_SECONDS`, `DISPATCHER_IDLE_POLL_INTERVAL_SECONDS`, `TX_CHANNEL_SIZE`, `RX_CHANNEL_SIZE`, `RESOLVER_UDP_CONNECTION_POOL_SIZE`, `STREAM_QUEUE_INITIAL_CAPACITY`, `ORPHAN_QUEUE_INITIAL_CAPACITY`, `DNS_RESPONSE_FRAGMENT_STORE_CAPACITY`, `SOCKS_UDP_ASSOCIATE_READ_TIMEOUT_SECONDS`, `CLIENT_TERMINAL_STREAM_RETENTION_SECONDS`, `CLIENT_CANCELLED_SETUP_RETENTION_SECONDS` |
Expand Down Expand Up @@ -854,7 +852,6 @@ Requirements:
| Go `1.25.0` | Build and test |
| Git | Version metadata and normal development |
| Python 3 | Optional local multi-target build helper |
| GCC or Clang | CGO race-detector tests |

Build current platform:

Expand Down Expand Up @@ -884,17 +881,6 @@ go test -v -run TestName ./internal/client
go test -race ./internal/client ./internal/udpserver
```

Reproduce the local poison, hijack, all-transport, and 40%/84%-loss environment
on Windows:

```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\test-hostile-network.ps1 -Count 10 -FullRace
```

The harness creates local hostile UDP/TCP responders, exercises native queries
over UDP/TCP/DoT/DoH, and performs actual Reed-Solomon reconstruction under
randomized loss. It does not contact public resolvers or require a live server.

Local multi-target build:

```bash
Expand Down
20 changes: 3 additions & 17 deletions client_config.toml.simple
Original file line number Diff line number Diff line change
Expand Up @@ -154,10 +154,9 @@ LOCAL_DNS_CACHE_FLUSH_INTERVAL_SECONDS = 60.0
RESOLVER_BALANCING_STRATEGY = 3

# How DNS queries reach resolvers:
# "auto" (default) - test UDP/53 and TCP/53 per resolver, then keep using the
# fastest healthy path for each resolver independently.
# A slow/failing UDP path can move to TCP without changing
# the rest of the resolver pool.
# "auto" (default) - probe over UDP/53 first; if NO resolver passes MTU
# testing, retry the whole fleet over DNS-over-TCP/53.
# Zero cost on UDP-working networks (TCP is never tried).
# "udp" - UDP only (legacy behavior).
# "tcp" - TCP/53 only, for networks that block or truncate UDP/53.
# "dot" - DNS-over-TLS (RFC 7858), normally :853.
Expand All @@ -174,19 +173,6 @@ RESOLVER_BALANCING_STRATEGY = 3
# DOH_LISTENER_ENABLED on the server (both off by default).
RESOLVER_TRANSPORT = "auto"

# Optional per-resolver transport policy. Unlisted resolvers inherit the global
# RESOLVER_TRANSPORT above. Keys may be an IP or IP:port. "auto" compares UDP
# and TCP for that resolver; explicit "udp"/"tcp" stay pinned. "dot"/"doh" are
# opt-in and retain UDP/TCP survival fallbacks if their encrypted path is blocked.
# Examples:
# RESOLVER_TRANSPORT_PATHS = { "1.1.1.1" = "tcp", "8.8.8.8:53" = "auto" }
RESOLVER_TRANSPORT_PATHS = {}

# Low-rate background path/MTU verification. At most one active resolver is
# checked at a time, at the current session MTU, so alternate-path health and
# latency stay fresh without taking bandwidth away from user traffic.
RESOLVER_TRANSPORT_BACKGROUND_SCAN_INTERVAL_SECONDS = 30.0

# Encrypted-resolver settings (used only by "dot"/"doh").
# RESOLVER_TLS_SERVER_NAME is the SNI + certificate name presented to the
# resolver; leave empty to use the resolver IP (which then needs an IP SAN or a
Expand Down
4 changes: 0 additions & 4 deletions docs/ANDROID_ENGINE_INTEGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,6 @@ existing launcher contract. The linker flags provide 16 KiB page compatibility.
and `WD_SCAN`.
- Generic SOCKS5 UDP, DNS fallback, loss recovery, adaptive duplication, and
server-advertised fairness remain part of this source tree.
- Poison-aware question validation, per-resolver transport selection,
packet-size-aware narrow-MTU routing, and rotating background path discovery
are implemented inside this engine. Android receives the same behavior without
a Kotlin port when its pinned CottenDNS SHA is advanced.

Pinning the engine SHA makes debug and release builds use identical engine code
and prevents stale prebuilt binaries from silently surviving an app merge.
109 changes: 10 additions & 99 deletions docs/ENGINEERING_CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -389,11 +389,10 @@ length-prefixed, routed through the **exact same** transport-agnostic
load-shedding, graceful shutdown — so all tunnel logic (sessions, FEC, channels,
encryption) is shared with UDP, no duplication.

**Client.** Resolver-local transport policy via
`RESOLVER_TRANSPORT = auto | udp | tcp`:
- **`auto` (default)** measures UDP and TCP/53 for every resolver, then keeps
each resolver on its fastest healthy path. A bad UDP path can switch without
moving the rest of the fleet.
**Client.** Client-wide transport via `RESOLVER_TRANSPORT = auto | udp | tcp`:
- **`auto` (default)** probes over UDP first; if **zero** resolvers pass MTU
testing, it flips to TCP and **re-probes the whole fleet over TCP/53**. On a
UDP-working network TCP is never attempted (zero cost).
- A `queryExchanger` abstraction makes the probe, session-init, and health paths
transport-agnostic.
- A persistent **per-resolver TCP connection manager** (`tcp_data.go`) serves the
Expand Down Expand Up @@ -667,9 +666,8 @@ doh ─► UDP ─► TCP/53 tcp ─► (no fallback)
auto ─► UDP ─► TCP/53
```

The chain is `resolverTransportChain()` and MTU discovery walks it independently
for each resolver. The background scanner keeps alternate paths measured after
startup.
The chain is `resolverTransportChain()`; the walk is in `RunInitialMTUTests`,
which re-probes the whole fleet on each step down.

### 17.2 How they are wired into the data path

Expand Down Expand Up @@ -709,9 +707,10 @@ from the entry plus the transport's own port/path, so `1.1.1.1` becomes
`https://1.1.1.1:443/dns-query`. Cloudflare, Google and Quad9 publish certificates
carrying their **IP as a SAN**, so (A) validates with no configuration at all.

Per-resolver overrides were added later in §25. The encrypted transport's
port/path and TLS identity remain shared, but individual resolvers can now select
`auto`, `udp`, `tcp`, `dot`, or `doh`.
*Caveat:* the transport and its port/path are client-wide, not per-resolver. You
cannot run one resolver over DoH while another stays on UDP, and providers using a
different path cannot be mixed in one profile. The hedging is sequential
(fallback), not parallel.

### 17.4 Certificate trust

Expand Down Expand Up @@ -1116,94 +1115,6 @@ than merely accepting it as a configuration value.

---

## 25. Poison-aware per-resolver transport and path MTU

Transport selection is now resolver-local instead of a whole-client fallback.
`RESOLVER_TRANSPORT_PATHS` can override the global policy by connection key,
resolver label, IP:port, or IP. `auto` measures UDP and TCP/53 for each resolver;
DoT and DoH remain explicit user choices and retain UDP/TCP survival fallbacks.

Initial MTU discovery probes every configured path separately and stores its RTT,
loss, upload MTU, and download MTU. A bounded background scanner performs full
MTU discovery on one rotating resolver/transport path at a time at
`RESOLVER_TRANSPORT_BACKGROUND_SCAN_INTERVAL_SECONDS`, keeping alternate paths
fresh without creating a scan burst or competing materially with user traffic.
Selection uses estimated delivered goodput and will move a resolver away from a
slow, failing, poisoned, or session-MTU-incompatible path. Bulk packets use only
the best path; sparse high-priority/control traffic may hedge one alternate at a
bounded interval, so duplication cannot cap bulk throughput.

The runtime scheduler now scores `(resolver, transport)` jointly for the actual
native packet type and payload size. MTU probe capacity is normalized for each
packet header before eligibility is checked. A resolver or transport below the
global session MTU is therefore not dead capacity: it can carry ACKs, controls,
and any data fragment that fits, while larger fragments remain on wider paths.
Stream affinity receives a small stability bias, not a hard pin, preventing
reordering between equivalent paths without trapping a stream on a materially
slower route. Bulk data never uses an unmeasured transport.

If local UDP transmission, persistent TCP/DoT dialing/writing, or a DoH exchange
fails, the exact unacknowledged DNS query is replayed immediately on the best
eligible alternate resolver/transport. Replay preserves the native frame,
session, and sequence identity, is capped at two path hops, and does not wait for
the full ARQ RTO. ARQ/NACK remains the correctness backstop after the bounded
fast replay is exhausted.

Inbound replies are bound to resolver address, local socket, transport, query ID,
and the complete DNS question. Same-ID replies carrying a different question are
treated as injection and ignored. When injected NXDOMAIN filtering is enabled,
the forged answer no longer consumes the outstanding request, allowing a genuine
answer arriving moments later to win. Poison is evidence that an alternate path
must be compared, not an automatic penalty against a UDP path that remains the
fastest working option.

Question fingerprints canonicalize ASCII letter case because DNS names are
case-insensitive; resolvers that normalize 0x20 casing are accepted without
weakening QTYPE/QCLASS or name matching. UDP replies carrying `TC=1` are treated
as explicit hard path failures and move that resolver toward TCP immediately
instead of spending multiple timeout windows on an answer that cannot fit.
Poison evidence accelerates a following timeout for two minutes, then expires so
an old incident cannot make a clean future network overly sensitive.

An unusually fast forged response also acts as a Happy-Eyeballs trigger. The
still-pending query is raced once on the best alternate path; the original is not
cancelled, and the first response that passes DNS-question and native tunnel
authentication atomically claims all replay siblings. Existing control hedges
count as the alternate, preventing poison from causing duplicate amplification.

Background path exploration is congestion-aware and capacity-budgeted. One
rotating full MTU refresh is charged per 4096 original foreground frames, a
conservative approximately 1-2% allowance using a 64-query scan cost model.
No scan starts when TX, encoded-TX, RX, or pending-query pressure is elevated.
Completely idle paths receive a stale-state refresh no more often than every two
minutes (or four configured scan intervals), but even that exception yields to a
single queued user packet.

The final speed audit ranks replay candidates across the complete eligible
`(resolver, transport)` set instead of preferring the same resolver by default.
The common non-duplicated response path no longer scans the complete pending map,
DNS question fingerprints are parsed once per normal ingress, and immutable
encoded DNS frames are retained rather than copied into each pending/stream
queue. Transport-manager publication is protected during runtime teardown.
Poison/timeout correlation also accepts the timeout deadline being a few
milliseconds earlier than the adjacent poison event, eliminating an
event-ordering-dependent missed fast switch found by repeated race testing.

After three successful samples, pending-query blackhole detection uses a
conservative RTT-derived deadline (`6 × RTT + 500 ms`, with a 1.5-second floor
and the configured request timeout as its ceiling). Slow/high-jitter paths retain
the configured timeout. Late genuine replies remain claimable during the
existing grace period and retract their timeout observation.

The server remains transport-agnostic: UDP, TCP, DoT, and DoH feed the same
authenticated native packet handler and keep the client's selected settings
dynamic. The server's Super-FEC band now chooses enough parity for a 90% modeled
block-recovery target, within Reed-Solomon and configured caps. Randomized loss
tests exercise actual encoding and reconstruction at 40% and 84% loss; ARQ
remains the correctness backstop when a block exceeds its parity budget.

---

*All changes keep ARQ as the correctness backstop; every optimization above is
designed to fail safe — if FEC, MTU grouping, a carrier, or a transport channel
does not help on a given path, the tunnel still delivers through the surviving
Expand Down
Loading
Loading