Repository navigation
Port to tokio (supersedes #856, closes #944) - #951
Merged
Merged
Conversation
async-std is discontinued upstream (RUSTSEC-2025-0052). tokio is already in the dependency tree through hickory-resolver, so switching adds no crate and removes 28 from Cargo.lock (async-std, async-io, polling, async-global-executor, blocking, futures-executor, ...). This is a like-for-like port. The scanner keeps its architecture and the library API keeps its signatures: - a single `FuturesUnordered` keeps at most `batch_size` sockets in flight from one future; nothing is spawned per socket; - connect and UDP receive timeouts use `tokio::time::timeout` and report the same `TimedOut` error that async-std's `io::timeout` did; - open TCP sockets are still shut down in both directions before they are closed (through `into_std()`, because tokio only has a write-side shutdown); - `--interval` sleeps with `tokio::time::sleep`. The binary runs the scan on a current-thread runtime that exists only for the scan. Address resolution (blocking DNS lookups and file reads) runs before the runtime is created and scripts run after it is dropped, so neither can stall the scan, and hickory's internal runtime is never nested in ours. Library users now have to poll the scan futures from a tokio runtime with the I/O and time drivers enabled. The crate and `Scanner` docs say so, and the crate example shows how. Builds on the earlier port in #856 by @vrtgs. Co-authored-by: Vrtgs <vrtgs@vrtgs.xyz>
The scan polls every in-flight socket from one future through a `FuturesUnordered`. tokio's cooperative scheduling gives that future a budget of 128 units per poll. Once it is spent, every tokio resource returns `Pending` and defers its wake-up, but `FuturesUnordered` still polls all the other ready sockets in that same poll. When thousands of connections finish together (a burst of refusals, or a batch of timeouts firing at once), the scan keeps re-polling sockets only to park them again, roughly 128 at a time. async-std had no budget. `tokio::task::unconstrained` restores that behaviour: one poll handles every ready socket. The future still returns `Pending`, and so lets the runtime poll I/O and timers, as soon as no socket is ready.
tokio's socket I/O is readiness based: `UdpSocket::send` and `recv` first wait until the reactor has reported the socket ready and only then make the system call. async-io, which async-std used, made the call first and only waited after `WouldBlock`. With a straight port, every UDP probe therefore made an extra trip through the event loop before its send. Probes to closed ports on the local host also no longer finished in their first poll: there the ICMP "port unreachable" is usually queued before `send` returns, so async-std's first `recv` already saw the refusal. Bind a non-blocking std socket instead, send the probe and try the first receive straight away. The socket is only handed to tokio, and registered with its reactor, when there is really something to wait for. Errors and timeouts are reported exactly as before.
The test suite deliberately never opens sockets (see #942), so nothing in CI showed whether a change to the scanner makes RustScan slower, hungrier or wrong. The tokio port (#944) needs exactly that evidence. The "Scan benchmark" workflow runs for pull requests that touch the scanner, its runtime or its dependencies, and on demand. It builds the change and its base in release mode on Linux x64/arm64, macOS and Windows runners. It then runs .github/scan-benchmark/scan_bench.py, which: - opens 315 TCP listeners and 32 UDP responders on 127.0.0.1 inside the runner. Nothing outside the runner is scanned. Ports stay out of the ephemeral range, so a scan can never connect a socket to itself; - scans them with both builds, alternating the build order between runs: full TCP sweeps with batch sizes 500/4500/10000, 8 loopback hosts, one port, a UDP sweep, and on Linux ports that iptables drops (filtered TCP and UDP, and a mixed sweep), so timeouts are measured too; - fails unless every run finds every listener and no build reports ports the base never found, or misses ports the base always found; - reports medians of throughput (from RustScan's own scan timer), wall time, time to the first open port, peak RSS, CPU time, and sampled peak descriptors and threads. The table goes to the job summary, the raw data to an artifact; - fails if the change's median throughput or wall time is more than 15% worse, or its peak RSS more than 25% worse, in any scenario. contributing.md documents the workflow as the place where scanning is measured, next to the rule that tests must not scan.
…s (to be reverted)
…te builds (to be reverted)" This reverts commit 30ac107.
The scan drives every socket from one future on a current-thread runtime, so the runtime only polls for I/O and fires timers between two polls of that future. A poll only ended once no socket was ready, which could take a long time: - starting a whole batch at once: 8,191 sockets took longer than a 500 ms timeout on the Windows runner, so connections that had succeeded during the fill were only seen after their timers fired. The benchmark found no open port at all with `-b 10000 -t 500` there; - UDP probes to closed ports on the local host finish straight away, so a whole sweep ran in one poll, and the replies from open ports waited until the end: the first open UDP port was reported 2.6x later than with async-std. `scan_sockets` now starts and finishes sockets in rounds of at most 128 (`WORK_PER_TURN`, both counted together) and calls `yield_now()` between rounds, which defers its wake-up until the runtime has polled the OS selector without blocking. Each round still keeps up to `batch_size` sockets in flight; the batch now fills over several rounds instead of in one go. The cost is one extra non-blocking `epoll_wait` (or equivalent) per 128 sockets.
A UDP probe that gets no answer straight away is handed to tokio to wait for the reply. On Linux, an ICMP "port unreachable" that arrives later only raises EPOLLERR on the socket. tokio does not count that as readable, so `UdpSocket::recv` slept until the timeout instead of returning the refusal. async-io treated EPOLLERR as readable. The straight port (before the eager probe) showed the effect: the UDP sweep of the local host took 4.1 s instead of 1.0 s, because every closed port waited for the 500 ms timeout. With the eager probe it only hit closed ports whose ICMP arrived after the first receive. That is rare on the local host but the common case for any remote host, where it turned every closed port into a timeout. `recv_or_error` waits for `READABLE | ERROR` readiness and hands out the queued error (`take_error`), so a refused probe ends as soon as the ICMP arrives, as before.
What the first runs showed: - Only answers that arrive while the probe is still being sent were measured: on the loopback interface, refusals and ICMP errors are queued before connect()/send() returns. A remote host answers a round trip later, which is the path where tokio's UDP receive missed ICMP errors. On Linux the workflow now also sets up a host 5 ms away (a network namespace behind a netem-delayed veth pair, with ICMP rate limiting off, running its own listeners), and the benchmark sweeps it over TCP and UDP. - Windows retries a refused connection for about a second. With `-t 500`, 8,191 concurrent attempts made even the base miss listeners. Windows now uses the default timeout and sweeps 8,191 ports (2,047 for the 500-socket batch). - A build now fails the check only if it misses more listeners than the base does (any miss at all if the base misses none). Misses by the base are reported as a warning in the summary. - The macOS runner has no `net.inet.icmp.icmplim`, which failed the "describe" step, and caps descriptors at 10240, so the 65536 limit could not be set. The describe step can no longer fail the job, and 10240 is tried as well.
hickory's synchronous resolver runs its own tokio runtime, and tokio panics when a runtime is dropped inside an asynchronous context. That was already true before, but library users now have to run the scanner on tokio, so they are more likely to call `parse_addresses` from async code. Document the two safe ways: call it before starting the runtime (as the binary does) or through `spawn_blocking`.
This was referenced Oct 1, 2026
Closed
This was referenced Oct 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RustScan moves off the discontinued
async-std(RUSTSEC-2025-0052) onto tokio. The scanner keeps its design and the library API keeps its signatures. A new opt-in CI benchmark compares the change againstmasterin GitHub Actions. On every platform the port is at least as fast asmasterand uses no more memory. On Linux x64, sweeps are 39–74% faster, the first open port is reported in 6 ms instead of 185 ms, and CPU time is about halved.Supersedes #856. It builds on @vrtgs's port (the
tokio::time::timeouthandling of UDP receives, the shape of the crate example), and the first commit credits them as co-author. Closes #944.Design
FuturesUnorderedkeeps at mostbatch_sizeconnection attempts in flight, and all of them are polled from the one future you await. port to tokio again #856 usedtokio-par-stream, which spawns a task per socket; that is gone, as areblackrock2,eitherandfutures-lite. tokio was already in the dependency tree through hickory-resolver, soCargo.lockloses 28 packages and gains none. The LCG port order is unchanged.run_scan()inmain.rsbuilds a current-thread runtime with I/O and timers enabled. The scan, the I/O driver and the timers all run on one thread. With a multi-threaded runtime, TCP sweeps ran atmaster's speed (−3% to +3%), while the current-thread build was 25–55% faster (table below); it also started worker threads that the scan never uses.parse_addresses(DNS through hickory, host files) and the config read stay synchronous and run before the runtime is created. Scripts run after it is dropped. hickory's sync resolver owns its own tokio runtime, so it must not run inside ours anyway;parse_addressesnow documents this for library users.WORK_PER_TURN = 128). Socket starts and completions are handled in rounds of at most 128 (about 1 ms of work on Linux), withtokio::task::yield_now()between rounds, so the runtime polls the OS selector and fires timers between rounds. The first CI round showed why: without it, starting a whole batch, or a run of UDP probes that finish straight away, happened in one poll. Answers that arrived meanwhile were seen late. On Windows, with-b 10000 -t 500, the timeouts fired first, and that build found 0 open ports in 9 of 9 runs.tokio::task::unconstrained), because the scan bounds its own polls. Under the budget,FuturesUnorderedre-polls every other ready socket once the budget runs out, only for each to returnPendingagain. That cost up to 30% throughput with large batches.send/recv. async-io tried the call first. Probes now go out on a non-blocking std socket, and the first receive is tried at once (on loopback the ICMP "port unreachable" is usually already queued). The socket is only registered with tokio if it has to wait.recv_or_error). On Linux, a queued ICMP error only raisesEPOLLERR, which tokio does not treat as readable, soUdpSocket::recvslept until the timeout. Every closed port of a remote host would have looked filtered and cost a full timeout. The wait now usesREADABLE | ERRORreadiness and returns the error fromtake_error().tokio::time::timeout, which reports the sameTimedOuterror as before.--tries,--closedand--intervalare unchanged. Open sockets are still shut down in both directions (throughinto_std(), because tokio only has a write-side shutdown). No socket options changed. Nagle doesn't matter because nothing is written.What changed
refactor(scanner): run the scanner on tokio instead of async-stdSend, and a scan with nothing to scan finishes on a current-thread runtime)perf(scanner): opt the scan out of tokio's cooperative budgetperf(scanner): send UDP probes before registering the socket with tokioci: add an opt-in scan benchmark comparing a change with its base.github/workflows/scan-benchmark.yml,.github/scan-benchmark/scan_bench.py,contributing.mdfix(scanner): let the runtime poll I/O while a batch is startedfix(scanner): see ICMP errors on UDP sockets that wait in tokioci(scan-benchmark): add a delayed host, fit Windows, compare missesdocs(address): say that parse_addresses must not run inside async codeThe history also has a temporary CI commit that compared intermediate builds, plus its revert. A squash merge drops both.
Library API
Scanner::new,run,run_with_status, the builder methods,parse_addresses/parse_address(still synchronous, still taking hickory'sResolver) andPortStrategy.Scanner::run/run_with_statusmust now be polled inside a tokio runtime with I/O and time enabled. Polling them from another executor (e.g.async_std::task::block_on) panics. They spawn nothing, so a current-thread runtime is enough, and the futures areSend, so they can also be spawned on a multi-threaded one. The crate docs, theScannerdocs and the crate example show how.Scan benchmark (GitHub Actions, medians)
The workflow runs for PRs that touch the scanner, its runtime or its dependencies, and can be started by hand. It builds the PR and its base in release mode and runs
scan_bench.pyon Linux x64 and arm64, macOS and Windows runners. The script scans:Builds run interleaved, 9 runs each (27 for
tcp-1-port). The job fails if any build misses more listeners than the base, or disagrees with it about other ports. It also fails if the PR's median throughput or wall time is more than 15% worse, or its peak RSS more than 25% worse, in any scenario. Unit tests still never open sockets, as #942 requires. Nothing outside the runner is scanned.Final run (all green): https://github.com/bee-san/RustScan/actions/runs/36914068659. Every job summary has the full tables, and the raw data is in the artifacts.
Linux x64 (AMD EPYC 7763, 4 vCPUs),
master(cfb8645) → this PR:tcp-1-port:-p 1024tcp-sweep: 32,767 ports,-b 4500tcp-sweep-b500tcp-sweep-b10000tcp-8-hosts: 127.0.0.1–8 × 4,096tcp-delayed: 32,767 ports, 5 ms awaytcp-filtered: 2,000 dropped,-b 500 -t 250tcp-mixed: 32,767 + 2,000 dropped,-t 500udp-sweep: 32,767 ports,-t 500udp-delayed: 32,767 ports, 5 ms awayudp-filtered: 2,000 dropped,-b 500 -t 250Peak threads were 2 → 1 in every scenario except
tcp-1-port(1 → 1). The filtered rows are bound by the timeout (500 sockets per 250 ms). Their identical throughput shows the timers fire as accurately as before.Other runners, same run (this PR vs
master;tcp-1-portaside):udp-sweep) to +90% (tcp-8-hosts). First open port 81–99% sooner. CPU time 48–57% lower. Peak RSS equal, or 33% lower at-b 10000. Filtered scenarios ±0.2%.udp-sweepandtcp-1-portonly): TCP throughput +10% to +45%, UDP +16%. Wall time 9–31% lower. First open port 55–98% sooner. Peak RSS 18–86% lower. CPU time 23–40% lower.Nothing regressed beyond noise. Every median that came out worse is within 0.2 ms or 1%: for example macOS
tcp-1-portwall time 4 → 4 ms (+5%, 0.2 ms; +0.1% in the previous run), and Windows' timeout-boundtcp-sweep-b500at +0.3%. The one exception is macOStcp-sweep-b500peak descriptors, 400 → 504. This build keeps the 500-socket batch full, whilemasternever filled it, and that is part of why the build is 45% faster there.Why these design choices. These numbers are from the first CI run, Linux x64, each build against
masterin the same run (https://github.com/bee-san/RustScan/actions/runs/36907201462):tcp-sweepthroughputtcp-sweep-b10000throughputudp-sweepwall timeudp-sweepfirst open portunconstrainedRemaining risks
parse_addressesstill blocks and drives hickory's own runtime. It's documented now. An async resolver (step (b) of Track: port RustScan to tokio (PR #856) #944) is left for a follow-up.-b. The batch fills in rounds of 128 and finished sockets are replaced at once, so against fast targets the number of sockets open at the same time stays well below-b(1,030 instead of 4,506 on loopback).-bis still the upper bound, and against slow or filtered targets the batch fills completely (see the filtered and delayed rows).-t 10a timeout can last up to 1 ms longer than with async-io.WORK_PER_TURNis fixed at 128. It wasn't tuned per platform. Each round costs one non-blocking selector poll.master. tokio supports them all, but the firstmasterbuild after merging is what will confirm it.