NOTES.md

Bench notes

Dated records and the decisions the code cannot show. Methodology lives in METHOD.md, operations in docs/operations.md. Numbers from different rigs or instance sizes never mix into one table.

Board change, 2026-10-26

• Each board point now compares the new median of its run with the new median of the baseline run 20260927T193322Z-ci-0cafb5b (#150): 100 x (new / new_A - 1). Before, the board multiplied the factors 1 + delta_pct / 100 of the runs between A and the point. The tarball of A is not in the binaries release, so no run can measure A again.
• The product moved away from the change of the new median over the runs. After 25 runs, it was +73.1% for the p99 of grpc-rapira, but the new median of the last run is 5.55 ms against 5.80 ms in the baseline run, which is -4.3%. For the p99 of hello-rapira-worker, the product was -18.2%, and the change of the median was +2.7%. The product adds the noise of each paired delta, and also the mean of the paired deltas: +2.4% per run for the p99 of grpc-rapira and -0.8% per run for the p99 of hello-rapira-worker, over the 25 runs.
• A CI run measures the build of the run before it as its base build. So each build has a new median in one run and a base median in the next run, on other instances. The 26 runs from 2026-09-27 to 2026-10-26 give 25 such differences for each target and measure. The largest |100 x (base / new - 1)| is the difference between two rigs.
• The largest p99 differences of grpc-rapira (+32.0%), hello-rapira-worker (+11.5%), and hello-rapira-dispatcher (+8.1%) come from one build: its new median in run 20260928T201742Z-ci-f41ff1d (#156) against its base median in the next run. The rig of #156 gave a low p99 to both of its builds: 4.18 ms and 4.22 ms for grpc-rapira.
• Every point also includes the offset of the rig of the baseline run. As the base build of the next run (#152), the baseline build gave other values than in the baseline run: for example -3.45% capacity and +5.47% p99 for hello-rapira-dispatcher. With the difference between two rigs alone as the floor (capacity 4.5%), five capacity points of hello-rapira-dispatcher would be red at -4.5% to -5.1%, although their paired deltas are -1.3% to +0.7%. So each floor adds the largest offset of the baseline build.

measure	two rigs, all targets	two rigs, HTTP/1.1 targets	two rigs, grpc-rapira	baseline offset	floor
capacity	4.12% (dispatcher-static)	4.12%	2.38%	3.45% (dispatcher)	8.0%
p99	32.03% (grpc)	11.51% (worker)	32.03%	5.47% (dispatcher), grpc-rapira 2.45%	17.0%, grpc-rapira 34.5%
RSS	3.34% (worker)	3.34%	2.01%	0.80% (grpc)	4.5%

• These floors replace the A/A floors of the section "A/A calibration, 2026-09-27" on the board. A capacity change under 8%, and a p99 change under 17% of an HTTP/1.1 target, stay gray. The paired delta in the tooltip compares the two builds of one run on the same instances.
• The 26 runs used 14 AMIs, the kernels 7.2.7 to 7.2.11, and PHP 8.5.11 and 8.5.12 in the nightly tarballs. The points include these differences. The floors include only the differences between two successive runs and the offset of the baseline run.
• With these floors, the RSS of every target is green from #152 on, 10% to 17% under the baseline run. All capacity and p99 points are gray.

Method change 3, 2026-09-27

• From this date each run measures two builds on the same instances: the new nightly build and the selected base build. The default base is the newest other non-smoke board build. The rig is one c7a.2xlarge server and two c7a.xlarge loaders in eu-central-1a. The base build comes from the binaries release of this repository.
• Each target runs two stage kinds for each build. The rate stage runs one rapira process per server vCPU at a rate that keeps the server at about 50% CPU, and gives the p99 latency and the RSS. The capacity stage runs 2 rapira processes at a rate above their capacity, and gives the achieved req/s. The ci suite has 3 rounds, 1000 HTTP connections, and 100 gRPC connections with 100 streams each.
• A pair is the base cell and the new cell of one round, one target, and one stage kind. The two cells of a pair run one after the other. A board point is the median paired delta of one run, with the smallest and the largest paired delta as a band. A point has a color only when the change is above the noise floor of an A/A run.
• The new flags are loader_busy, loader_skew, and not_saturated. A run with void cells is complete and publishes. A run where every new cell of one target is void is broken and does not publish.
• PROCESSES, make compare, and rig compare are removed. The stage kind sets the process count, and the pairs inside one run replace the comparison of two runs.
• Each run writes runs/<id>/run.json with the schema rapira-bench-run/3. The board draws only schema 3 runs. The numbers of the earlier runs do not compare with the numbers of this method.
• A CI run holds the rig for about 36 minutes, which costs about 0.57 USD at on-demand prices in eu-central-1.
• The A/A calibration run with NIGHTLY=ac56141 BASE=ac56141 sets the rates of suites/ci.toml and the noise floors in NOISE_FLOOR_PCT and TARGET_NOISE_FLOOR_PCT of board/app.js. The next section, "A/A calibration, 2026-09-27", records its results.

A/A calibration, 2026-09-27

Rig: one c7a.2xlarge server and two c7a.xlarge loaders in eu-central-1a, placement group rapira-bench, AMI ami-05d038400faa82df7 with the kernel 7.2.7-200.fc44.x86_64. NIGHTLY=ac56141 BASE=ac56141: the two builds are the same tarball of the binaries release, so every paired delta is noise. The three passes ran on one rig.

The target tables show the current targets. The run totals and noise measurements include all targets measured at the time.

• Pass 1: 20260927T104446Z-ci-ac56141, 1 round, the initial rates.
• Check pass: 20260927T105717Z-ci-ac56141, 1 round, the rates from pass 1.
• Pass 2: 20260927T110949Z-ci-ac56141, 3 rounds, the final rates of suites/ci.toml. The run is complete and has no void cell.

The rate rule: the rate stage rate is the calibration rate x 50 / the server busy percent, rounded down to a multiple of 1000. The capacity stage rate is 2 x the achieved capacity, rounded up to a multiple of 10000. The values are the medians over the two builds. Pass 1:

target	rate	server busy	new rate	cap rate	achieved capacity	new cap rate	cap loader busy
hello-rapira-classic	40000	21.5%	93000	300000	132039	270000	23%
hello-rapira-worker	60000	28.5%	105000	300000	295340 (held)	600000	50%
hello-rapira-dispatcher	60000	29.0%	103000	300000	295340 (held)	600000	47%
hello-rapira-dispatcher-static	60000	35.5%	84000	300000	247534	500000	40%
grpc-rapira	50000	18.5%	135000	300000	168388	340000	34%

Every new rate stage rate differs from its calibration rate by more than 30%, and 2 capacity cells held. So a check pass ran with the new rates:

target	rate	server busy	rule rate	cap rate	achieved capacity	new cap rate	cap loader busy
hello-rapira-classic	93000	42.0%	110000	270000	133462	270000	24%
hello-rapira-worker	105000	24.0%	218000	600000	371395	750000	55%
hello-rapira-dispatcher	103000	23.5%	219000	600000	384091	770000	58%
hello-rapira-dispatcher-static	84000	26.5%	158000	500000	252278	510000	41%
grpc-rapira	135000	44.0%	153000	340000	166592	340000	34%

• All capacity cells of the check pass saturated, and the capacity loaders stayed at 58% or less. So stages.cap.processes stays 2. The final capacity rates are the "new cap rate" column of the check pass.
• The final rate stage rates of hello-rapira-classic (110000) and grpc-rapira (153000) follow the rule.
• The final rate stage rates of hello-rapira-worker (200000), hello-rapira-dispatcher (200000), and hello-rapira-dispatcher-static (150000) are below the rule. The server CPU of these targets increases much slower than the rate: the worker is at 28% at 60000 req/s and at 24% at 105000 req/s. At the rule rate the loader busy would increase to 60 to 70%.

Pass 2, per target: the server busy range of the 6 rate cells, and the median of the 6 capacity cells.

target	rate	rate server busy	rate loader max	capacity median	cap server busy	cap loader max
hello-rapira-classic	110000	46-66%	28%	132253	53-54%	24%
hello-rapira-worker	200000	49-50%	40%	366073	73%	55%
hello-rapira-dispatcher	200000	46-47%	38%	380001	66-67%	57%
hello-rapira-dispatcher-static	150000	42-58%	35%	250670	63%	41%
grpc-rapira	153000	47-50%	19%	167199	72-73%	35%

• All 36 rate cells held their rate. No capacity cell held. The largest loader busy of the run is 57%.
• The largest |paired delta| of single pairs, counted pairs only, and the floors of board/app.js:

measure	all targets	HTTP/1.1 targets	grpc-rapira	floor
capacity	2.30% (dispatcher, round 1)	2.30%	2.14%	2.5%
p99	19.35% (grpc, round 2)	2.92% (dispatcher, round 1)	19.35%	3.0%, grpc-rapira 19.5%
RSS	1.54%	1.54%	0.74%	2.0%

• A floor is the largest |paired delta| rounded up to a multiple of 0.5%. One p99 floor of 19.5% for all targets would keep a real 10% p99 regression of an HTTP/1.1 target gray. So the p99 floor of the measure leaves out grpc-rapira, and grpc-rapira gets its own p99 floor in TARGET_NOISE_FLOOR_PCT.
• The median deltas (the board points) of pass 2: capacity -1.2% to +1.4%, HTTP/1.1 p99 -0.6% to +0.5%, RSS -0.1% to +0.5%, and grpc-rapira p99 +17.2% (pairs +17.2%, +19.4%, -1.3%). With these floors no point gets a color: every band crosses zero, or the delta is under the floor.
• Two capacity bands are strictly on one side of zero in this A/A run: hello-rapira-worker at -1.7% to -1.0% (pairs -1.0%, -1.7%, -1.2%, with alternating order) and hello-rapira-dispatcher-static at +0.7% to +1.7%. Only the floor keeps these two points gray, so the band rule alone is not sufficient.

The gRPC p99 noise:

• The pair deltas are +17.2% (round 1, base first), +19.4% (round 2, new first), and -1.3% (round 3, base first). The two large deltas have opposite orders, so the order in the pair does not cause them.
• The p99 of the two loaders moves together in each cell: 6815 us and 6773 us in the new cell of round 2, 5400 us and 5154 us in the base cell of round 1. The variation comes with each server start, not from one loader.
• h2load runs 4 threads per loader with 12 or 13 clients each, at 18% loader CPU. The loader is not the limit.
• The HTTP/1.1 targets stay at or below 2.92% per pair.
• The 100 gRPC connections spread unevenly over the 8 server processes, and a large spread comes with a high p99. The spread does not explain all of the noise, and more connections do not lower it. The section "gRPC connection check, 2026-09-27" has the data.

The loader_skew flag on the hello dispatcher with static middleware:

• r1-base-rate-hello-rapira-dispatcher-static: thread 1.444 ms, median 1.203 ms, gap 0.241 ms. The p99 is 2709 us against 2613 us of its pair cell (+3.7%, above the 3.0% HTTP/1.1 p99 floor).
• In this cell the p99 is above the p99 of the pair cell by more than the p99 floor, so the flag does not trip on noise. rig/flags.py has no absolute floor for the skew gap. The p99 measure has 2 counted pairs, so its point is gray.

Time and cost, at 0.93704 USD/h:

• make up ran from 10:43:02Z to 10:44:26Z (84 s), and make down from 11:44:22Z to 11:45:00Z (38 s). The rig ran from 10:43:02Z to 11:45:00Z: 62.0 minutes for the three passes, which cost 0.97 USD.
• The suite time: pass 1 11.5 minutes (24 cells), the check pass 11.5 minutes (24 cells), pass 2 34.4 minutes (72 cells).
• A CI run: the 72 cells take 34.4 minutes, 28.7 s per cell on average. The nominal time is 26 s per rate cell and 20 s per capacity cell, so the overhead is about 5.7 s per cell. make up and make down take 2.0 minutes. A CI run holds the rig for about 36.4 minutes, which costs about 0.57 USD. The EBS volumes (3 x 40 GB gp3) and the 3 public IPv4 addresses add about 0.02 USD.
• The upper bound of a normal run, with the cells 20% longer: 34.4 x 1.2 + 2.0 = 43.3, about 43 minutes, which costs about 0.68 USD.

gRPC connection check, 2026-09-27

Rig: the shape of the A/A calibration, on a new rig. Two gRPC-only A/A runs of 5 rounds at 153000 req/s, with ac56141 as both builds: 20260927T122454Z-grpc-c100-ac56141 with 100 connections and 20260927T123624Z-grpc-c800-ac56141 with 800 connections. A sampler on the server recorded once per second the established connections of each rapira process, the CPU time of each CPU, and the received packets of each ENA queue. The sampler and the two suite files are working files and are not in the repository. The rig ran for about 23.5 minutes, which cost about 0.37 USD.

• With 100 connections, the busiest process held 1.5 to 2.9 times the mean of 12.5 connections, and the least busy process held 1 to 5. The three rate cells with the highest p99 (6853 us, 6709 us, 6414 us) had the largest spread (2.88, 2.88, 2.48 times the mean). Between 1.5 and 2.1 times the mean, the spread shows no relation to the p99.
• The ENA receive queues do not explain the p99. The busiest queue got 1.21 to 1.46 times the mean packet count, and the busy time of the 8 CPUs differed by 1 to 3 points in each rate cell.
• With 800 connections, the busiest process held 1.15 to 1.27 times the mean of 100 connections, but the p99 noise did not go down. Without round 4, the p99 standard deviation over the rate cells is 6.8% with 800 connections and 6.4% with 100 connections, and the largest pair delta is 15.6% and 16.3%. In round 4 both cells were fast (3488 us and 5173 us) at a higher server CPU (67% and 62%), so a second effect changes the latency for longer than one cell. Its cause is not known.
• With 800 connections, most processes stopped at 114 or 115 connections. No code in rapira explains this value.
• 800 connections cost more: the server CPU at 153000 req/s is 60 to 67% against 48 to 51%, and the loader CPU is about 2 times higher.
• All 8 A/A pair deltas of the gRPC p99 with 100 connections (pass 2 of the A/A calibration and this check) are under 19.5%. So the ci suite keeps 100 gRPC connections and the gRPC p99 floor of 19.5%.
• The code of the rapira accept path explains the spread, but no trace confirms it. During a burst of new connections, a worker that is already running accepts one connection after another, and a worker that wakes from sleep finds the queue empty. rapira spreads connections evenly only when they arrive slower than a worker wakes. With 2 processes (the capacity stage), the spread is 50/50 or 51/49.

Method change, 2026-09-25

• From this date the rig measures with the staged rate ladder of METHOD.md. wrk2 loads the HTTP/1.1 targets and k6 loads the gRPC targets, from four c7a.xlarge loaders against one c7a.8xlarge server. This method was replaced the same day, see the next section.
• The numbers before this date come from closed-loop wrk and h2load passes and fixed-rate k6 passes from one c7a.4xlarge loader at a fixed connection count. The numbers after this date are the held rate, the peak successful rate, and the unloaded latency of the ladder. Do not compare numbers from before and after this date, and do not put them in one table.
• Each run now writes runs/<id>/run.json with the schema rapira-bench-run/1. The directories under results/ stay as the record of the old method.
• FrankenPHP now runs the glibc release asset 1.12.7 with the production worker shape, and every PHP runtime uses the shared servers/php.ini. The FrankenPHP and php-fpm numbers before this date used other settings.
• The fixed c7a.8xlarge server and c7a.4xlarge loader pair no longer applies. The loader count and the instance types are knobs, and the Makefile quota check adds the vCPUs of the server and all loaders.

Method change 2, 2026-09-25

• From this date the rig measures one constant rate per target, as METHOD.md defines: 250000 req/s over 5000 connections for the HTTP targets and 100000 req/s over 100 connections for the gRPC target, 60 s measured after a 10 s warm-up, from one c7a.2xlarge loader against one c7a.8xlarge server. The numbers are the p99 latency and the RSS of the rapira process tree next to the achieved rate.
• The current targets are rapira only: hello in the classic, worker, and dispatcher modes, the dispatcher behind the static middleware, and gRPC echo. FrankenPHP, php-fpm, nginx, RoadRunner, Symfony, Laravel, the static file rows, the gRPC-Web and Connect rows, k6, the base build, and the full and ab suites are removed.
• Each run writes runs/<id>/run.json with the schema rapira-bench-run/2. The ladder run of the same day, 20260925T190409Z-ci-73b9d30, has the schema 1: its numbers do not compare with the runs after it, and the board does not draw it.

First ladder run, 2026-09-25

Rig: one c7a.8xlarge server and four c7a.xlarge loaders, eu-central-1a, a server build of rapira feature/grpc-connectrpc at 73b9d30, the ci suite at 256 connections, one round. Run: 20260925T190409Z-ci-73b9d30, published as the first board point. The run has 14 targets: a suite edit that was reverted the same day had removed the two gRPC targets.

• Every fast row ends with server_unsaturated: at the 1.28M stage the hello worker achieves 952k req/s with the server at 84% busy and the loaders at 70%, and the static row achieves 1.14M with the server at 59% and the loaders at 81%. 256 HTTP/1.1 connections at about 0.25 ms per request bound the in-flight rate near 1M req/s. The ci suite moves to 1024 connections from the next run, so numbers from this run and later runs do not compare on the fast rows.
• The FrankenPHP worker collapses at 160k req/s on hello: 56k achieved with the server at 15%, then the probe fails (died).
• The rapira WARN void did not fire on any cell: the server log stays empty through the overloaded stage.
• The ci suite takes 52 minutes of cells on this rig; the CloudWatch CPU chart averages that to 40% over 5-minute periods.

gRPC baseline (rapira PHP dispatcher vs RoadRunner), 2026-09-24

Rig: c7a.8xlarge server + c7a.4xlarge loader, eu-central-1a, AMI ami-066f414c6023b2bf9, plain release build of rapira at feature/grpc-connectrpc e421bbc (pr_sha 924399d is that commit plus a since-removed example that the run did not use), PHP 8.5.10 NTS with opcache and PECL protobuf 5.36.2, RoadRunner 2025.1.15 with php opcache.enable_cli=1 and 32 workers, h2load nghttp2/1.70.0, k6 2.2.0, wrk 4.2.0, 32 workers on every leg, 512 connections and 15 s on every saturated pass, lowc at 32 connections for 10 s, k6 open loop at 20000 req/s with 256 VUs (1 s ramp then 15 s), medians of 3 interleaved rounds. Run dir: results/20260924T191959Z-c7a.8xlarge-grpc.

Closed loop (h2load, c=512, medians; the hello row also under wrk):

leg	request	req/s (median)	spread	Pss MiB
rapira-http-h1 (wrk)	GET /?name=you	939,155	1.0%	230.6
rapira-http-h1	GET /?name=you (h2load --h1)	919,936	1.3%	230.6
rapira-connect-h1	Connect proto over HTTP/1.1	815,299	0.3%	205.7
rapira-connectjson-h1	Connect JSON over HTTP/1.1	766,225	4.9%	223.6
rapira-grpcweb-h1	gRPC-Web over HTTP/1.1	827,481	1.0%	209.3
rapira-connect-h2c	Connect proto over h2c	815,498	1.9%	249.6
rapira-grpc	gRPC over h2c	810,216	0.6%	248.0
rr-grpc	gRPC over h2c	137,787	0.9%	542.2

Low concurrency (h2load, c=32, medians of p50 / p99 / avg):

leg	p50	p99	avg
rapira-http-h1	0.16ms	0.25ms	0.16ms
rapira-connect-h1	0.19ms	0.29ms	0.19ms
rapira-connectjson-h1	0.18ms	0.29ms	0.18ms
rapira-grpcweb-h1	0.18ms	0.28ms	0.18ms
rapira-connect-h2c	0.15ms	0.25ms	0.16ms
rapira-grpc	0.16ms	0.26ms	0.16ms
rr-grpc	0.30ms	1.22ms	0.34ms

Open loop (k6, target 20000 req/s, achieved req/s is the average over the 16 s run including the 1 s ramp; rapira-connect-h2c carries no k6 pass):

leg	req/s (achieved)	p50	p99	p99.9	failed checks
rapira-http-h1	19,371	0.13ms	0.23ms	0.92ms	0
rapira-connect-h1	19,369	0.13ms	0.43ms	1.98ms	0
rapira-connectjson-h1	19,370	0.13ms	0.26ms	0.96ms	0
rapira-grpcweb-h1	19,371	0.13ms	0.33ms	1.10ms	0
rapira-grpc	19,370	0.16ms	0.49ms	1.12ms	0
rr-grpc	19,370	0.24ms	1.16ms	2.90ms	0

• Flags: every rapira row carries server_unsaturated (server CPU 75 to 83%, loader 62 to 74%) and lowc_doubled (5 to 10 of 32 workers held two connections during the lowc warm-up); rr-grpc carries server_unsaturated (server 69%, loader 27%). No generator_bound, ena_throttled, worker_churn, or log_growth on any cell; no voided or invalid cell.
• The connection sweep (5 s cells, rapira-grpc and rapira-http-h1): rapira-grpc 575k at c=256, 812k at 512, 1,047k at 1024 with generator_bound; hello h2load --h1 712k, 920k, 1,257k with generator_bound at 1024. One c7a.4xlarge loader binds before the server reaches 90% CPU on these rows, so the c=512 rows are not a server maximum, and the c=1024 values are floors. Run dirs: results/20260924T191554Z-c7a.8xlarge-grpc (256), results/20260924T191335Z-c7a.8xlarge-grpc (512), and results/20260924T191704Z-c7a.8xlarge-grpc (1024).
• The thread check on rapira-grpc at c=512, 5 s cells: h2load -t8 gives 684k req/s (results/20260924T191813Z-c7a.8xlarge-grpc) and -t16 gives 812k (results/20260924T191335Z-c7a.8xlarge-grpc), so the loader thread count also limits the c=512 rows.
• The forced-error check: a dispatcher that answers fail() on every 500th call was voided by the byte rule alone (short responses: data=151664695 succeeded=1669983 expect_len=91, results/20260924T191846Z-c7a.8xlarge-grpc); the probe and the log rule did not fire.
• RoadRunner runs with logs.level: error and its grpc and server channels at panic, because both log one ERROR line per queued call that h2load cancels at the end of a pass (560 lines at c=512 in the container check).
• The open-loop rate ramps from 0 to 20000 req/s over 1 s because k6 starts its schedule before its VUs are active; the first smoke run dropped 77 to 147 iterations at the start with a constant rate.
• What the tables do not show: the Rust transport ceiling of the plan was removed on 2026-09-24 as not a fair row; the suite benches the rapira binary only.

AWS baseline, 2026-08-30

Rig: c7a.8xlarge server + c7a.4xlarge loader, eu-central-1a, plain release builds (no frame pointers), 32 workers, wrk c=8000 d=15s, medians of 3 interleaved rounds. Run dirs: results/20260830T140609Z-c7a.8xlarge-ab and results/20260830T142132Z-c7a.8xlarge-fleet, provenance in each run-meta.json.

A/B, main 5d188a0 (pr) vs release v0.7.0 0612b16 (base), hello workload:

mode	v0.7.0	main	delta
dispatcher	1,087,148	1,268,192	+16.7%
worker	1,014,621	1,211,527	+19.4%
classic	501,769	491,877	-2.0% (inside the ~7% round spread)

• Every v0.7.0 cell carried wrk read errors (connection resets): ~700 per 15s cell in classic, 7 to 18 in the resident modes. Every main cell had zero. The strict report voided the v0.7.0 cells; the table above is from the raw wrk output with that caveat. The reset fix and the throughput gain both trace to the per-connection PHPHandler work.
• main lowc p50: 0.14 to 0.16 ms across modes at c=32.

Fleet (main, plain build), hello workload:

leg	req/s (median)	note
swoole	1,285,881	generator_bound: a pair ceiling, not a swoole ceiling
rapira-dispatcher	1,267,571	at the pair ceiling; one round PPS-throttled to 728k, median unaffected
rapira-worker	1,149,696
rapira-static-miss	574,679
rapira-classic	480,928
roadrunner	143,996	ext-protobuf loaded
fpm	124,502
franken	104,467	server_unsaturated at the num 32 / num_threads 33 parity pool
franken-static-miss	103,620	equals franken hello: php_server stats the docroot every request

• The pair ceiling is ~1.27M req/s hello: the loader sits at ~96% CPU and the Nitro PPS allowance starts shaving packets in the same zone. A server faster than that needs a bigger loader or two loaders.
• Static hit (27 KiB css): rapira and franken both peg ~55.7k req/s = 11.5 Gbps, the wire itself. Throughput does not separate them at this file size; the tail does: franken timed out 61 to 68 requests per cell, rapira 11 to 19. All hit cells voided on those timeouts by design.
• Static miss on rapira costs ~50% of worker throughput at 32 cores (575k vs 1,150k). The maindev rig measured ~35% at smaller scale. The probe path is worth a look in core.
• k6 probe (16-core loader, VUS=256): rapira modes 0.82 to 0.89 ms avg, fpm and roadrunner ~1.9 ms, franken 2.7 ms, zero failed checks on every leg.

Full bench (frameworks, main vs v0.7.0), 2026-08-30

Rig: the fixed c7a pair, plain release builds, eu-central-1a, 32 workers/threads/children per leg, wrk c=2048 d=15s, lowc c=32, k6 VUS=256, medians of 3 interleaved rounds. Run dir: results/20260830T160204Z-c7a.8xlarge-frameworks. Servers: rapira main 5d188a0 (pr), rapira v0.7.0 0612b16 (base, classic only), FrankenPHP 1.12.4 (static, its own PHP 8.5.8), php-fpm + nginx (Fedora PHP 8.5.9 NTS, same build rapira embeds). Apps: Symfony 7.4.17 (prod, warmed cache, dump-env) and Laravel 12.68.0 + Octane 2.19.1 (config and route cached, array session and cache). The Laravel worker legs run Octane machinery on both servers (stock octane FrankenPHP entry vs an Octane-Worker rapira bridge); the Symfony worker legs run one byte-identical kernel-loop entry on both.

Symfony (hello route through the full kernel):

server	mode	req/s (median)	lowc p50	lowc p99	spread
rapira-main	worker	186,010	0.27ms	0.60ms	3.0%
franken	worker	92,338	0.36ms	1.26ms	1.2%
rapira-main	classic	51,054	0.90ms	2.29ms	0.6%
rapira-0.7.0	classic	50,002*	0.89ms	1.87ms	0.2%
franken	classic	42,362	0.88ms	2.31ms	7.4%
fpm	classic	38,955	0.86ms	2.12ms	1.3%

Laravel (Octane on the worker legs):

server	mode	req/s (median)	lowc p50	lowc p99	spread
rapira-main	worker	33,268	1.57ms	3.44ms	1.0%
franken	worker	30,422	1.19ms	3.04ms	0.5%
rapira-main	classic	14,604	3.94ms	7.98ms	0.6%
rapira-0.7.0	classic	14,474*	3.92ms	7.99ms	0.7%
fpm	classic	13,412	2.27ms	4.49ms	1.3%
franken	classic	12,587	2.68ms	5.22ms	2.0%

* Every v0.7.0 cell carried a handful of wrk read errors (10 to 33 resets per 15s cell, the same class the hello A/B documented) and was voided by the strict report; these two rows are medians from the raw wrk output with that caveat. All other cells are clean; zero failed k6 checks anywhere.

• Symfony worker: rapira 2.0x franken. Laravel worker: rapira +9% over stock Octane-on-franken; Octane's per-request machinery, not the server, dominates that leg.
• Worker over classic: Symfony 3.6x on rapira, 2.2x on franken; Laravel 2.3x on rapira, 2.4x on franken.
• Classic: rapira main leads both frameworks (+31% over fpm on Symfony, +9% on Laravel), and main vs v0.7.0 classic is flat (+2.1% / +0.9%), consistent with the hello A/B.
• Laravel classic lowc oddity: rapira p50 3.94ms vs fpm 2.27ms per request despite higher rapira throughput at saturation; Symfony classic shows no such gap (0.90 vs 0.86). A per-request cost on big codebases in classic mode is worth a look in core; the r2 cell measured 2.46ms, so round variance is high on this metric.
• v0.7.0 flush bug found while building the legs: PHP flush() after output makes v0.7.0 seal the response lengthless and the front closes the connection (~every request under symfony/runtime's Response::send(true) tail; 170k resets per 5s cell). Fixed on main. The symfony classic entry now uses send(false) on all servers; the residual v0.7.0 resets above are the pre-existing hello-class ones.
• Flags: symfony-franken worker and classic plus symfony-fpm ran server_unsaturated at c=2048 (their pool shape, matching the hello fleet); the rapira and laravel legs saturated.

Static cache bench (feature/cache-for-static vs main vs FrankenPHP), 2026-08-31

Rig: the fixed c7a pair, plain release builds, eu-central-1a, AMI ami-040c604473c52f25f, 32 workers/threads per leg, wrk c=8000 d=15s, lowc c=32, k6 VUS=256, medians of 3 interleaved rounds, new make bench_static driver. Run dir: results/20260831T190826Z-c7a.8xlarge-static. Servers: rapira feature/cache-for-static 649b8c7 (pr, per-worker in-memory file cache: 16 MiB/process, 256 KiB/file cap, 1 s freshness), rapira main 5d188a0 (base, uncached ServeDir), FrankenPHP 1.12.4. Apps behind the fallthrough: the hello worker and the Symfony 7.4.18 kernel-loop worker. hit fetches tiny.css (128 B) with PHP never running; miss requests the hello URL through the same server shape so the static probe runs and falls through to PHP; plain is the same rapira worker with no static middleware (franken has no such shape).

Asset size is the whole game: 27 KiB pegs the 12.5 Gbps wire at ~56k req/s on every leg (2026-08-30 fleet run), and even 1 KiB put the cache leg at ~11 Gbps with shaved bw allowances and the loader over its 6.25 Gbps sustained baseline (first launch of this run, aborted after one cell). At 128 B every hit leg is CPU-bound except the cache legs, which run into the ~1.27M req/s pair ceiling instead (loader ~95% CPU): the cache hit rows are floors, not ceilings, with busy_server only ~72%.

Hello app:

leg	req/s (median)	lowc p50	lowc p99	spread
rapira-cache hit	>= 1,266,878 (pair ceiling)	0.12ms	0.18ms	0.6%
rapira-cache miss	973,109	0.14ms	0.25ms	2.0%
rapira-cache plain	1,161,508	0.14ms	0.24ms	0.9%
rapira-main hit	261,909	0.24ms	0.43ms	1.0%
rapira-main miss	578,811	0.17ms	0.31ms	0.6%
rapira-main plain	1,174,781	0.14ms	0.23ms	1.4%
franken hit	139,749	0.21ms	1.22ms	0.4%
franken miss	104,213	0.28ms	1.77ms	15.0%

Symfony (kernel-loop worker):

leg	req/s (median)	lowc p50	lowc p99	spread
rapira-cache hit	>= 1,263,239 (pair ceiling)	0.12ms	0.19ms	1.0%
rapira-cache miss	173,863	0.27ms	0.62ms	4.7%
rapira-cache plain	173,755	0.25ms	0.54ms	1.1%
rapira-main hit	259,373	0.25ms	0.43ms	1.6%
rapira-main miss	137,706	0.32ms	0.83ms	3.5%
rapira-main plain	176,592	0.25ms	0.45ms	0.7%
franken hit	145,077	0.20ms	1.21ms	0.8%
franken miss	89,695	0.34ms	1.79ms	0.6%

• Hit: the cache is worth at least 4.8x over main's uncached ServeDir path (1,267k floor vs 262k clean server ceiling at 96% busy) and at least 8.7x over franken's php_server file path; the true cache ceiling is unmeasurable on this pair. lowc per-request p50 halves (0.12ms vs 0.24-0.25ms).
• Main's uncached hit (262k) is slower than its own miss fallthrough to resident hello PHP (579k): stat + open + read through spawn_blocking per request costs more than a full PHP hello round trip. The maindev-era note on ServeDir vs Caddy pointed the same way.
• Miss (the branch's regression control) improved, not regressed: the probe tax vs plain is -50.7% on main hello and -16.2% on the branch (973k vs 579k, +68%); on symfony the branch's tax vanishes entirely (173.9k miss vs 173.8k plain, main -22.0%). The 2026-08-30 fleet's "static miss costs ~50% of worker" finding is specific to main.
• Plain (middleware off): cache vs main is -1.1% hello / -1.6% symfony, inside or bordering the round spread; no baseline regression from the branch.
• Flags: every cache hit cell is generator_bound at the pair ceiling, and four cells across the run grazed the PPS allowance by 20 to 946 packets (~0.004% of a cell's packets; ena_throttled per the strict rule, magnitudes noise). franken legs ran server_unsaturated at its usual pool shape. Zero voided cells, zero failed k6 checks, 48/48 cells.
• Rig note: WRK_CONNS defaulted to the hello 250x (c=8000) for all legs including symfony; the symfony worker at ~174k req/s holds a ~46ms queue at that depth, well under the 5s timeout, and no cell voided. A future symfony-heavy static run could pin 64x for parity with bench_frameworks.

Decisions and their reasons

• Loader capacity. wrk2 sends about 31000 req/s per loader vCPU at about 92% CPU with 625 connections per thread, which is about 34000 req/s per vCPU at 100% by linear scale. Measured in the runs of 2026-09-25 to 2026-09-27: 250000 req/s from one c7a.2xlarge loader of 8 vCPUs at 90 to 93% CPU. A loader above 70% busy CPU flags loader_busy.
• Plain builds for published tables: the prebuilt competitors do not carry frame pointers, so a frame-pointer rapira would understate its own gap. Perf sessions rebuild with frame pointers on demand.
• Strict voiding: a cell with resets, timeouts, or missing generator output is listed and excluded, never averaged. The v0.7.0 run shows why: averaging reset-y cells would have hidden the finding.
• vCPU quota L-1216C47A raised 32 to 64 on 2026-08-30; the default rig needs 16.
• Null-run calibrations on the c7a.xlarge pair (2026-08-30, before the sizes were fixed): dispatcher ~240k, worker ~210k, classic ~80k req/s at c=1000, null deltas within noise at one round.

Maindev era, retired rig

The pre-AWS harness ran servers on maindev (32 cores) with the load from a Mac over LAN or on-box. Its numbers are not comparable to the AWS baseline and are recorded here only as history. The harness itself is in git history before the 2026-08-30 wipe.

• Framework bench 2026-08-29 (on-box wrk 12t/500c/15s, medians of 3, 32 workers, rapira at feature/static-middleware): hello rapira 1,068,750 vs franken 86,349. Laravel classic 6,807 vs 5,736; Laravel worker 38,262 (bridge) vs 23,089 (Octane). Symfony classic 26,911 vs 21,968; Symfony worker 148,033 vs 72,839. Worker over classic: 5.6x Laravel, 5.5x Symfony. Laravel is ~4x heavier than Symfony per matching cell.
• Static legs 2026-08-29 (182 KiB css, on-box): rapira hit 75,191 at 5.4 ms, rapira miss 692,822 at 0.69 ms (miss tax ~-35% vs the 1,068k hello ceiling), franken hit 191,234 at 6.0 ms (plain Caddy file_server). Caddy beat the ServeDir hit path ~2.5x on-box: sendfile from page cache vs a probe plus spawn_blocking per 64 KiB chunk through hyper. Over a 12.5 Gbps wire this difference disappears; on faster links or loopback it returns.
• Framework rig operational lessons that shaped this rig: FrankenPHP ignores TERM while draining (stop is TERM, wait, KILL); an unpinned franken pool auto-sizes and breaks worker parity; backgrounded servers must close their stdio or the ssh pipe hangs.
• The Mac-as-loader ceiling (~140k rps over LAN) is what moved benching to EC2; see rapira-rs/rapira#97.