|
| 1 | +# Watch Mode Benchmark: Evented vs Polling |
| 2 | + |
| 3 | +Benchmark comparing the old 250ms polling watch implementation (npm `hunkdiff@0.17.0`) against the evented Chokidar-based hybrid observer (`elucid/file-watch` PR #531). |
| 4 | + |
| 5 | +## Setup |
| 6 | + |
| 7 | +- **Repo:** `modem/modem` (large production monorepo) |
| 8 | +- **Branch state:** dirty working tree with untracked files |
| 9 | +- **OLD binary:** `/Users/justin/.npm-global/bin/hunk` (npm release 0.17.0, 250ms `setInterval` polling) |
| 10 | +- **NEW binary:** `/Users/justin/.local/bin/hunk` (PR build from `elucid/file-watch`, Chokidar + 10s safety poll) |
| 11 | +- **Command:** `hunk diff --watch` |
| 12 | + |
| 13 | +## Method |
| 14 | + |
| 15 | +### Git subprocess counting |
| 16 | + |
| 17 | +A wrapper script was placed ahead of real `git` on PATH. It logs every invocation with a timestamp to a per-version log file, then `exec`s the real git binary: |
| 18 | + |
| 19 | +```bash |
| 20 | +#!/bin/bash |
| 21 | +printf '%s %s\n' "$(date +%s.%N)" "$*" >> "${HUNK_BENCH_GIT_LOG}" |
| 22 | +exec /nix/store/.../git "$@" |
| 23 | +``` |
| 24 | + |
| 25 | +Both versions were launched in separate herdr panes with `HUNK_BENCH_GIT_LOG` and `PATH` set via `--env` on `herdr pane split`. After both rendered their initial diff, the log files were zeroed and CPU times recorded. The benchmark then sampled every 10 seconds for 60 seconds using `ps -p $PID -o cputime=` and `wc -l` on the log files. |
| 26 | + |
| 27 | +### Startup time |
| 28 | + |
| 29 | +Both versions were launched sequentially in herdr panes on the same repo. A polling loop read each pane's visible content via `herdr pane read` at ~20ms intervals, looking for hunk's menu bar (`File View`). The elapsed time from `herdr pane run` to first detection was recorded. Each version was quit (`q`) and relaunched between trials. Five trials were run per version. |
| 30 | + |
| 31 | +## Results |
| 32 | + |
| 33 | +### Idle overhead (60 seconds, no file changes) |
| 34 | + |
| 35 | +Sampled at 10-second intervals: |
| 36 | + |
| 37 | +``` |
| 38 | +t=10s OLD cpu=0:02.68 git= 199 | NEW cpu=0:01.61 git= 6 |
| 39 | +t=20s OLD cpu=0:02.93 git= 319 | NEW cpu=0:01.63 git= 9 |
| 40 | +t=30s OLD cpu=0:03.14 git= 439 | NEW cpu=0:01.65 git= 12 |
| 41 | +t=40s OLD cpu=0:03.35 git= 559 | NEW cpu=0:01.67 git= 15 |
| 42 | +t=50s OLD cpu=0:03.55 git= 679 | NEW cpu=0:01.69 git= 18 |
| 43 | +t=60s OLD cpu=0:03.76 git= 799 | NEW cpu=0:01.71 git= 21 |
| 44 | +``` |
| 45 | + |
| 46 | +| Metric | OLD (polling) | NEW (evented) | Improvement | |
| 47 | +| -------------------------------- | ------------- | ------------- | ------------- | |
| 48 | +| Git invocations in 60s | 799 | 21 | **38× fewer** | |
| 49 | +| Git calls/second | ~13.3/s | ~0.35/s | — | |
| 50 | +| CPU time consumed (idle portion) | ~1.50s | ~0.13s | **~12× less** | |
| 51 | + |
| 52 | +The old version fires multiple git commands per 250ms poll cycle (~13/s). The new version fires a small batch every ~10 seconds (safety poll only). |
| 53 | + |
| 54 | +### Startup time (5 trials) |
| 55 | + |
| 56 | +| Trial | OLD | NEW | |
| 57 | +| -------- | ---------- | ---------- | |
| 58 | +| 1 | 1708ms | 1674ms | |
| 59 | +| 2 | 1669ms | 1703ms | |
| 60 | +| 3 | 1671ms | 1670ms | |
| 61 | +| 4 | 1662ms | 1756ms | |
| 62 | +| 5 | 1684ms | 1691ms | |
| 63 | +| **Mean** | **1679ms** | **1699ms** | |
| 64 | + |
| 65 | +Startup time is identical within noise (~1.7s). Both versions are dominated by the initial `git diff` load cost on this repo. The PR's improvement is entirely in the idle steady-state after the diff is rendered. |
| 66 | + |
| 67 | +### Refresh latency (evented PR, separate E2E test repo) |
| 68 | + |
| 69 | +| Scenario | Latency | |
| 70 | +| --------------------------- | ------- | |
| 71 | +| Simple tracked-file write | ~340ms | |
| 72 | +| Atomic temp-file + rename | ~418ms | |
| 73 | +| Large 241-line diff update | ~406ms | |
| 74 | +| New untracked file creation | ~682ms | |
| 75 | + |
| 76 | +All refreshes were passive — no keyboard or mouse input was sent to hunk. |
| 77 | + |
| 78 | +## Conclusion |
| 79 | + |
| 80 | +The evented observer eliminates virtually all idle CPU and subprocess overhead while maintaining identical startup performance and sub-second passive refresh latency. The improvement scales with the number of open `--watch` sessions: each idle tab drops from ~13 git calls/second to ~0.35. |
0 commit comments