|
36 | 36 |
|
37 | 37 | Claude reads `auth.py`. Then reads it again. Then a third time after compaction wipes the session. You pay for every token. |
38 | 38 |
|
39 | | -Long sessions accumulate waste three ways. Screenshots cross the model at full resolution. A single PNG can land at 3.3 MB. The agent re-reads files it already parsed earlier in the same conversation. And when a session compacts, the summary LLM doesn't know which files were edited or which symbols mattered, so it preserves the wrong things. |
| 39 | +Long sessions accumulate waste four ways. Screenshots cross the model at full resolution. A single PNG can land at 3.3 MB. The agent re-reads files it already parsed earlier in the same conversation. When a session compacts, the summary LLM doesn't know which files were edited or which symbols mattered, so it preserves the wrong things. And every `pytest`, `npm install`, `docker build`, or `git log` dumps thousands of lines of progress bars, deprecation warnings, and passing-test names that bury the one line that actually matters. |
40 | 40 |
|
41 | | -Each one is preventable. Token-Goat intercepts all three, automatically. |
| 41 | +Each one is preventable. Token-Goat intercepts all four, automatically. |
42 | 42 |
|
43 | 43 | ## What changes |
44 | 44 |
|
45 | 45 | | Without Token-Goat | With Token-Goat | |
46 | 46 | |--------------------|-----------------| |
47 | | -| 3.3 MB screenshot lands in model context | 84 KB compressed copy — 97.4% smaller | |
| 47 | +| 3.3 MB screenshot lands in model context | 84 KB compressed copy, 97.4% smaller | |
48 | 48 | | Agent re-reads files from earlier in the session | "Already read this" reminder with narrow slice suggestion | |
49 | 49 | | Compaction forgets which files were edited | Structured session manifest injected before compact | |
50 | | -| Full file read for one function or section | `token-goat read file::symbol` — about 85% smaller | |
| 50 | +| Full file read for one function or section | `token-goat read file::symbol`, about 85% smaller | |
| 51 | +| `pytest` dumps 150 PASSED lines + dots + tracebacks | Failures-first view, 80 to 97% smaller | |
| 52 | +| `npm install` floods deprecation warnings + spinner | Errors kept; warnings collapsed by package, ~90% smaller | |
| 53 | +| `docker build` emits sha256 digests + transfer progress | Step headers + errors kept; noise dropped, ~75% smaller | |
| 54 | +| `ruff` / `eslint` / `mypy` repeat the same rule 50 times | Grouped by rule with first 3 examples, ~80% smaller | |
51 | 55 |
|
52 | 56 | > Four hours of use on the author's machine: **59.7 MB** of data that never hit the model, with an estimated **11.5 million tokens** avoided. |
53 | 57 |
|
@@ -133,6 +137,28 @@ out: ~4 KB # top-ranked files + key symbols (92% smaller) |
133 | 137 |
|
134 | 138 | `--budget` is a hard cap. Below 6 KB the output automatically switches to short-label mode (`f:` files, `s:` symbols, `c:` calls) to fit more signal per byte. `token-goat map --compact` is a shortcut for a 300-token budget when you only need the high-rank cluster. |
135 | 139 |
|
| 140 | +**5. Bash output compression** |
| 141 | + |
| 142 | +``` |
| 143 | +# Without token-goat: pytest dumps every PASSED line + dots + tracebacks. |
| 144 | +$ pytest -v tests/ |
| 145 | +... (3 KB of output, 150 PASSED lines, 1 FAILED at the bottom) |
| 146 | +
|
| 147 | +# With token-goat: the PreToolUse hook rewrites the command to |
| 148 | +# `token-goat compress --filter pytest`. The wrapper runs pytest, captures |
| 149 | +# stdout+stderr, applies the per-tool filter, and prints failures first. |
| 150 | +$ token-goat compress --filter pytest --cmd "pytest -v tests/" |
| 151 | += test session starts = |
| 152 | +collected 150 items |
| 153 | +FAILED tests/test_x.py::test_one |
| 154 | += 1 failed, 149 passed in 2.3s = |
| 155 | +
|
| 156 | +[token-goat: collapsed 149 PASSED lines] |
| 157 | +[token-goat: pytest filter compressed 4.8 KiB to 0.1 KiB (97% saved)] |
| 158 | +``` |
| 159 | + |
| 160 | +Twelve built-in filters cover the noisiest dev commands: `pytest`, `jest` / `vitest`, `cargo`, `npm` / `pnpm` / `yarn`, `docker`, `kubectl` / `helm`, `aws`, `ruff` / `eslint` / `mypy`, `git`, `make` / `gradle` / `mvn` / `go`, `terraform`, and `pip`. Each one strips ANSI escapes, collapses `\r` progress bars, dedupes repeated lines, groups linter issues by rule, keeps every error block verbatim, and caps total output at 1000 lines / 64 KiB. Disable globally with `TOKEN_GOAT_BASH_COMPRESS=0`, per-filter via `[bash_compress] disabled_filters = ["docker"]` in config.toml, or preview the output of any command with `token-goat compress --cmd '<your command>'`. |
| 161 | + |
136 | 162 | ## Install |
137 | 163 |
|
138 | 164 | **Windows requirements:** Windows 10 or 11 · Python 3.11, 3.12, or 3.13 · [uv](https://docs.astral.sh/uv/) (`winget install astral-sh.uv`) |
|
0 commit comments