You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit bfce503
Browse filesBrowse the repository at this point in the historyBrowse files
Exact spreadsheet calculations, exact matches first, only the needed columns
- Counts, distinct values, top N (by rows or by different people), sums,
averages, min, max and lists, computed by code over every row. ask_document
turns plainly worded questions into a plan that Jev checks against the
question (verdict "calculated"); the new calculate tool takes plans directly.
- Spreadsheet lookups: rows holding the exact value asked about come first
(match: exact / partial), show only the columns the question needs, and
count identical records exactly.
- Load progress while still_reading, a coverage report, and a note that
document_tokens is an estimate.
- "List everyone" questions in Word files also return the headings of
neighbouring sections; the disagreement flag needs two relevant passages.
- Benchmarks: PDF 8/8, Word 11/11 (panel question now needs all 7 members),
Excel 11/11 with the 3 counts exact (9, 869, Roob Inc 286).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: README.md
+41-14Lines changed: 41 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,8 @@ ReadyCode Reader is a free, open-source [MCP](https://modelcontextprotocol.io) s
7
7
-**Not in the document:** if the file can't answer, your AI is told `not_in_document` instead of getting passages to guess from.
8
8
-**Passages disagree:** if two passages give different values, your AI is told to report both.
9
9
-**Hidden instructions:** passages that try to instruct the AI reading them are flagged and treated as data only.
10
-
-**Needs a calculation:** spreadsheet questions like "how many…" or "list every…" are flagged instead of being estimated from a few rows.
10
+
-**Exact spreadsheet calculations:** "how many…", "which … appears most often", totals, averages and "list everyone who…" are computed by code over every row and come back as numbers, with the sheets, column and conditions used. A model never estimates them.
11
+
-**Exact matches first:** a spreadsheet row holding the exact name you asked about comes before look-alikes ("Nat Becker" before "Prof. Nat Becker II"), and rows show only the columns the question needs.
11
12
12
13
## Standard AI vs Reader
13
14
@@ -17,22 +18,32 @@ Same questions, same AI, fresh sessions: once the normal way (the AI reads the f
17
18
|---|---|---|
18
19
|**Codex**, 200-page NASA PDF, 8 questions | 8/8 correct · **913,468 tokens** · about 3 min | 8/8 correct · **41,997 tokens** · about 20 s |
19
20
|**Grok 4.7 in Cursor**, same PDF | 8/8 correct · read the whole book (about 45,000 tokens) · about 4 min | 8/8 correct · **4,693 tokens** of evidence · about 2 min |
21
+
|**Codex** and **Grok 4.7 in Cursor**, 263,000-token Word report, 11 questions | 11/11 · 4–5 tool calls · 2–4 min · no citations | 11/11 · 2 calls · 30 s–1 min · section and paragraph on every answer |
22
+
|**Codex** and **Grok 4.7 in Cursor**, 100 MB Excel workbook (1.1 million rows), 11 questions | 11/11 · wrote its own parser script · 7 steps | 8/8 lookups in 2 calls, no code; declined the 3 counting questions (since fixed, see below) |
20
23
21
-
Token counts for Codex come from Codex's own session log. That's **95% fewer tokens and about 9× faster** for the same answers. Most of the saving comes from steps: an agent re-sends its whole conversation at every step, and with Reader it needed 2 steps instead of 17.
24
+
Token counts for Codex come from Codex's own session log. On the PDF that's **95% fewer tokens and about 9× faster** for the same answers. Most of the saving comes from steps: an agent re-sends its whole conversation at every step, and with Reader it needed 2 steps instead of 17. On the Word report the gain was speed, fewer steps and citations rather than tokens. Full results: [`benchmark/head-to-head.md`](benchmark/head-to-head.md).
22
25
23
26
## Measured on large files
24
27
25
28
Every expected answer below was checked by code against the file, and every "not in the document" question was checked to be truly absent. Scripts and question files are in [`benchmark/`](benchmark/).
26
29
27
30
| File | Size as text | Correct | Tokens returned | Time for all questions | Check cost |
28
31
|---|---|---|---|---|---|
29
-
| NASA *Earth at Night* (PDF, 200 pages) | 42,282 tokens | 8/8 | 3,487 |2.3 s | $0.0027 |
30
-
| Australian Universities Accord Final Report (Word) | 263,487 tokens | 11/11 |15,790| 1.7 s | $0.0042|
31
-
| Sample contacts workbook (Excel, 100 MB, 1.1 million rows) | 271,424,159 tokens | 11/11 | 2,323|0.9 s | $0.0038|
32
+
| NASA *Earth at Night* (PDF, 200 pages) | 42,282 tokens | 8/8 | 3,487 |1.6 s | $0.0027 |
33
+
| Australian Universities Accord Final Report (Word) | 263,487 tokens | 11/11 |14,991| 1.1 s | $0.0044|
The Word report is larger than many AI context windows, and the spreadsheet is more than a hundred times larger than any. Each set includes questions the file cannot answer; Reader said `not_in_document` every time. On the spreadsheet it also declined 3 counting and ranking questions rather than guess.
36
+
The Word report is larger than many AI context windows, and the spreadsheet is more than a hundred times larger than any. Each set includes questions the file cannot answer; Reader said `not_in_document` every time.
34
37
35
-
Honest limits of these results: they are our own runs on three files, not an independent study, and head-to-head runs against a standard AI exist so far only for the PDF.
38
+
On the spreadsheet, 8 questions are lookups. Each is answered with only the name and the column asked about, 13–40 tokens each. The other 3 are exact calculations over all 1.1 million rows:
39
+
40
+
- "How many people work at Smith-Hickle?": 9 rows, and Reader adds that they hold only 1 different name.
41
+
- "List everyone whose job title is Floor Layer": 869 rows, 150 different people, listed.
42
+
- "Which company appears most often?": Roob Inc, 286 rows.
43
+
44
+
In the Word set, "Who else was on the panel?" needs all 7 other members, including two ex-officio members found only through section headings.
45
+
46
+
Honest limits of these results: they are our own runs on three files, not an independent study. The head-to-head runs above were made before exact calculations existed and have not been re-run since.
**Claude Desktop**: add the same entry as Cursor under `mcpServers` in `claude_desktop_config.json`.
66
77
67
-
Restart your tool and check that `readycode-reader` shows three tools: `load_document`, `ask_document` and `list_documents`.
78
+
Restart your tool and check that `readycode-reader` shows four tools: `load_document`, `ask_document`, `calculate` and `list_documents`.
68
79
69
80
## Use
70
81
@@ -76,9 +87,22 @@ Your AI loads the file, sees its contents, asks Reader specific questions and an
76
87
77
88
-`load_document(path)` reads a PDF, Word (.docx), Excel (.xlsx), TXT, Markdown, CSV or JSON file. It returns the file's size and **its contents**: a PDF's bookmarks with page numbers, the Word headings, or a spreadsheet's sheets, columns and an example row. That tells the AI what the file covers before it asks anything.
78
89
-`ask_document(questions: [...])` asks up to 20 questions in one call. Each answer carries its passages with page numbers (and printed page numbers), Word sections and paragraphs, or spreadsheet sheet and row.
90
+
-`calculate(operation, column, where, sheet, unique_by, n)` runs an exact calculation over a loaded spreadsheet: `count`, `distinct`, `top` (most or fewest), `sum`, `average`, `min`, `max` or `list`. It takes conditions (equals, contains, not equals, above, below), one sheet or all, and can count people instead of rows (`unique_by: "Name"`). You rarely need it: `ask_document` turns plainly worded questions into these plans, has each plan checked against the question, and answers with verdict `calculated`.
79
91
-`list_documents()` lists what is loaded.
80
92
81
-
Very large files keep reading in the background: `load_document` may answer `still_reading`, and `ask_document` waits for the file to finish.
93
+
A calculated answer looks like this:
94
+
95
+
```json
96
+
{ "question": "How many people in the file work at Smith-Hickle?", "verdict": "calculated",
97
+
"answer": "9 rows where Company is \"Smith-Hickle\" in all 4 sheets, holding 1 different Name value.",
98
+
"plan": "Count the rows where Company is \"Smith-Hickle\" in all sheets, and count the different Name values among them.",
"result": 9, "unique": 1, "method": "Computed by code over every row; nothing estimated." } }
103
+
```
104
+
105
+
Very large files keep reading in the background: `load_document` may answer `still_reading` with how far it has got, and `ask_document` waits for the file to finish. `load_document` also reports what was read (every sheet and row, and anything skipped), so a "not in the document" answer can be trusted. `document_tokens` is an estimate of the whole file's size as text (about 3.5 characters per token), for comparison; it is not model usage.
82
106
83
107
## Try it in your browser
84
108
@@ -94,17 +118,18 @@ The same engine runs in a web page: choose a file, add your OpenRouter key, ask.
94
118
- is there enough here to answer it?
95
119
- do these passages disagree?
96
120
- does any passage try to instruct the AI?
97
-
- for spreadsheets, does it need a calculation over every row?
121
+
- for spreadsheets, does it need a calculation over every row, and does the calculation plan Reader made fit the question?
98
122
99
123
Nothing is set up per document. The check costs about $0.0004 a question on your key and takes about a second.
100
-
5.**Return** only the passages that answer, most relevant first, with the verdicts and token counts.
124
+
5.**Calculate** (spreadsheets): when a question needs every row, code runs the plan over the whole workbook. Values that differ only in capitals or spacing count as one, and rows can be counted as rows or as different people. A 1.1-million-row workbook takes about 1–2 seconds.
125
+
6.**Return** only the passages that answer, most relevant first, with the verdicts and token counts. For "list everyone" questions in Word files, the headings of neighbouring sections come too, since a list often runs over several sections.
101
126
102
127
## Limits (read these)
103
128
104
129
-**Picture-only pages** (scans, images) have no text to search and are listed as such; pictures inside Word files are counted, not read. Reading them needs a vision model; that's planned.
105
130
-**Summaries of a whole document** are a different job. Reader answers specific questions.
106
-
-**Spreadsheet calculations**(counts, totals, averages, rankings, "list every row that…") are flagged, not answered. Exact calculations are planned.
107
-
-**The "passages disagree" check**sometimes fires when passages don't really disagree. Treat it as "double-check these sources".
131
+
-**Spreadsheet calculations**cover counts, distinct values, rankings, totals, averages, lowest and highest values, and lists, with simple conditions. Formulas, pivot tables, date ranges and combining sheets with different columns are not supported yet. When Reader can't turn a question into a plan it is sure of, it answers `needs_calculation` rather than guess, and your AI can call `calculate` directly.
132
+
-**The "passages disagree" check**now needs two relevant passages about the same thing, but it can still fire when passages don't really disagree. Treat it as "double-check these sources".
108
133
-**Old formats** (`.doc`, `.xls`, `.xlsb`) aren't supported; save them as `.docx` or `.xlsx`.
109
134
-**It depends on Jev.** If the decision model is unavailable, Reader still returns the top search results, marked `unchecked`.
110
135
-**Check the passages.** Every answer comes with its sources so you can.
@@ -120,4 +145,6 @@ The same engine runs in a web page: choose a file, add your OpenRouter key, ask.
120
145
121
146
Apache-2.0. Built by [ReadyCode](https://readycode.ai). If you build on it, keep the NOTICE file, and we'd love a link back. "ReadyCode" is our name; please don't use it for forks.
122
147
123
-
**More tools from ReadyCode, and early access to the hosted version:**[readycode.ai](https://readycode.ai)
148
+
**Source:**[github.com/ReadycodeAI/readycode-reader](https://github.com/ReadycodeAI/readycode-reader). Issues and ideas are welcome there.
149
+
150
+
**News about Reader and new ReadyCode tools, and early access to the hosted version:**[readycode.ai/reader](https://readycode.ai/reader)
Copy file name to clipboardExpand all lines: benchmark/README.md
+19-7Lines changed: 19 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,21 +17,33 @@ A question passes when:
17
17
18
18
-**answer questions:** Reader's verdict is `answer_from_passages` (or `low_confidence`) and the passages it returned contain every expected phrase;
19
19
-**not-in-the-document questions:** the verdict is `not_in_document`;
20
-
-**calculation questions** (spreadsheets): the verdict is `needs_calculation`.
20
+
-**calculation questions** (spreadsheets): the verdict is `calculated` and the number matches the one the benchmark computes itself over every row (the matching row count, or the top value and its row count).
21
+
22
+
For a question that asks for a complete list, the headings Reader returns in `nearby_sections` count as returned text: they are how a list that runs over several sections is found.
21
23
22
24
## The files
23
25
24
26
-**NASA *Earth at Night*** (2019, 200-page PDF, NP-2019-07-2739-HQ): free from nasa.gov. Questions: [`nasa-earth-at-night.json`](nasa-earth-at-night.json), 6 answerable and 2 that the book cannot answer.
25
-
-**Australian Universities Accord – Final Report** (Australian Government, Word .docx, about 263,000 tokens): published by the Department of Education. Questions: [`universities-accord-word.json`](universities-accord-word.json), 9 answerable and 2 that the report cannot answer.
26
-
-**A 100 MB contacts workbook** (1.1 million rows of fake people generated with the Faker library, 4 sheets, columns Name, Email, Phone, Address, Company, Text, Description, Job Title). [`excel-lookups.cjs`](excel-lookups.cjs) picks rows at random (fixed seed) and computes each expected answer over every row: 5 lookups by name, 2 reverse lookups (by email and by phone), 1 name that isn't in the file, and 3 countingor ranking questions. It works on any workbook with those columns.
27
+
-**Australian Universities Accord – Final Report** (Australian Government, Word .docx, about 263,000 tokens): published by the Department of Education. Questions: [`universities-accord-word.json`](universities-accord-word.json), 9 answerable and 2 that the report cannot answer. "Who else was on the Review Panel?" needs all 7 other members, two of them ex-officio.
28
+
-**A 100 MB contacts workbook** (1.1 million rows of fake people generated with the Faker library, 4 sheets, columns Name, Email, Phone, Address, Company, Text, Description, Job Title). [`excel-lookups.cjs`](excel-lookups.cjs) picks rows at random (fixed seed) and computes each expected answer over every row: 5 lookups by name, 2 reverse lookups (by email and by phone), 1 name that isn't in the file, and 3 counting, listing or ranking questions whose answers it also computes itself. It works on any workbook with those columns.
27
29
28
30
## Results (30 September 2026, Jev `typesafe/jev-1.13`)
29
31
30
32
| File | Correct | Document tokens | Tokens returned | Load | All questions | Check cost |
31
33
|---|---|---|---|---|---|---|
32
-
| NASA PDF | 8/8 | 42,282 | 3,487 | 0.6 s | 2.3 s | $0.0027 |
33
-
| Word report | 11/11 | 263,487 | 15,790 | under 1 s (cached) | 1.7 s | $0.0042 |
34
-
| Excel workbook | 11/11 | 271,424,159 | 2,323 | 14.3 s | 0.9 s | $0.0038 |
34
+
| NASA PDF | 8/8 | 42,282 | 3,487 | under 1 s (cached) | 1.6 s | $0.0027 |
35
+
| Word report | 11/11 | 263,487 | 14,991 | under 1 s (cached) | 1.1 s | $0.0044 |
36
+
| Excel workbook | 11/11 | 271,424,159 (estimate) | 2,494 | 16.2 s | 2.4 s | $0.0022 |
37
+
38
+
On the workbook, the 8 lookups return only the columns each question needs (13–40 tokens each). The 3 calculations are exact:
39
+
40
+
- 9 rows at Smith-Hickle, which hold 1 different name.
41
+
- 869 Floor Layer rows, 150 different people, listed.
42
+
- Roob Inc as the most frequent company, with 286 rows.
43
+
44
+
The list is 1,934 of the 2,494 tokens.
45
+
46
+
Earlier results on the same day, before exact calculations, column trimming and the list change: Word 15,790 tokens returned; Excel 2,323 tokens returned, with the 3 calculation questions declined (`needs_calculation`) rather than answered.
35
47
36
48
## Standard AI vs Reader (NASA PDF)
37
49
@@ -44,4 +56,4 @@ The same 8 questions were given to each AI twice, in fresh sessions: once readin
44
56
45
57
Most of Codex's saving is cached tokens, which cost less, so the cost saving is smaller than the token saving: about 85% rather than 95%.
46
58
47
-
Head-to-head runs for the Word and Excel files haven't been done yet.
59
+
Head-to-head runs for the Word and Excel files, against Codex and Grok, are in [`head-to-head.md`](head-to-head.md).
0 commit comments