Show billed input math on the user bubble and each reply's length

README.md

@@ -14,8 +14,8 @@ Inspired by [Scott Hanselman's "AI without the BS, for humans" keynote at NDC Lo
 - **Taught once, then out of the way.** Three coach marks run in order for a first-time visitor: the color rule on the first settled reply, the reply tabs on the newest unlocked one, and the cost card once two replies exist (that one opens the card's **Details** for you). Steps 1 and 2 advance on the gesture they teach — hovering a word, picking a tab — as well as on **Got it**, so following the instruction is never punished with an extra click. Progress is a single number in `localStorage`, so they do not come back. After that the same three sentences stay reachable behind `?` buttons on the legend, the tab strip, and the conversation-cost line; all three read from `lib/coachCopy.js`, so a mark and its `?` cannot drift apart.
 - **What else was considered.** Hover or tap a word for the candidate list, headed **What it considered**. It opens on **Of all words**: the model’s real probabilities across the whole vocabulary, which do *not* add up to 100% — the honest number first. **What-if: only these** re-scales just the shown candidates to add up to 100% at the current temperature, which is a different quantity and says so. If the sampled token was outside the top 5, it still gets its own row (“landed — not in the top 5”) with a real percentage instead of `0.00%`; under **Of all words**, that is its exact model probability.
 - **Temperature, live.** The candidate set is frozen when the card opens, so moving the temperature slider never makes rows appear or vanish — only the odds move. Adjusting temperature does not dismiss a pinned card, because watching the odds shift is the lesson. At `0` the top candidate takes 100% and everything else goes to zero, which is winner-take-all sampling made visible (the card rounds those zeros to `<0.001%`). The full panel behind the header's **Controls** button is grouped into **Sampling** (temperature, top-p, presence penalty, a **Make it repeatable** determinism switch), **Memory** (**Forget older turns** and **Exchanges replayed**), **Delivery** (**Stream the reply**), and **Tools** (**Let it call a weather tool**). The button always carries a `temp N.N` chip, plus one chip for every switch that has moved off its default — `streaming off`, `memory none` or `memory last N`, `tool on`, `repeatable` — so no switch can be on without the header saying so. The two secondary sliders, top-p and presence penalty, earn no chip even though every request carries them, so the panel is the only place that reports where they sit.
-- **Your text is tokens too.** The composer tokenizes what you type with `o200k_base` and the user bubble shows the pieces as alternating tints, with an `≈ N tokens` count. Send **strawberry** and watch it arrive as three pieces, not ten letters.
-- **Three full replies, and where they fork.** Each turn requests `n=3` completions. A strip reading “3 replies were written. You are reading reply” fronts tabs **1 / 2 / 3**, each with a confidence dot, and an unlocked strip carries a `?` that repeats the coach-mark sentence; older turns lock after the next user message, and a padlock explains why. A ring marks the first token where the three replies diverge — everything before it is identical, because the same prompt and the same weights produced the same tokens until the dice landed differently. Each tab also reports perplexity (“picking from ~N plausible words”).
+- **Your text is tokens too.** The composer tokenizes what you type with `o200k_base` and the user bubble shows the pieces as alternating tints, with an `≈ N tokens` count that becomes an addition once the reply's usage lands: `N this message + M system and wrappers = P input tokens` on the first send, and `R from earlier turns + L last reply + N this message + W wrappers = P input tokens` after that. The terms sum to the billed input; the remainder is what the tokenizer cannot name. When tools first ride along, that remainder's label names the schema. Send **strawberry** and watch it arrive as three pieces, not ten letters.
+- **Three full replies, and where they fork.** Each turn requests `n=3` completions. A strip reading “3 replies were written. You are reading reply” fronts tabs **1 / 2 / 3**, each with a confidence dot, and an unlocked strip carries a `?` that repeats the coach-mark sentence; older turns lock after the next user message, and a padlock explains why. A ring marks the first token where the three replies diverge — everything before it is identical, because the same prompt and the same weights produced the same tokens until the dice landed differently. Each tab also reports its length and perplexity (“14 tokens · picking from ~N plausible words”), and a muted `N tokens` sits after the last word of the reply you are reading.
 - **Conversation cost, in tokens first.** The API has no memory. Every turn resends the whole prompt, so input tokens climb as a staircase — one stacked bar per request, split into replayed, cached, and new. A card at the foot of the transcript, **What each request carried**, leads with the two numbers that actually teach the lesson: *Sent this turn* (`143 in · 13 out`, or `270 + 341 in · 49 out · 2 requests` on a tool turn) and *Conversation so far* in tokens. Dollars are the footnote, not the headline — open **Details** for the staircase, the running spend, the literal JSON array that was sent, and a **How is this priced?** disclosure holding the rate card (for the default model, $0.15 / 1M in, $0.60 / 1M out, $0.075 / 1M cached in) and this turn's input / cached-input / output split. Each message also carries its own usage line, now tokens only (`N in · M out`), with the dollar figure moved into its expandable breakdown as `— this turn at list price`.
 - **Fractions of a cent, spelled out.** A turn on `gpt-4o-mini` costs far less than a cent, and `$0.00` teaches nothing. `formatUsd` in `lib/openaiRates.js` prints `$0.02` at or above a cent, `$0.01` for anything from about two-thirds of a cent up (that *is* “about a cent”), `≈ 1/167 of a cent` below that, and `less than 1/10,000 of a cent` at the floor. Beside the conversation total, `formatScale` multiplies the last turn by a million to give the number a size a person can hold: `a million chats like this ≈ $60.00`.
 - **No memory, made visible.** The model has no memory of its own; the app replays the transcript every request. Turn on **Forget older turns** and the request stops carrying the top of the chat — a line appears in the transcript, the turns above it dim, and the model can no longer answer a question about a fact you seeded before the line. The transcript and your saved conversation keep everything; only the request shrinks. The system prompt never falls off, because the server adds it every time. The empty screen offers the demo as a path rather than a puzzle: a **Give it a fact to remember** chip seeds “My name is Ada. Remember it.”, and once that reply settles a **Now make it forget** chip appears, flips **Exchanges replayed** to `0`, and asks “What is my name?” in one click. The weather question has the same shape: after any reply to a prompt that mentions weather, with the tool off, a **Now give it the tool** chip turns the tool on and re-sends the same question, so the two replies sit side by side.

components/ChatInterface.js

@@ -17,7 +17,7 @@ import { pruneForStorage } from '../lib/persistence';
 import { abortedFields, abortedTurn } from '../lib/abortedTurn';
 import { buildOutboundMessages, KEEP_ALL, KEEP_TURNS_DEFAULT } from '../lib/contextWindow';
 import { knowledgeCutoff } from '../lib/modelFacts';
-import { formatTokenSummary } from '../lib/usage';
+import { formatTokenSummary, offeredTools, selectedReplyTokens } from '../lib/usage';
 import { needsCutoffNote, mentionsWeather } from '../lib/cutoffRelevance';
 import { COACH_TEXT_COLOR, COACH_TEXT_TABS, COACH_TEXT_COST } from '../lib/coachCopy';
 
@@ -933,6 +933,12 @@ export default function ChatInterface() {
             const cutoffPrompt = [...messages.slice(0, index)]
               .reverse()
               .find((item) => item.role === 'user') ?? null;
+            const followingAssistant = message.role === 'user'
+              ? messages.slice(index + 1).find((item) => item.role === 'assistant')
+              : null;
+            const priorAssistant = message.role === 'user'
+              ? [...messages.slice(0, index)].reverse().find((item) => item.role === 'assistant' && item.usage?.prompt_tokens != null)
+              : null;
             const node = (
             <Message
               key={index}
@@ -944,6 +950,11 @@ export default function ChatInterface() {
               sessionBilled={message.role === 'assistant' ? billedThrough : null}
               replayedIn={message.role === 'assistant' ? replayedIn : null}
               addedIn={message.role === 'assistant' ? addedIn : null}
+              promptIn={roundPrompt(followingAssistant, 'first')}
+              toolsOffered={offeredTools(followingAssistant)}
+              replayedPromptIn={roundPrompt(priorAssistant, 'first')}
+              lastReplyTokens={selectedReplyTokens(priorAssistant)}
+              previousToolsOffered={offeredTools(priorAssistant)}
               tabsLocked={messages.slice(index + 1).some((item) => item.role === 'user')}
               tokenizer={tokenizer}
               forgotten={forgetting.truncated && index < forgetting.cutoffIndex}

components/Message.js

@@ -4,7 +4,7 @@
 import { tokenizeForDisplay, isPartialChunk } from '../lib/tokenizer';
 import { sampledLogprob, findForkIndex, completionStats, formatPerplexity, confidenceColor, confidenceParts, confidenceBand } from '../lib/completionStats';
 import { rateFor, turnCost, formatUsd } from '../lib/openaiRates';
-import { formatTokenSummary } from '../lib/usage';
+import { formatTokenSummary, formatUserTokenLine, offeredTools } from '../lib/usage';
 import { knowledgeCutoff } from '../lib/modelFacts';
 import { mentionsWeather } from '../lib/cutoffRelevance';
 import { COACH_TEXT_TABS } from '../lib/coachCopy';
@@ -61,7 +61,7 @@
   return { '--conf-rgb': parts.rgb, '--conf-rgb-dark': parts.rgbDark, '--conf-a': parts.alpha, '--conf-on': 1 };
 };
 
-function Message({ message, onSelect, messageIndex, coach = null, onCoachAdvance, sessionBilled, replayedIn, addedIn, tabsLocked = false, tokenizer, forgotten = false, showCutoffDetail = false, cutoffPrompt = null }) {
+function Message({ message, onSelect, messageIndex, coach = null, onCoachAdvance, sessionBilled, replayedIn, addedIn, tabsLocked = false, tokenizer, forgotten = false, showCutoffDetail = false, cutoffPrompt = null, promptIn = null, toolsOffered = false, replayedPromptIn = null, lastReplyTokens = null, previousToolsOffered = false }) {
   const { role, completions, activeIndex = 0, content } = message;
   const isStreaming = !!message.isStreaming;
   const [hoveredToken, setHoveredToken] = useState(null);
@@ -151,7 +151,7 @@
     };
     document.addEventListener('pointerdown', onDocPointerDown);
     return () => document.removeEventListener('pointerdown', onDocPointerDown);
   }, [hoveredToken]);
 
   useEffect(() => {
     if (!hoveredToken) return undefined;
@@ -162,7 +162,7 @@
     };
     document.addEventListener('keydown', onKeyDown);
     return () => document.removeEventListener('keydown', onKeyDown);
   }, [hoveredToken]);
 
   const handleTokenMouseEnter = (token, index, event) => {
     if (pinned) return;
@@ -323,6 +323,9 @@
             </Fragment>
           );
         })}
+        {tokenProbabilities.length > 0 && (
+          <span className="reply-token-count">{tokenProbabilities.length} tokens</span>
+        )}
       </div>
     );
   };
@@ -348,6 +351,7 @@
                     const stats = tabStats[index];
                     const parts = [
                       `Response ${index + 1}`,
+                      stats?.tokenCount != null && `${stats.tokenCount} tokens`,
                       stats && formatPerplexity(stats.perplexity),
                     ].filter(Boolean);
                     return (
@@ -514,14 +518,26 @@
                   </button>
                 </span>
               )}
-              {role === 'user' && userChunks && (
-                <span
-                  className="token-usage"
-                  title="Counted with the same tokenizer the model family uses. The reply's 'new' number runs a little higher — the chat wrapper rides along with every turn."
-                >
-                  ≈ {userChunks.length} tokens
-                </span>
-              )}
+              {role === 'user' && userChunks && (() => {
+                const line = formatUserTokenLine({
+                  messageTokens: userChunks.length,
+                  promptTokens: promptIn,
+                  replayedTokens: replayedPromptIn,
+                  lastReplyTokens,
+                  toolsOffered,
+                  previousToolsOffered,
+                });
+                const title = !Number.isFinite(promptIn)
+                  ? 'Counted with the same tokenizer the model family uses. The reply\'s \'new\' number runs a little higher — the chat wrapper rides along with every turn.'
+                  : toolsOffered
+                    ? 'Each term is billed tokens except this message and last reply, which are the tokenizer cut of those bodies. The remainder is wrappers — and the tool schema, when it first rides along. They add up to the input tokens the API billed.'
+                    : 'Each term is billed tokens except this message and last reply, which are the tokenizer cut of those bodies. The remainder is the system prompt and the chat wrapper. They add up to the input tokens the API billed.';
+                return (
+                  <span className="token-usage" title={title}>
+                    {line}
+                  </span>
+                );
+              })()}
               {message.usage?.prompt_tokens != null && (() => {
                 const summary = formatTokenSummary(message.usage);
                 return (
@@ -545,6 +561,9 @@
           {usageOpen && message.usage?.prompt_tokens != null && (
             <div id={usageId} className="token-usage-details">
               <span>{rounds ? `${message.usage.prompt_tokens} in — everything sent this turn, across two requests` : `${message.usage.prompt_tokens} in — everything sent this request`}</span>
+              {offeredTools(message) && (
+                <span>the weather tool schema rode with this request — counted in the prompt</span>
+              )}
               {rounds && rounds.length > 1 && rounds.map((r, i) => (
                 <span key={i}>
                   {r.prompt_tokens} in · {r.completion_tokens} out — {i === 0 ? 'first request, the one that ended in a tool call' : 'next request, the same prompt plus the tool call and its result'}