diff --git a/README.md b/README.md index 9756f3ad0..95abaf034 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,9 @@ These four packages release together under one aligned version. Packages named `@pretable-internal/*` are repo-local implementation details. Do not build application code against them. +See the [roadmap](./ROADMAP.md) for current priorities and the financial-grade +product direction. + ## Install ```bash @@ -125,6 +128,7 @@ streaming hypotheses remain directional rather than fully satisfied. ## Repository Layout ```text +ROADMAP.md Current product priorities and sequencing apps/bench Benchmark lab and browser test target apps/website Documentation and marketing site packages/core Public framework-agnostic grid primitives diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 000000000..a05d53345 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,93 @@ +# Pretable roadmap + +Last reviewed: 2026-08-10 + +Pretable is a batteries-included React data grid built to be the fastest and +best grid for demanding, financial-grade applications. The roadmap uses two +reference workloads: live portfolio-management systems and collaborative +financial planning. + +## Governing principles + +- Performance and correctness are product features: every major capability gets + an explicit complexity/memory budget, deterministic workload, and committed + evidence. +- Pretable ships working grid behavior and UI, not interfaces alone. +- One canonical derived model governs rendering, focus, selection, editing, + clipboard, accessibility, and virtualization. +- The grid remains finance-capable but domain-agnostic; applications own + valuation, formulas, permissions, storage endpoints, and retention policy. +- Public state and protocols are versioned and migration-aware. + +## Current baseline + +The aligned public-package baseline is `0.0.14`. The released grid already +includes local sorting and typed filtering, selection, keyboard navigation, +copy and validated bulk paste, typed asynchronous editing, column virtualization +and layout, configurable row grouping and aggregation with an overflow-safe +group panel, number-column alignment with tabular number/date figures, +variable-height rows, and batched streaming transactions. Package changelogs +and generated API reports are the source of truth for shipped behavior. + +See the current [core changelog](packages/core/CHANGELOG.md), +[React changelog](packages/react/CHANGELOG.md), +[core API report](packages/core/core.api.md), +[React API report](packages/react/react.api.md), and +[committed comparative benchmark evidence](status/milestones/2026-05-12-comparator-aware-evaluators.hypotheses.json). + +## Now — harden and measure + +- Close remaining public API/documentation gaps and make stable row identity + consistent across entry points. +- Finish grouping adoption with committed benchmark evidence. +- Add deterministic PMS and financial-planning benchmark profiles without + weakening existing 60 Hz, zero-gap, and interaction-continuity gates. + +## Next — describe and manipulate financial views + +1. Add a financial field schema, nested column bands, field chooser, formatting + descriptors, and view-management surface. +2. Ship saved-view persistence: a versioned portable view document, migrations, + schema reconciliation, browser-local storage, remote store adapter, and + shared/personal view layers. +3. Add a typed command foundation with atomic edit/paste batches, inverses, and + bounded local undo/redo. Saved views precede command history. + +## Later — remote scale, analytics, and collaboration + +1. Add a remote/live row model with bounded caching, typed query plans, + cancellation, partial/stale/error state, and ordered resynchronization. +2. Add revisioned mutations with command IDs, base revisions, optimistic and + pessimistic execution, conflicts, rollback, retries, and resync. +3. Develop parallel application tracks: + - PMS analytics: incremental grouping/aggregation, totals, pivot, + drill-through, context actions, and export. + - Financial planning: range editing, fill, row creation/reordering, formula + and provenance surfaces, and time/scenario comparison columns. +4. Add provider-backed durable audit/version history, history UI, + revert-as-new-command, redaction, retention hooks, and collaborative updates. +5. Converge both tracks with shared/personal configuration, capability-aware UI, + tree data, localization, and accessibility hardening. + +## Not planned + +- A hosted Pretable backend or authentication system. +- Finance-domain valuation, formula, scenario, or permission policy. +- A chart, page-layout, or financial-application platform. +- Persistence of complete runtime grid snapshots. + +## Persistence and history sequence + +Saved views → command foundation and local undo/redo → revisioned mutations → +durable audit/version history. + +See the approved [persistence and history design](docs/superpowers/specs/2026-08-09-persistence-and-history-design.md) +and its [documentation implementation plan](docs/superpowers/plans/2026-08-09-persistence-and-history-documentation.md). + +## How roadmap items ship + +An item moves into implementation only after an approved focused design and +implementation plan. It is complete only when public API reports, consumer +documentation, correctness tests, and relevant benchmark evidence agree with the +released behavior. The roadmap tracks outcomes; changelogs track shipped work, +and dated plans remain historical execution recipes. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 000000000..af5300939 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,59 @@ +# Pretable documentation + +This directory preserves product decisions, execution recipes, research, and +handoffs. It is an archive with explicit authority rules, not the current +roadmap. + +## Authority + +1. Package changelogs and generated API reports describe shipped behavior. +2. [ROADMAP.md](../ROADMAP.md) describes current prioritization. +3. docs/superpowers/specs/ contains approved design and decision records. +4. docs/superpowers/plans/ contains execution recipes, not live status. +5. status/milestones/ contains committed performance evidence. +6. docs/research/ and handoffs are historical unless a current document links + to them explicitly. + +When documents disagree, use the highest applicable source above. Public +consumer documentation must describe released behavior and should not advertise +speculative roadmap APIs. + +## Lifecycle + +New design specs use `docs/superpowers/specs/YYYY-MM-DD--design.md`. +Implementation plans use `docs/superpowers/plans/YYYY-MM-DD-.md`. + +Every new spec should include: + +- Date +- Status: draft, approved, planned, in-progress, shipped, or superseded +- Supersedes / Superseded by when applicable +- implementation plan, PR/commit, and released version when those exist + +Status meanings: + +- draft: under discussion +- approved: design accepted; no implementation plan is implied +- planned: an implementation plan exists +- in-progress: implementation is active +- shipped: released behavior exists; changelogs/API reports remain authoritative +- superseded: retained for history and linked to its replacement + +Unchecked boxes in an old plan do not prove work remains. Confirm shipped state +from changelogs, API reports, and implementation history. + +## Repository map + +- superpowers/specs/: dated designs and decision records +- superpowers/plans/: dated implementation recipes +- research/: diagnostics, closeouts, and historical memory +- handoffs/ and superpowers/handoffs/: point-in-time transfer notes +- ../status/milestones/: committed benchmark evidence + +## Maintenance + +- Keep ROADMAP.md short and outcome-oriented. +- Mark superseded documents; do not delete decision history. +- Link corrections to the evidence they replace. +- Update public documentation, API reports, and changelogs with shipped APIs. +- Review roadmap claims against committed evidence before publishing them. diff --git a/docs/research/repo-memory.md b/docs/research/repo-memory.md index 1658ab21d..856ce4bb8 100644 --- a/docs/research/repo-memory.md +++ b/docs/research/repo-memory.md @@ -1,5 +1,9 @@ # Repo Memory +> **Status: Historical through 2026-05-15.** This chronological record is +> preserved as project history and is not a current backlog. See the canonical +> [ROADMAP.md](../../ROADMAP.md) for current priorities. + ## 2026-04-12 ### Prototype direction diff --git a/docs/superpowers/plans/2026-08-09-persistence-and-history-documentation.md b/docs/superpowers/plans/2026-08-09-persistence-and-history-documentation.md new file mode 100644 index 000000000..23cdb326f --- /dev/null +++ b/docs/superpowers/plans/2026-08-09-persistence-and-history-documentation.md @@ -0,0 +1,435 @@ +# Persistence and History Documentation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking. + +**Goal:** Establish one concise financial-grade product roadmap, document repository planning authority, and place saved views, undo/redo, revisioned mutations, and durable audit history in an explicit delivery sequence. + +**Architecture:** ROADMAP.md becomes the only current prioritization document; docs/README.md explains how that roadmap relates to specs, plans, shipped API truth, research, and benchmark evidence. Historical roadmap/memory files remain intact behind additive banners, while the root README links the canonical roadmap. This change is documentation-only and does not add or advertise persistence/history APIs. + +**Tech Stack:** Markdown, Prettier, Git path-existence checks + +--- + +## Scope and file responsibilities + +**Read for authority:** + +- docs/superpowers/specs/2026-08-09-persistence-and-history-design.md — approved architecture and acceptance criteria. +- README.md — product thesis, package status, evidence, and repository map. +- GOVERNANCE.md — maintainer-led, evidence-backed roadmap policy. +- Package CHANGELOG.md and generated API reports — shipped truth. + +**Create:** + +- ROADMAP.md — current product priorities and sequencing only. +- docs/README.md — documentation authority, lifecycle, and metadata conventions. + +**Modify:** + +- README.md — link the canonical roadmap and name it in the repository map. +- docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md — additive historical banner only. +- docs/research/repo-memory.md — additive historical banner only. + +**Do not modify:** + +- Consumer MDX, packages, dependencies, API reports, changelogs, or benchmark artifacts. +- Original historical content below the two new banners. +- Product APIs or implementation. + +### Task 1: Create the canonical financial-grade roadmap + +**Files:** + +- Create: ROADMAP.md +- Reference: docs/superpowers/specs/2026-08-09-persistence-and-history-design.md +- Reference: packages/core/CHANGELOG.md +- Reference: packages/react/CHANGELOG.md +- Reference: status/milestones/2026-05-12-comparator-aware-evaluators.hypotheses.json + +- [ ] **Step 1: Confirm the roadmap does not already exist** + +Run: + +```bash +test ! -e ROADMAP.md +``` + +Expected: exit 0 with no output. + +- [ ] **Step 2: Create ROADMAP.md** + +Write a concise document with this exact structure and content: + +```markdown +# Pretable roadmap + +Last reviewed: 2026-08-10 + +Pretable is a batteries-included React data grid built to be the fastest and +best grid for demanding, financial-grade applications. The roadmap uses two +reference workloads: live portfolio-management systems and collaborative +financial planning. + +## Governing principles + +- Performance and correctness are product features: every major capability gets + an explicit complexity/memory budget, deterministic workload, and committed + evidence. +- Pretable ships working grid behavior and UI, not interfaces alone. +- One canonical derived model governs rendering, focus, selection, editing, + clipboard, accessibility, and virtualization. +- The grid remains finance-capable but domain-agnostic; applications own + valuation, formulas, permissions, storage endpoints, and retention policy. +- Public state and protocols are versioned and migration-aware. + +## Current baseline + +The aligned public-package baseline is `0.0.14`. The released grid already +includes local sorting and typed filtering, selection, keyboard navigation, +copy and validated bulk paste, typed asynchronous editing, column virtualization +and layout, configurable row grouping and aggregation with an overflow-safe +group panel, number-column alignment with tabular number/date figures, +variable-height rows, and batched streaming transactions. Package changelogs +and generated API reports are the source of truth for shipped behavior. + +See the current [core changelog](packages/core/CHANGELOG.md), +[React changelog](packages/react/CHANGELOG.md), +[core API report](packages/core/core.api.md), +[React API report](packages/react/react.api.md), and +[committed comparative benchmark evidence](status/milestones/2026-05-12-comparator-aware-evaluators.hypotheses.json). + +## Now — harden and measure + +- Close remaining public API/documentation gaps and make stable row identity + consistent across entry points. +- Finish grouping adoption with committed benchmark evidence. +- Add deterministic PMS and financial-planning benchmark profiles without + weakening existing 60 Hz, zero-gap, and interaction-continuity gates. + +## Next — describe and manipulate financial views + +1. Add a financial field schema, nested column bands, field chooser, formatting + descriptors, and view-management surface. +2. Ship saved-view persistence: a versioned portable view document, migrations, + schema reconciliation, browser-local storage, remote store adapter, and + shared/personal view layers. +3. Add a typed command foundation with atomic edit/paste batches, inverses, and + bounded local undo/redo. Saved views precede command history. + +## Later — remote scale, analytics, and collaboration + +1. Add a remote/live row model with bounded caching, typed query plans, + cancellation, partial/stale/error state, and ordered resynchronization. +2. Add revisioned mutations with command IDs, base revisions, optimistic and + pessimistic execution, conflicts, rollback, retries, and resync. +3. Develop parallel application tracks: + - PMS analytics: incremental grouping/aggregation, totals, pivot, + drill-through, context actions, and export. + - Financial planning: range editing, fill, row creation/reordering, formula + and provenance surfaces, and time/scenario comparison columns. +4. Add provider-backed durable audit/version history, history UI, + revert-as-new-command, redaction, retention hooks, and collaborative updates. +5. Converge both tracks with shared/personal configuration, capability-aware UI, + tree data, localization, and accessibility hardening. + +## Not planned + +- A hosted Pretable backend or authentication system. +- Finance-domain valuation, formula, scenario, or permission policy. +- A chart, page-layout, or financial-application platform. +- Persistence of complete runtime grid snapshots. + +## Persistence and history sequence + +Saved views → command foundation and local undo/redo → revisioned mutations → +durable audit/version history. + +See the approved [persistence and history design](docs/superpowers/specs/2026-08-09-persistence-and-history-design.md) +and its [documentation implementation plan](docs/superpowers/plans/2026-08-09-persistence-and-history-documentation.md). + +## How roadmap items ship + +An item moves into implementation only after an approved focused design and +implementation plan. It is complete only when public API reports, consumer +documentation, correctness tests, and relevant benchmark evidence agree with the +released behavior. The roadmap tracks outcomes; changelogs track shipped work, +and dated plans remain historical execution recipes. +``` + +Do not add delivery dates, promises, owners, checkboxes, speculative public type names, or links to sibling repositories. + +- [ ] **Step 3: Verify roadmap scope and sequencing** + +Run: + +```bash +rg -n "^## (Now|Next|Later|Not planned)|Saved views.*command foundation" ROADMAP.md +rg -n "revisioned mutations|durable audit/version history" ROADMAP.md +rg -n "hosted Pretable backend|complete runtime grid snapshots" ROADMAP.md +``` + +Expected: all four tiers, the four-step sequence, and the explicit non-goals appear. + +- [ ] **Step 4: Format and validate the roadmap** + +Run: + +```bash +pnpm exec prettier --write ROADMAP.md +pnpm exec prettier --check ROADMAP.md +git diff --check -- ROADMAP.md +test -e docs/superpowers/specs/2026-08-09-persistence-and-history-design.md +``` + +Expected: Prettier passes, Git is silent, and the design target exists. + +- [ ] **Step 5: Commit the roadmap** + +Run: + +```bash +git add ROADMAP.md +git commit -m "docs: add financial-grade roadmap" +``` + +Expected: one commit adding only ROADMAP.md. + +### Task 2: Define documentation authority and preserve historical context + +**Files:** + +- Create: docs/README.md +- Modify: docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md:1 +- Modify: docs/research/repo-memory.md:1 + +- [ ] **Step 1: Create docs/README.md** + +Write: + +```markdown +# Pretable documentation + +This directory preserves product decisions, execution recipes, research, and +handoffs. It is an archive with explicit authority rules, not the current +roadmap. + +## Authority + +1. Package changelogs and generated API reports describe shipped behavior. +2. [ROADMAP.md](../ROADMAP.md) describes current prioritization. +3. docs/superpowers/specs/ contains approved design and decision records. +4. docs/superpowers/plans/ contains execution recipes, not live status. +5. status/milestones/ contains committed performance evidence. +6. docs/research/ and handoffs are historical unless a current document links + to them explicitly. + +When documents disagree, use the highest applicable source above. Public +consumer documentation must describe released behavior and should not advertise +speculative roadmap APIs. + +## Lifecycle + +New design specs use docs/superpowers/specs/YYYY-MM-DD--design.md. +Implementation plans use docs/superpowers/plans/YYYY-MM-DD-.md. + +Every new spec should include: + +- Date +- Status: draft, approved, planned, in-progress, shipped, or superseded +- Supersedes / Superseded by when applicable +- implementation plan, PR/commit, and released version when those exist + +Status meanings: + +- draft: under discussion +- approved: design accepted; no implementation plan is implied +- planned: an implementation plan exists +- in-progress: implementation is active +- shipped: released behavior exists; changelogs/API reports remain authoritative +- superseded: retained for history and linked to its replacement + +Unchecked boxes in an old plan do not prove work remains. Confirm shipped state +from changelogs, API reports, and implementation history. + +## Repository map + +- superpowers/specs/: dated designs and decision records +- superpowers/plans/: dated implementation recipes +- research/: diagnostics, closeouts, and historical memory +- handoffs/ and superpowers/handoffs/: point-in-time transfer notes +- ../status/milestones/: committed benchmark evidence + +## Maintenance + +- Keep ROADMAP.md short and outcome-oriented. +- Mark superseded documents; do not delete decision history. +- Link corrections to the evidence they replace. +- Update public documentation, API reports, and changelogs with shipped APIs. +- Review roadmap claims against committed evidence before publishing them. +``` + +- [ ] **Step 2: Add the April-roadmap historical banner** + +Insert immediately after its title: + +```markdown +> **Status: Historical.** This point-in-time roadmap is preserved for decision +> history; its projects are no longer the current backlog. See the canonical +> [ROADMAP.md](../../../ROADMAP.md) for current priorities. +``` + +Do not modify original content below the banner. + +- [ ] **Step 3: Add the repo-memory historical banner** + +Insert immediately after its title: + +```markdown +> **Status: Historical through 2026-05-15.** This chronological record is +> preserved as project history and is not a current backlog. See the canonical +> [ROADMAP.md](../../ROADMAP.md) for current priorities. +``` + +Do not modify original content below the banner. + +- [ ] **Step 4: Verify the banners are additive** + +Run: + +```bash +git diff --unified=3 -- docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md docs/research/repo-memory.md +test -e ROADMAP.md +``` + +Expected: only one banner is added to each historical file and the roadmap exists. + +- [ ] **Step 5: Format and validate** + +Run: + +```bash +pnpm exec prettier --write docs/README.md +pnpm exec prettier --check docs/README.md +git diff --check +``` + +Expected: checks pass. Do not run Prettier with --write on the historical files, because original archival content must remain byte-for-byte unchanged below each banner. + +- [ ] **Step 6: Commit the index and notices** + +Run: + +```bash +git add docs/README.md docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md docs/research/repo-memory.md +git commit -m "docs: define planning document authority" +``` + +Expected: one commit containing the index and two additive banners. + +### Task 3: Link the roadmap and verify the complete documentation slice + +**Files:** + +- Modify: README.md:16-29 +- Modify: README.md:124-139 +- Verify: all approved documentation paths + +- [ ] **Step 1: Link the roadmap from README status** + +After the public/internal package-boundary paragraph, add: + +```markdown +See the [roadmap](./ROADMAP.md) for current priorities and the financial-grade +product direction. +``` + +- [ ] **Step 2: Add the roadmap to the repository map** + +Add this as the first line inside the Repository Layout code block: + +```text +ROADMAP.md Current product priorities and sequencing +``` + +Do not fix unrelated README drift in this PR. + +- [ ] **Step 3: Verify required files and links exist** + +Run: + +```bash +test -e ROADMAP.md +test -e docs/README.md +test -e docs/superpowers/specs/2026-08-09-persistence-and-history-design.md +test -e docs/superpowers/plans/2026-08-09-persistence-and-history-documentation.md +test -e docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md +test -e docs/research/repo-memory.md +test -e packages/core/CHANGELOG.md +test -e packages/react/CHANGELOG.md +test -e packages/core/core.api.md +test -e packages/react/react.api.md +test -e status/milestones/2026-05-12-comparator-aware-evaluators.hypotheses.json +``` + +Expected: every command exits 0. + +- [ ] **Step 4: Verify the documentation-only boundary** + +Run: + +```bash +git diff --name-only origin/main...HEAD +git status --short +``` + +Expected paths are limited to: + +```text +README.md +ROADMAP.md +docs/README.md +docs/research/repo-memory.md +docs/superpowers/plans/2026-08-09-persistence-and-history-documentation.md +docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md +docs/superpowers/specs/2026-08-09-persistence-and-history-design.md +``` + +- [ ] **Step 5: Run final validation** + +Run: + +```bash +pnpm exec prettier --check README.md ROADMAP.md docs/README.md docs/research/repo-memory.md docs/superpowers/plans/2026-08-09-persistence-and-history-documentation.md docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md docs/superpowers/specs/2026-08-09-persistence-and-history-design.md +git diff --check origin/main...HEAD +git diff --check +pnpm test +``` + +Expected: formatting passes for all seven approved Markdown paths; both committed +and working-tree diffs have no whitespace errors; tests pass. If the known S7 +five-second timeout recurs only under concurrent package execution, rerun pnpm +--filter @pretable-internal/scenario-data test to record whether the isolated +suite passes, but do not alter product code in this PR. + +- [ ] **Step 6: Commit the README link** + +Run: + +```bash +git add README.md +git commit -m "docs: link the canonical roadmap" +``` + +Expected: one focused README commit. + +- [ ] **Step 7: Inspect the branch** + +Run: + +```bash +git log --oneline origin/main..HEAD +git diff --stat origin/main...HEAD +git status --short --branch +``` + +Expected: documentation-only commits, the seven approved paths, and a clean working tree. diff --git a/docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md b/docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md index 3571dca15..21da7bd11 100644 --- a/docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md +++ b/docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md @@ -1,5 +1,9 @@ # Roadmap: Fix-Then-Expand +> **Status: Historical.** This point-in-time roadmap is preserved for decision +> history; its projects are no longer the current backlog. See the canonical +> [ROADMAP.md](../../../ROADMAP.md) for current priorities. + ## Goal Prioritized backlog of the next five projects for pretable, sequenced to prove interactions before expanding scope. diff --git a/docs/superpowers/specs/2026-08-09-persistence-and-history-design.md b/docs/superpowers/specs/2026-08-09-persistence-and-history-design.md new file mode 100644 index 000000000..280d23612 --- /dev/null +++ b/docs/superpowers/specs/2026-08-09-persistence-and-history-design.md @@ -0,0 +1,479 @@ +# Persistence and history — design + +**Date:** 2026-08-09 + +**Status:** planned + +**Implementation plan:** +[`2026-08-09-persistence-and-history-documentation.md`](../plans/2026-08-09-persistence-and-history-documentation.md) + +## Context + +Pretable is becoming a batteries-included, financial-grade grid. Two reference +workloads guide that direction: + +- live portfolio-management grids with dense data, high-frequency updates, + grouping, aggregation, pivoting, drill-through, and saved analytical views; +- collaborative financial-planning grids with remote data, large edits and + pastes, validation, conflicts, undo/redo, comparisons, and durable history. + +Pretable already has useful in-memory continuity primitives. The React surface +can control sort, filters, selection, focus, grouping, column widths, column +order, and pinning. `setRows` reconciles focus, selection, and editing by stable +row id. Editing and paste emit application-owned callbacks, and transactions +batch local add/update/remove operations. + +Those primitives are not a persistence or history contract: + +- `PretableSurfaceState` is experimental and incomplete as a saved-view format. +- `PretableGridSnapshot` mixes durable intent with derived and transient state. +- no public codec, migration, view store, undo/redo controller, revision model, + audit provider, or resynchronization protocol exists; +- edits and paste explicitly defer undo, while applications currently own all + durable mutation behavior. + +Before this documentation slice, the repository lacked a current roadmap. Dated specs and plans preserve +valuable history, but the only explicit roadmap is an April snapshot whose +projects are already complete. `docs/research/repo-memory.md` stops in May even +though implementation and design work continued through August. + +This design establishes both the product architecture and the documentation +structure needed to plan persistence and history without promising unshipped +public APIs. + +## Goals + +1. Establish one canonical, concise roadmap for a financial-grade Pretable. +2. Define documentation authority and lifecycle so shipped truth, plans, + decisions, and evidence cannot be confused. +3. Separate saved-view persistence, command undo/redo, and durable audit history + while giving them a compatible revision and command vocabulary. +4. Keep Pretable batteries-included: ship working state machines, stores, + components, and defaults rather than interfaces alone. +5. Preserve application ownership of finance-domain logic, authentication, + canonical remote storage, and retention policy. +6. Make every future implementation slice independently shippable and + benchmarkable. + +## Non-goals + +- Implementing persistence or history in this documentation slice. +- Freezing exact public TypeScript names before focused child designs. +- Persisting complete runtime snapshots. +- Building a hosted Pretable backend, authentication system, or finance model. +- Defining formulas, scenario semantics, valuation logic, or application ACLs. +- Publishing speculative APIs in the consumer documentation site. +- Retrofitting metadata onto every historical spec and plan in one change. + +## Documentation architecture + +### Canonical roadmap + +Create `ROADMAP.md` at the repository root and link it from the README status +section. It is the only current prioritization document. + +The roadmap stays concise and contains: + +- product thesis and governing performance principles; +- current baseline and last-reviewed date; +- `Now`, `Next`, `Later`, and `Not planned` outcomes; +- links to approved designs, implementation plans, releases, and benchmark + evidence; +- explicit placement of saved views, command history, and durable history in + the broader financial-grade roadmap. + +The roadmap tracks outcomes, not task checkboxes. Shipped work links to package +changelogs and API reports rather than relying on unchecked boxes in historical +plans. + +The initial tiers are fixed for this documentation slice: + +- **Now — harden and measure:** close remaining public API/documentation gaps, + make stable row identity consistent across entry points, finish grouping + adoption with committed benchmark evidence, and add PMS and + financial-planning benchmark profiles. +- **Next — describe and manipulate financial views:** add the financial field + schema, nested headers, view-management surface, saved-view persistence + (persistence slice 1), and the typed command foundation with bounded local + undo/redo (history slice 2). Saved views precede the command foundation within + this tier. +- **Later — remote scale, analytics, and collaboration:** add the remote/live row + model, revisioned mutations (slice 3), parallel PMS analytics and planning- + interaction tracks, durable audit history (slice 4), and their production + convergence. Slice 3 precedes slice 4. +- **Not planned:** a hosted Pretable backend, authentication system, finance- + domain model, chart/page platform, or persistence of complete runtime + snapshots. + +### Documentation index + +Create `docs/README.md` to define authority and lifecycle: + +1. package changelogs and generated API reports are shipped truth; +2. `ROADMAP.md` is current prioritization; +3. dated design specs capture approved decisions; +4. dated implementation plans are execution recipes, not status trackers; +5. `status/milestones` contains committed performance evidence; +6. research and handoffs are historical context unless a current document links + to them explicitly. + +New specs should carry date, status, supersession links when applicable, and +implementation references once shipped. Supported statuses are `draft`, +`approved`, `planned`, `in-progress`, `shipped`, and `superseded`. + +### Historical documents + +Preserve +`docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md` and +`docs/research/repo-memory.md`, but add an additive banner at the top of each +file marking it historical and pointing to `ROADMAP.md`. Do not rewrite their +original content or use them as live backlogs. + +### Consumer documentation + +Public MDX pages describe released behavior only. Each implementation slice +updates its consumer guides when the corresponding API ships. Until then, the +current editing, paste, controlled-state, and streaming limitations remain +explicit. + +## Product model + +Persistence and history are three related systems with distinct guarantees. + +| System | Purpose | Default lifetime | +| --------------------- | ----------------------------------------------------- | ------------------------- | +| View persistence | Restore query and presentation intent | Across sessions | +| Command history | Reverse recent local grid operations | Bounded current session | +| Audit/version history | Explain accepted revisions across actors and sessions | Durable, provider-defined | + +All three systems can use versioned records and structured results. Command +history and audit/version history may additionally share command IDs, data +revisions, and typed mutation operations; view-store revisions remain separate +from data revisions. They do not share retention, reversibility, or +authorization guarantees. + +## Saved-view persistence + +### Separate durable and runtime types + +`PretableGridSnapshot` remains a runtime observation model. A separate, +JSON-safe view document becomes the durable contract. The conceptual shape is: + +```ts +interface PretableViewDocument { + formatVersion: number; + schemaId?: string; + state: PretablePersistedViewState; + extensions?: Record; +} +``` + +Exact names require a child design, but the separation is fixed. + +### Durable state + +The persisted view can contain: + +- column order, width, visibility, and pinning; +- sort and typed filter expressions; +- row grouping, pivot, and aggregation choices; +- optional expansion state; +- density and display preferences; +- namespaced extension data. + +It excludes: + +- row data and remote-data caches; +- derived visible rows and aggregate results; +- focus, selection, and scroll position by default; +- edit drafts, pending requests, validation errors, and loading state; +- open menus, hover state, telemetry, and renderer measurements. + +### State layers + +Persistence applies deterministic layers: + +```text +column defaults +→ shared saved view +→ personal overlay +→ runtime controlled state +``` + +A shared view captures report intent. A personal overlay can retain widths, +expansion, density, or other preferences without mutating the shared view. +Runtime controlled state always wins for the current render. + +### Built-in behavior and adapters + +Pretable should eventually ship: + +- view encode, decode, validation, and migration; +- schema reconciliation for removed, renamed, and new fields; +- memory and browser-local stores; +- a remote `ViewStore` contract; +- a batteries-included persistence option on the top-level React grid; +- save, save-as, rename, reset, delete, and default-view UI; +- configurable debounced autosave; +- explicit shared and personal scopes. + +Store records wrap the portable document with key, revision, owner, timestamps, +and scope. Storage metadata does not belong in the portable document. + +### Compatibility and failure rules + +- Corrupt or unsupported views never prevent the grid from rendering. +- Missing columns are ignored and new columns receive defaults. +- Renames require explicit schema migration rather than label matching. +- Unknown extension namespaces survive a decode/save round trip. +- Remote saves use an expected revision and never overwrite concurrent changes + silently. +- Conflicts keep the local dirty view and offer reload, authorized overwrite, + or save-as-new. +- Storage failures leave a usable in-memory grid and emit typed diagnostics. + +## Commands and local history + +Every user mutation becomes a typed command before it changes row data. The +conceptual envelope is: + +```ts +interface GridCommand { + commandId: string; + baseRevision?: string; + operations: readonly GridOperation[]; + atomic: boolean; + metadata?: Record; +} +``` + +The conceptual envelope keeps `baseRevision` optional for local mode; +provider-backed submissions require it. + +The command controller records an inverse against the state on which the +command was based. + +### Command grouping + +- Repeated typing in one cell may coalesce into one command. +- Paste, fill, and multi-cell edits are one atomic command by default. +- Row insertion, deletion, and reordering have built-in inverses. +- Custom operations are undoable only when the application supplies an inverse. +- View changes use a separate history channel so ordinary data undo does not + unexpectedly restore a filter or column width. +- Remote changes from other actors update revision and audit state but do not + enter the local undo stack. + +The in-memory history is bounded by count and/or memory. Eviction removes the +oldest reversible entries and never affects durable audit records. + +### Undo and redo + +In local mode, undo applies the recorded inverse and moves the original command +to the redo stack. In provider-backed mode, undo submits the inverse as a new +command against the current revision. Redo is also a new command. Neither action +deletes or rewrites durable history. + +Redo clears when a new local command or an incompatible remote revision changes +the branch. + +## Mutation lifecycle + +```text +gesture +→ command + inverse +├─ local mode → local apply → bounded command history +└─ provider mode → optional optimistic apply → mutation provider + → accepted or rejected result + ├─ accepted → bounded command history + └─ accepted + audit provider configured → durable audit event +``` + +The mutation and audit providers are independent, optional branches. Local mode +does not require a revision, mutation provider, or durable audit event. A host +may also feed provider-authored history events into the audit surface without +using Pretable's mutation provider. + +Providers return a typed result: + +- accepted with the new revision; +- validation error with operation or cell context; +- forbidden; +- revision conflict with current revision and retry guidance; +- transient failure known not to have been accepted; or +- outcome-unknown failure that requires command-status lookup and, where + needed, resynchronization. + +In provider mode, `commandId` is an idempotency key. Before applying a command, +the provider atomically reserves its ID with a payload identity or digest, so +exactly one submission can execute. For a documented reservation/result +retention horizon, identical concurrent or repeated submissions observe +in-progress status or the stored terminal accepted result; a payload mismatch +for a reused ID is rejected. All retries in that horizon reuse the identical +command ID and payload. + +When a response is lost and acceptance is uncertain, command-status lookup is +primary. Resynchronizing revision may restore the client state but does not by +itself prove non-acceptance. After the retention horizon expires, retry is +prohibited unless authoritative command status or application/provider +reconciliation proves that specific command was not accepted; the uncertain +operation is never issued under a new command ID. + +Pessimistic mode applies and records only after acceptance. Optimistic mode +applies immediately and retains the inverse, but does not expose the command as +undoable until acceptance. + +Validation and authorization rejection roll back optimistic changes. Revision +conflicts roll back or resynchronize before retry. Failures known not to have +been accepted follow an explicit retry policy; outcome-unknown failures use the +command-status lookup and any needed resynchronization under the constraints +above and never disappear silently. + +Commands are atomic by default. Partial application requires an explicitly +non-atomic command and per-operation results. + +## Durable audit history + +A provider-backed history event conceptually contains: + +```ts +interface GridHistoryEvent { + revision: string; + commandId: string; + actor?: GridActor; + committedAt: string; + operations: readonly GridOperationSummary[]; + undoOf?: string; + redoOf?: string; + source: string; + metadata?: Record; +} +``` + +Pretable should provide paginated history state and a built-in history panel. +The provider controls retention, permissions, redaction, actor identity, and +whether detailed values or summaries are available. + +Each event has provider- and authorization-gated revert capability. A revert +creates a new command only when the provider supplies an executable inverse and +current authorization permits it. Summary-only and redacted events are +non-revertible by default unless the provider separately retains and authorizes +an inverse; the built-in UI disables the action and exposes the reason when an +inverse is unavailable or forbidden. Derived calculation events may appear in +the timeline, but are reversible only when their provider supplies a valid +inverse. + +## Ownership boundary + +Pretable owns: + +- versioned codecs, migrations, and schema reconciliation; +- local stores and remote-store contracts; +- command construction, grouping, inverse generation, and bounded undo/redo; +- pending, validation, rejection, conflict, and resync state machines; +- view-management, undo/redo, and history UI; +- provider contracts, diagnostics, and reusable contract tests. + +Applications own: + +- remote storage, canonical revisions, and durable audit retention; +- authentication, authorization, redaction, and actor identity; +- finance-domain validation and operations; +- inverses for custom operations; +- formulas, scenarios, valuation, and other business semantics. + +## Delivery sequence + +### Slice 1: saved views + +- stable view document; +- codec, validation, migration, and schema reconciliation; +- memory and browser-local stores; +- remote store contract; +- view-management UI. + +### Slice 2: command foundation + +- typed commands and operation batches; +- atomic edit and paste semantics; +- inverse generation; +- bounded local undo/redo. + +### Slice 3: revisioned mutations + +- mutation provider; +- command IDs and base revisions; +- optimistic and pessimistic execution; +- conflict, rollback, retry, and resynchronization. + +### Slice 4: durable history + +- audit/history provider; +- paginated and filtered history UI; +- actor metadata, redaction, and retention hooks; +- revert-as-new-command; +- collaborative revision updates. + +Saved views intentionally precede the collaborative mutation protocol. The two +systems share versioning concepts, but view restoration does not need to wait for +multi-user data history. + +Each slice receives its own focused design and implementation plan when promoted +to `Now`. + +## Verification strategy + +Future child designs must include: + +- golden fixtures for every saved-view format version; +- round-trip, validation, and migration property tests; +- compatibility tests for removed, renamed, and new columns; +- command/inverse invariants showing that an operation followed by its inverse + restores prior state; +- atomic batch, rollback, retry, conflict, and resync tests; +- reusable contract suites for view, mutation, and history providers; +- end-to-end reload, save-conflict, paste-undo, remote-revision, and history- + redaction flows; +- memory bounds for caches and history; +- performance evidence showing history capture does not add full-grid work; +- financial workload benchmarks for large undo batches and view restoration; +- API-report and changeset updates for every shipped public capability. + +The current benchmark discipline remains governing: no feature is complete +without explicit complexity and memory budgets, a deterministic workload, and +correctness checks for focus, selection, editing, clipboard, accessibility, and +streaming continuity. + +## Documentation-slice acceptance criteria + +- The documentation-only change set is limited to `ROADMAP.md`, + `docs/README.md`, `README.md`, this spec, + `docs/superpowers/plans/2026-08-09-persistence-and-history-documentation.md`, + `docs/superpowers/specs/2026-04-20-roadmap-fix-then-expand-design.md`, and + `docs/research/repo-memory.md`. +- `ROADMAP.md` is the linked canonical prioritization document. +- `docs/README.md` explains documentation authority and lifecycle. +- This design captures the approved persistence/history architecture. +- The April roadmap and `repo-memory.md` are visibly historical and point to the + current roadmap. +- The roadmap places saved views, command history, revisioned mutations, and + durable audit history in the broader financial-grade delivery sequence. +- No consumer documentation presents speculative APIs as shipped. +- `git diff --check` and Prettier pass for all changed Markdown files. +- Every new repository-relative link resolves to an existing file, verified by + explicit path-existence checks recorded in the implementation plan. + +## Risks + +1. **Snapshot leakage.** Consumers may persist runtime snapshots before the + durable format ships. Documentation must explicitly discourage this. +2. **Undo/audit conflation.** A local inverse stack cannot provide compliance or + multi-user history. The public vocabulary must preserve the distinction. +3. **Unbounded memory.** Inverses for large pastes or deletions can be expensive. + Child designs must set count and byte budgets. +4. **False portability.** Custom functions, aggregators, and renderers are not + serializable. Saved views persist stable registry keys, never closures. +5. **Revision ambiguity.** View-store revisions and data revisions may use the + same shape but remain separate domains. +6. **Premature API freeze.** This architecture fixes responsibilities and + semantics, not every final public type name.