Skip to content

feat(today): mark key clauses and sourced numbers with jev - #111473

Open
puemos wants to merge 6 commits into
posthog/today-report-stack-3-frontendfrom
posthog/today-report-stack-4-jev
Open

puemos wants to merge 6 commits into
posthog/today-report-stack-3-frontendfrom
posthog/today-report-stack-4-jev

Conversation

@puemos

@puemos puemos commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Changes

  • Behind today-report-jev, a number in the lead or impact gets a pen mark only when a signal or the agent's research states the same result.
  • Hovering a marked number shows that source sentence with the exact number underlined. The backend sends the offsets.
  • The clauses that state the problem, the cause and the fix get a mark. Hovering one shows the report sentences that explain it.
  • When a finding quotes code in several places, Jev picks the excerpt the finding describes.
  • A line under the lead warns when the newest marked figure is more than a week old.
  • Three endpoints serve the marks: POST today/reports/<id>/key_clauses, GET today/reports/<id>/figure_marks and POST today/excerpt_choice.
  • They answer 404 unless the organization approved AI data processing, has AI credits, and has the flag.
  • They answer 402 when the gateway reports no AI credits left, and 503 when Jev is unavailable or misses its 45 second deadline. The page then shows the text without marks.
  • Burst and sustained throttles cover all three endpoints, on top of the default API throttles.
  • Key clauses and figure marks read report data, so a scoped API key needs task:read as well, like the page.
  • Each Jev answer is cached per team for 30 days, so a report costs one set of calls, not one per viewer.
  • Jev requests carry up to 32 rows each, at most 4 at a time. Answers from a batch that succeeds stay cached when another batch fails.
  • Offsets count UTF-16 units, so marks land on the right characters after an emoji.
  • Marks hide again when the flag turns off, and a figure mark inside a key clause sits outside the clause mark, so hover cards never nest.
  • Sample reports never ask Jev. Impact numbers keep their value-based highlight.
  • New event: today report mark opened, with report_id and kind (impact, figure or key_clause). It shows whether people read the marks.

How a number gets a mark

  1. Code finds each number in the lead and impact, and the source sentences in signals and research notes that state the exact same value. Notes written by people are excluded. Zeros never get a mark.
  2. Jev (jevk5-fp8-0.2, pinned) answers four Choice questions in stages. Each has named options and its own threshold. A later question goes only to the numbers that passed the one before:
    • What kind of number is it: a measured result, or a date, ID, window length or limit?
    • Which source states it? This is asked in both option orders, and the two picks must agree.
    • Does that source state the same result, about the same thing, group and period?
    • Does the source sentence say what the number counts?
  3. A number gets a mark only when every answer passes. At most four marks show, with numbers about people first.

Measured on reports from PostHog's own project, read only, with every shown mark labelled blind:

Set Reports Marks Right Reports with a mark
Tuning 500 134 134 96
Holdout 500 142 142 104
Fresh, never used for tuning 500 149 149 116

Note

The thresholds hold for the pinned model only. A model change needs a fresh labelled run before it ships. A looser kind threshold looked clean on the first two sets and made four bad marks on the fresh set, so it was reverted.

Mark from a signal Mark from the agent's research
pr-figure-mark-signal pr-figure-mark-research

The figure mark screenshots use an invented report on the local stack.

Warning

The riskiest part is the clause splitter in products/today/backend/logic/key_clauses.py. It ports the browser's Intl.Segmenter sentence rules (UAX #29) to Python. A wrong split moves a mark to the wrong words.

  • Mechanical: the marked text replaces the plain TodayInlineMarkdown from the layer below. Generated types follow the serializers.

How did you test this code?

  • Ran the today backend tests, the signals page source and sections tests, the Today Jest tests, the frontend type check, repo-wide mypy and tach locally.
  • Measured the figure marks on 1,500 reports as in the table above. The evaluation asked Jev through HogQL decide(), which sends the same request as GatewayJev.
  • Not checked: GatewayJev against a live gateway. The tests use a fake System One client.

Test rationale:

  • test_figures.py: scaled ranges, units and rounding edges match only the same amount.
  • test_figure_sources.py: a zero, a number Jev does not read as a measured result, a source pick that changes with option order, and a source that does not say what it counts each give no mark. Each case also checks that Jev gets only the questions the next stage needs.
  • test_key_clauses.py: number ranges stay whole, an unsure role gets no mark, only clauses the report explains are returned, at most two, and offsets after an emoji match the browser.
  • test_code_excerpts.py: only a sure pick chooses an excerpt.
  • test_jev.py: distinct texts share one request, a cached answer costs none, and a failed batch keeps the answers of the batch that succeeded.
  • test_api.py: the endpoints answer 404 without the flag, 402 without AI credits and 503 without Jev, and the figure marks carry the date and signal of their source. Excerpt choice goes through the real GatewayJev with a fake gateway. A key with only today:read gets 403 from the page, key clauses and figure marks.
  • test_report_page_source.py: notes written by people are flagged, so they are never quoted as the agent's research.
  • todayFigureSources.test.ts, todayMarkedRuns.test.ts, todayFigures.test.ts: a backend mark lands after code and links, a mark whose text does not match the page is dropped, a repeated value gets the exact underline, and a figure never sits inside a clause mark.
  • todayQuotedCode.test.ts: excerpts from every cited file reach Jev with their file.
  • todayReportLogic.test.ts: one request per mark while a request is in flight, and no marks after the flag turns off.

Release status

  • No feature flag controls this change
  • This change is behind a feature flag and is not available to users
  • This change makes a previously flagged feature available to everyone

The marks need both today-rail-nav and today-report-jev.

Automatic notifications

  • Publish to changelog?

Docs update

None. No existing doc covers the Today page.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: PostHog Desktop (Claude Code) and Claude Code, claude-opus-5-5

  • Split out of feat(today): redesign the report page around impact, proposal, and evidence #110989, which stays open as a backup until the stack lands. With this layer on top, the stack reproduces that PR merged with master. The one difference is api.zod.schemas.ts, which master's current generator now writes.
  • The figure mark questions follow the TypeSafe Jev docs: Choice questions with named options, a reverse-order check for option bias, and a threshold per question. The requester asked for the highest precision over coverage.
  • A second review round added the batching, the 402 answer, the throttles, the staged questions, the UTF-16 offsets and the flag gating. feat(today): redesign the report page around impact, proposal, and evidence #110989 carries the same fixes.
  • Skills invoked: /stacking-prs, /improving-drf-endpoints, /writing-tests, /writing-dataclasses, /writing-ui-components, /adopting-generated-api-types, /writing-pr-descriptions.

🤖 Generated with Claude Code


Created with PostHog Desktop

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

✅ Complexity (TypeScript) — clean

Cyclomatic complexity above the limit in changed typescript files (10 for production files, 15 for test files). Warn only: worth simplifying when you next touch these functions.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

⚠️ Bundle size — 🔺 +11.0 KiB (+0.0%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 69.89 MiB · 🔺 +11.0 KiB (+0.0%)

File Size Δ vs base
posthog-app/src/scenes/project-homepage/today/TodayReportPage.js 64.0 KiB 🔺 +7.8 KiB (+13.9%)
render-query/src/render-query/render-query.js 18.77 MiB 🔺 +3.2 KiB (+0.0%)

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1.65 MiB · 23 files 🔺 +678 B (+0.0%) █████████░ 89.5% of 1.84 MiB
logged-out boot: index + App + bootApp (preloaded by every page, including /login)
src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
3.74 MiB · 662 files 🔺 +714 B (+0.0%) █████████░ 92.7% of 4.03 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
7.60 MiB · 2,414 files 🔺 +4.3 KiB (+0.1%) █████████░ 91.2% of 8.34 MiB
dashboard scene
src/scenes/dashboard/Dashboard.tsx
9.70 MiB · 3,402 files 🔺 +3.8 KiB (+0.0%) ███████░░░ 72.0% of 13.48 MiB
today home path
src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
7.62 MiB · 2,422 files 🔺 +4.3 KiB (+0.1%) █████████░ 88.8% of 8.58 MiB
events scene
src/scenes/activity/explore/EventsScene.tsx
9.32 MiB · 3,254 files 🔺 +3.8 KiB (+0.0%) ███████░░░ 73.7% of 12.64 MiB
replay detail scene
src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
12.15 MiB · 4,140 files 🔺 +3.8 KiB (+0.0%) ████████░░ 77.3% of 15.72 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/layout/navigation-3000/navigationLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/dashboard/dashboardLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/lemon-ui/LemonMarkdown/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/RichContentEditor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/CodeSnippet/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/taxonomy/core-filter-definitions-by-group.json stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/player/sessionRecordingPlayerLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 src/scenes/project-homepage/today/TodayReportPage.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB src/index.tsx
1.3 KiB src/RootErrorBoundary.tsx
912 B ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
839 B src/scenes/ChunkLoadErrorBoundary.tsx
Largest files eagerly shipped from src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
Size File
306.2 KiB ../node_modules/.pnpm/posthog-js@1.435.5_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
221.0 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
100.5 KiB src/lib/api.ts
92.7 KiB src/products.tsx
69.4 KiB src/lib/lemon-ui/icons/icons.tsx
40.1 KiB src/lib/utils/eventUsageLogic.ts
38.7 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
33.9 KiB ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js
29.0 KiB ../node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/core/schemas.js
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
306.2 KiB ../node_modules/.pnpm/posthog-js@1.435.5_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
280.3 KiB src/taxonomy/core-filter-definitions-by-group.json
221.0 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
111.8 KiB ../packages/quill/packages/quill/dist/index.js
100.5 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
92.7 KiB src/products.tsx
90.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
Largest files eagerly shipped from src/scenes/dashboard/Dashboard.tsx
Size File
306.2 KiB ../node_modules/.pnpm/posthog-js@1.435.5_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
280.3 KiB src/taxonomy/core-filter-definitions-by-group.json
221.0 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
181.8 KiB src/queries/validators.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
111.8 KiB ../packages/quill/packages/quill/dist/index.js
100.5 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
92.7 KiB src/products.tsx
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
Size File
306.2 KiB ../node_modules/.pnpm/posthog-js@1.435.5_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
280.3 KiB src/taxonomy/core-filter-definitions-by-group.json
221.0 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
111.8 KiB ../packages/quill/packages/quill/dist/index.js
100.5 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
92.7 KiB src/products.tsx
90.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
Largest files eagerly shipped from src/scenes/activity/explore/EventsScene.tsx
Size File
306.2 KiB ../node_modules/.pnpm/posthog-js@1.435.5_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
280.3 KiB src/taxonomy/core-filter-definitions-by-group.json
221.0 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
181.8 KiB src/queries/validators.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
111.8 KiB ../packages/quill/packages/quill/dist/index.js
100.5 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
92.7 KiB src/products.tsx
Largest files eagerly shipped from src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
Size File
315.5 KiB ../node_modules/.pnpm/posthog-js@1.435.5_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/rrweb.js
306.2 KiB ../node_modules/.pnpm/posthog-js@1.435.5_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
280.3 KiB src/taxonomy/core-filter-definitions-by-group.json
221.0 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
181.8 KiB src/queries/validators.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
111.8 KiB ../packages/quill/packages/quill/dist/index.js
100.5 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

✅ Toolbar bundle — eager 2.21 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric Size Δ vs base Budget
Eager (shipped)
entry + static imports
2.21 MiB · 19 files 🔺 +116 B (+0.0%) ████░░░░░░ 38.6% of 5.72 MiB
Deferred (lazy) 2.11 MiB · 44 files no change n/a — loads on demand
Loader dist/toolbar.js 1.2 KiB no change █░░░░░░░░░ 6.0% of 19.5 KiB
Largest eagerly-shipped chunks
Size File
845.0 KiB dist/toolbar/toolbar-app-IMU442X7.css
657.6 KiB dist/toolbar/chunk-chunk-QRQSENDA.js
259.4 KiB dist/toolbar/chunk-chunk-7JWMBALG.js
138.2 KiB dist/toolbar/chunk-chunk-AAQSQGRK.js
131.8 KiB dist/toolbar/chunk-chunk-FDH2IBXT.js
75.2 KiB dist/toolbar/toolbar-app-3OJK4CX7.js
69.0 KiB dist/toolbar/chunk-chunk-TSAL54PB.js
35.6 KiB dist/toolbar/chunk-chunk-HBLANGBE.js
21.0 KiB dist/toolbar/chunk-chunk-LFOX5MDR.js
6.8 KiB dist/toolbar/chunk-chunk-DV7IWQNF.js

Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile

✅ Dist folder size — 🔺 +220.1 KiB (+0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 962.01 MiB · 🔺 +220.1 KiB (+0.0%)

ℹ️ MCP UI apps size — 33 app(s), 17633.6 KB JS

Built size of each MCP UI app (main.js + styles.css).

App JS CSS
debug 597.9 KB 199.4 KB
action 454.1 KB 199.4 KB
action-list 564.2 KB 199.4 KB
cohort 453.1 KB 199.4 KB
cohort-list 563.2 KB 199.4 KB
email-template 452.9 KB 199.4 KB
error-details 469.6 KB 199.4 KB
error-issue 454.5 KB 199.4 KB
error-issue-list 564.8 KB 199.4 KB
experiment 561.3 KB 199.4 KB
experiment-list 564.9 KB 199.4 KB
experiment-results 566.3 KB 199.4 KB
feature-flag 566.8 KB 199.4 KB
feature-flag-list 570.5 KB 199.4 KB
feature-flag-testing 457.3 KB 199.4 KB
inline-scan 453.6 KB 199.4 KB
insight-actors 562.3 KB 199.4 KB
invite-email-preview 452.3 KB 199.4 KB
llm-costs 559.3 KB 199.4 KB
session-recording 455.3 KB 199.4 KB
survey 454.7 KB 199.4 KB
survey-global-stats 561.9 KB 199.4 KB
survey-list 564.9 KB 199.4 KB
survey-stats 561.9 KB 199.4 KB
trace-span 453.5 KB 199.4 KB
trace-span-list 564.1 KB 199.4 KB
vision-observation-list 563.3 KB 199.4 KB
workflow 453.4 KB 199.4 KB
workflow-list 563.5 KB 199.4 KB
loops-review 457.8 KB 199.4 KB
query-results 774.3 KB 199.4 KB
render-ui 858.4 KB 199.4 KB
visual-review-snapshots 457.9 KB 199.4 KB
✅ Playwright — all passed

All tests passed.

View test results →

⚠️ Backend coverage — 97.0% of changed backend lines covered — 66 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ███████████████████░ 97.0% (2,244 / 2,310)

File Patch Uncovered changed lines
products/today/backend/logic/report_text.py 88.9% 32–34, 43
products/today/backend/logic/signal_previews.py 89.4% 67, 84, 105, 122–125, 137–138, 147, 184, 202–205, 217
products/today/backend/logic/sentences.py 90.1% 14–15, 23, 45, 77–78, 91–92
products/today/backend/facade/api.py 90.5% 68, 84
products/today/backend/logic/figure_sources.py 94.2% 113–114, 125, 157, 165, 194–195, 197–198, 237, 260
products/today/backend/logic/formats.py 95.1% 45, 48, 55, 104
products/today/backend/logic/impact.py 95.2% 83–84, 110–111, 123, 143
products/today/backend/logic/evidence.py 97.0% 71–72
products/today/backend/logic/figures.py 97.3% 66, 92, 109, 141
products/today/backend/tests/factories.py 97.6% 74
products/today/backend/presentation/views.py 97.8% 199, 220
products/signals/backend/report_page_source.py 98.3% 53
products/today/backend/logic/prose.py 98.4% 61
products/today/backend/logic/report_page.py 98.5% 52
products/today/backend/logic/jev.py 98.9% 50
products/today/backend/logic/signal_text.py 99.2% 149
products/today/backend/logic/key_clauses.py 99.2% 203

🤖 Agents: add a test only if an uncovered line exposes a realistic regression that existing tests miss. Otherwise explain why no new test is needed under "How did you test this code?". Gap list: the patch-coverage artifact on this run (gh run download 37188946345 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)
Product Coverage Lines
platform_features ██░░░░░░░░░░░░░░░░░░ 12.1% 7 / 58
demo ███████████░░░░░░░░░ 53.4% 1,445 / 2,707
data_tools ████████████░░░░░░░░ 61.2% 90 / 147
warehouse_sources_queue ██████████████░░░░░░ 68.2% 1,868 / 2,740
ai_gateway ███████████████░░░░░ 75.0% 9 / 12
aeo ███████████████░░░░░ 76.3% 617 / 809
batch_exports ████████████████░░░░ 81.2% 21,552 / 26,529
apm █████████████████░░░ 84.1% 1,306 / 1,553
cdp ██████████████████░░ 88.3% 4,559 / 5,164
ml_inference ██████████████████░░ 88.8% 539 / 607
mcp_analytics ██████████████████░░ 89.2% 5,038 / 5,651
product_tours ██████████████████░░ 89.3% 1,340 / 1,500
dashboards ██████████████████░░ 89.6% 6,926 / 7,730
notebooks ██████████████████░░ 90.2% 15,514 / 17,207
signals ██████████████████░░ 90.4% 60,211 / 66,572
cohorts ██████████████████░░ 90.5% 8,534 / 9,434
data_warehouse ██████████████████░░ 90.6% 14,375 / 15,860
streamlit_apps ██████████████████░░ 90.8% 2,679 / 2,951
managed_warehouse ██████████████████░░ 91.0% 10,252 / 11,263
data_modeling ██████████████████░░ 91.2% 10,562 / 11,584
tasks ██████████████████░░ 91.4% 80,071 / 87,649
exports ██████████████████░░ 91.7% 9,684 / 10,566
business_knowledge ██████████████████░░ 92.0% 8,472 / 9,208
engineering_analytics ██████████████████░░ 92.2% 11,497 / 12,475
ai_training ██████████████████░░ 92.2% 356 / 386
webmcp ███████████████████░ 92.5% 248 / 268
early_access_features ███████████████████░ 92.6% 1,339 / 1,446
conversations ███████████████████░ 92.6% 29,225 / 31,559
visual_review ███████████████████░ 92.7% 10,000 / 10,785
managed_migrations ███████████████████░ 92.7% 1,581 / 1,705
stamphog ███████████████████░ 92.8% 8,109 / 8,742
canvas ███████████████████░ 92.9% 7,167 / 7,715
approvals ███████████████████░ 93.0% 3,974 / 4,271
mcp_registry ███████████████████░ 93.1% 1,670 / 1,794
notifications ███████████████████░ 93.2% 1,144 / 1,228
error_tracking ███████████████████░ 93.3% 16,389 / 17,573
surveys ███████████████████░ 93.4% 6,644 / 7,113
autoresearch ███████████████████░ 93.6% 8,971 / 9,589
slack_app ███████████████████░ 93.7% 14,611 / 15,600
alerts ███████████████████░ 93.8% 6,740 / 7,186
context_layer ███████████████████░ 93.8% 3,415 / 3,639
web_analytics ███████████████████░ 93.9% 23,763 / 25,313
billing_alerts ███████████████████░ 94.1% 2,094 / 2,226
mcp_store ███████████████████░ 94.3% 8,959 / 9,501
wizard ███████████████████░ 94.7% 6,169 / 6,515
reminders ███████████████████░ 94.8% 760 / 802
workflows ███████████████████░ 94.9% 15,693 / 16,543
ai_observability ███████████████████░ 94.9% 26,288 / 27,689
review_hog ███████████████████░ 95.0% 11,750 / 12,362
annotations ███████████████████░ 95.1% 817 / 859
customer_analytics ███████████████████░ 95.2% 26,116 / 27,428
legal_documents ███████████████████░ 95.2% 2,311 / 2,427
marketing_analytics ███████████████████░ 95.3% 19,451 / 20,414
today ███████████████████░ 95.3% 3,015 / 3,163
endpoints ███████████████████░ 95.3% 9,345 / 9,801
posthog_ai ███████████████████░ 95.4% 2,530 / 2,653
actions ███████████████████░ 95.5% 756 / 792
logs ███████████████████░ 95.5% 15,459 / 16,191
experiments ███████████████████░ 95.6% 33,547 / 35,103
data_catalog ███████████████████░ 95.6% 4,402 / 4,606
tracing ███████████████████░ 95.6% 3,536 / 3,699
replay_vision ███████████████████░ 95.7% 30,295 / 31,660
growth ███████████████████░ 95.7% 11,381 / 11,888
skills ███████████████████░ 95.8% 6,972 / 7,274
messaging ███████████████████░ 95.9% 3,834 / 3,999
product_analytics ███████████████████░ 96.0% 28,521 / 29,696
alerts_platform ███████████████████░ 96.3% 3,554 / 3,691
revenue_analytics ███████████████████░ 96.4% 1,889 / 1,959
user_interviews ███████████████████░ 96.5% 2,870 / 2,974
feature_flags ███████████████████░ 96.6% 27,119 / 28,060
access_control ███████████████████░ 96.7% 7,739 / 8,007
warehouse_sources ███████████████████░ 97.3% 475,604 / 488,703
data_quality ████████████████████ 97.5% 7,701 / 7,895
links ████████████████████ 97.9% 234 / 239
metrics ████████████████████ 98.0% 4,270 / 4,356
security ████████████████████ 98.0% 1,304 / 1,330
analytics_platform ████████████████████ 98.3% 2,784 / 2,833
pulse ████████████████████ 98.5% 2,046 / 2,078
live_debugger ████████████████████ 99.2% 626 / 631
field_notes ████████████████████ 99.4% 172 / 173

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

@trunk-io

trunk-io Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

Failed Test Failure Summary Logs
Scenes-App/Replay Vision/Observations tab in the player TimelineWithScans smoke-test The test failed because the player controls were still visible when they were expected to be hidden. Logs ↗︎
compareTopLevelSections() reports a modifiers change when the current query overrides the team default A TypeError occurred because the code attempted to access the 'add' property of an undefined object. Logs ↗︎

View Full Report ↗︎ ⋅ Docs

@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
🧰 Additional context used
📚 Code guidelines (14)
.cursor/rules/react-typescript.mdc — auto-discovered
.agents/skills/using-kea-disposables/SKILL.md — configured
.agents/skills/writing-ui-components/SKILL.md — configured
.agents/security.md — configured
.agents/skills/sending-notifications/SKILL.md — configured
docs/published/handbook/engineering/type-system.md — configured
.agents/skills/writing-tests/SKILL.md — configured
docs/internal/person-data-access.md — configured
.agents/skills/adopting-generated-api-types/SKILL.md — configured
.agents/skills/placing-product-frontend-code/SKILL.md — configured
.claude/commands/conventions.md — configured
.agents/skills/writing-kea-logics/SKILL.md — configured
.agents/skills/writing-user-facing-copy/SKILL.md — configured
.agents/skills/writing-code-comments/SKILL.md — configured

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
  • Review profile: QUIET
  • Plan: Enterprise
  • Run ID: 9a9a08a7-9726-4df2-9c72-50a72a9a10e8
📥 Commits

Reviewing files that changed from the base of the PR and between 6417681 and 494abdb.

📒 Files selected for processing (3)
  • frontend/src/scenes/project-homepage/today/todayReportLogic.ts
  • products/signals/backend/facade/api.py
  • products/signals/backend/test/test_report_page_source.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Walkthrough

Walkthrough

The change adds JEV-backed figure-source matching, key-clause extraction, and code-excerpt selection for Today reports. New backend contracts and endpoints expose these operations, while frontend state loads and maps their results. Today report text now renders figure marks and key-clause cards, with source details and stale-evidence dates. Tests cover parsing, matching, endpoint behavior, and frontend text segmentation.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 494ab

The reviewed changes hide fetched annotations when JEV is disabled, handle stale impact evidence, respect request limits, and preserve selected code-file identities. No concrete merge-blocking risk remains.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 494ab

The new analysis flow has server-side eligibility checks, report-read controls, tenant-separated caching, throttling, and bounded requests. No introduced security vulnerability was established. Authorization for additional research excerpts and some rollback behavior still require confirmation.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The supported exposure is authenticated, eligible team members invoking analysis billed to their team. Report-source responses additionally require task-viewer authority. Cache reuse is team-wide, but its keys include the complete question and input, and its values are model decisions rather than source payloads.

Security Findings and Attack Paths

  • inferred — Additional research quotes do not by themselves establish an authorization bypass. Today already retrieves the full report, and an existing client API retrieves report artefacts. The effective permissions of that artefact route remain unverified, so the new quote response cannot yet be classified as unauthorized disclosure.

Trust Boundaries and Controls

  • observed — Backend eligibility and report authorization precede report analysis. The excerpt endpoint has a different boundary: it accepts caller-provided material and returns an index, without loading report contents. Frontend excerpt failures, null indices and unresolved candidate indices retain the original quote.

Resilience and Maintainability Implications

  • observed — All three new endpoints add user-scoped burst and sustained throttles alongside the default throttles. Frontend failed figure requests clear their pending cache; failed key-clause requests release pending texts. These paths permit later retries without accepting a partial successful annotation response.

Hardening Proposals

  • proposed — If concurrent cold-cache requests can materially consume shared credits or gateway capacity, consider team-scoped request coalescing and a shared concurrency budget. The inspected cache and semaphore operate without cross-request single-flight coordination; this is a resource-control proposal, not an established denial-of-service finding.
🚥 Pre-merge checks | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description clearly explains the problem, user-visible changes, testing, and release status. It omits required flow diagrams for the new request paths and several required agent-context details. Add separate before-and-after Mermaid flowcharts for the changed request paths. Complete the agent context with the session link, CodeRabbit CLI pass or skip reason, duplicate-PR search result, patch-coverage result, new-events-schema deter…
Full details: Description check

Resolution

Add separate before-and-after Mermaid flowcharts for the changed request paths. Complete the agent context with the session link, CodeRabbit CLI pass or skip reason, duplicate-PR search result, patch-coverage result, new-events-schema determination and result, and public-artifact statement.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (4)
frontend/src/scenes/project-homepage/today/todayReportLogic.ts-305-306 (1)

305-306: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Hide JEV annotations when the flag is off.

When setFeatureFlags disables today-report-jev, stored figure marks still flow through leadMarks and impactMarks, and TodayMarkedText still renders stored key clauses. An in-flight key-clause request can also store results after the flag changes. Gate both figure selectors and the clauses passed to markedRuns on asksJev.

Suggested fix
diff --git a/frontend/src/scenes/project-homepage/today/todayReportLogic.ts b/frontend/src/scenes/project-homepage/today/todayReportLogic.ts
@@
         leadMarks: [
-            (s) => [s.lead, s.figureMarks, s.signals],
-            (lead: string, figureMarks: FigureMarkApi[] | null, signals: SignalViewApi[]): TodayMarkedFigure[] =>
-                markedFigures(lead, marksIn(figureMarks, 'lead'), signals),
+            (s) => [s.asksJev, s.lead, s.figureMarks, s.signals],
+            (
+                asksJev: boolean,
+                lead: string,
+                figureMarks: FigureMarkApi[] | null,
+                signals: SignalViewApi[]
+            ): TodayMarkedFigure[] =>
+                asksJev ? markedFigures(lead, marksIn(figureMarks, 'lead'), signals) : [],
         ],
         impactMarks: [
-            (s) => [s.impactText, s.figureMarks, s.signals],
-            (impactText: string, figureMarks: FigureMarkApi[] | null, signals: SignalViewApi[]): TodayMarkedFigure[] =>
-                markedFigures(impactText, marksIn(figureMarks, 'impact'), signals),
+            (s) => [s.asksJev, s.impactText, s.figureMarks, s.signals],
+            (
+                asksJev: boolean,
+                impactText: string,
+                figureMarks: FigureMarkApi[] | null,
+                signals: SignalViewApi[]
+            ): TodayMarkedFigure[] =>
+                asksJev ? markedFigures(impactText, marksIn(figureMarks, 'impact'), signals) : [],
         ],
diff --git a/frontend/src/scenes/project-homepage/today/TodayMarkedText.tsx b/frontend/src/scenes/project-homepage/today/TodayMarkedText.tsx
@@
-    const { keyClauses } = useValues(todayReportLogic({ reportId }))
-    const runs = markedRuns(markdown, marked, keyClauses[renderedText(markdown)] ?? [])
+    const { asksJev, keyClauses } = useValues(todayReportLogic({ reportId }))
+    const runs = markedRuns(markdown, marked, asksJev ? keyClauses[renderedText(markdown)] ?? [] : [])
frontend/src/scenes/project-homepage/today/TodayReportHeader.tsx-107-112 (1)

107-112: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Render the evidence-age warning when the lead is empty.

An impact-only report can contain a stale marked figure, but the header mounts TodayEvidenceAge only when lead is truthy. Render it after the lead conditional so the warning also appears for impact-only reports.

🐛 Suggested fix
                 {lead ? (
                     <div data-today-figures>
                         <Text size="sm" render={<p />} className="leading-relaxed text-pretty">
                             <TodayMarkedText markdown={lead} marked={leadMarks} reportId={report.id} />
                         </Text>
-                        <TodayEvidenceAge report={report} />
                     </div>
                 ) : (
                     <Text variant="muted" render={<p />}>
                         No summary yet. An agent is still investigating.
                     </Text>
                 )}
+                <TodayEvidenceAge report={report} />
frontend/src/scenes/project-homepage/today/todayMarkedRuns.ts-47-80 (1)

47-80: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Split code and link segments at key-clause boundaries.

When a clause starts or ends inside an inline code or link segment, placedPieces keeps that segment whole. The containment check then rejects the whole piece, so the clause mark can lose part of its text.

For example, Users report failures in \cart because service drops tokens repeatedly` during checkout.can produce a clause starting atbecause. The code piece starts at cart, so it is assigned to null` instead of the clause.

Suggested fix
+function clausePieces(
+    text: string,
+    segment: number,
+    offset: number,
+    keyClauses: KeyClauseApi[],
+    makePiece: (key: string, text: string) => TodayMarkedPiece
+): PlacedPiece[] {
+    const cuts = [
+        0,
+        text.length,
+        ...keyClauses.flatMap((keyClause) => [keyClause.start - offset, keyClause.end - offset]),
+    ]
+        .filter((point) => point >= 0 && point <= text.length)
+        .sort((first, second) => first - second)
+
+    return [...new Set(cuts)].slice(0, -1).map((from, index) => {
+        const to = [...new Set(cuts)][index + 1]
+        const key = from === 0 ? `${segment}` : `${segment}-${from}`
+        return { piece: makePiece(key, text.slice(from, to)), from: offset + from, to: offset + to }
+    })
+}
+
 function placedPieces(markdown: string, figures: TodayMarkedFigure[], keyClauses: KeyClauseApi[]): PlacedPiece[] {
@@
             case 'code':
-                return [{ piece: { kind: 'code', key, text: segment.text }, from, to: offset }]
+                return clausePieces(segment.text, index, from, keyClauses, (key, text) => ({
+                    kind: 'code',
+                    key,
+                    text,
+                }))
             case 'link':
-                return [{ piece: { kind: 'link', key, text: segment.text, href: segment.href }, from, to: offset }]
+                return clausePieces(segment.text, index, from, keyClauses, (key, text) => ({
+                    kind: 'link',
+                    key,
+                    text,
+                    href: segment.href,
+                }))
products/today/backend/logic/report_page.py-133-151 (1)

133-151: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Align frontend date normalization with backend offsets.

When a report lead contains an out-of-range date such as 2026-13-01 before a matched number, the backend keeps the ten-character date while Day.js normalizes it to 1 Jan. The frontend then applies backend offsets to shorter text and discards the valid figure mark.

Suggested fix
 function readableDates(text: string): string {
     return text.replace(ISO_DATE, (match) => {
+        const month = Number(match.slice(5, 7))
+        const day = Number(match.slice(8, 10))
+        if (month < 1 || month > 12 || day < 1 || day > 31) {
+            return match
+        }
         const date = dayjs(match)
         return date.isValid() ? shortDate(date) : match
     })
 }

ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
  • Review profile: QUIET
  • Plan: Enterprise
  • Run ID: ae244eee-a568-4da1-ba9f-0b37dc643e6c
📥 Commits

Reviewing files that changed from the base of the PR and between adde1e8 and 8ac6bc2.

⛔ Files ignored due to path filters (3)
  • products/today/frontend/generated/api.schemas.ts is excluded by !**/generated/**
  • products/today/frontend/generated/api.ts is excluded by !**/generated/**
  • products/today/frontend/generated/api.zod.ts is excluded by !**/generated/**
📒 Files selected for processing (42)
  • frontend/src/lib/constants.tsx
  • frontend/src/scenes/project-homepage/today/Today.scss
  • frontend/src/scenes/project-homepage/today/TodayEvidenceAge.tsx
  • frontend/src/scenes/project-homepage/today/TodayFigureCard.tsx
  • frontend/src/scenes/project-homepage/today/TodayHoverMark.tsx
  • frontend/src/scenes/project-homepage/today/TodayInlineMarkdown.tsx
  • frontend/src/scenes/project-homepage/today/TodayKeyClauseMark.tsx
  • frontend/src/scenes/project-homepage/today/TodayMarkedText.tsx
  • frontend/src/scenes/project-homepage/today/TodayReportAbstract.tsx
  • frontend/src/scenes/project-homepage/today/TodayReportHeader.tsx
  • frontend/src/scenes/project-homepage/today/TodayReportProposal.tsx
  • frontend/src/scenes/project-homepage/today/todayFigureSources.test.ts
  • frontend/src/scenes/project-homepage/today/todayFigureSources.ts
  • frontend/src/scenes/project-homepage/today/todayFigures.test.ts
  • frontend/src/scenes/project-homepage/today/todayFigures.ts
  • frontend/src/scenes/project-homepage/today/todayMarkedRuns.test.ts
  • frontend/src/scenes/project-homepage/today/todayMarkedRuns.ts
  • frontend/src/scenes/project-homepage/today/todayQuotedCode.ts
  • frontend/src/scenes/project-homepage/today/todayReportLogic.ts
  • products/signals/backend/facade/api.py
  • products/signals/backend/report_page_source.py
  • products/signals/backend/test/test_report_page_source.py
  • products/today/backend/facade/api.py
  • products/today/backend/facade/contracts.py
  • products/today/backend/facade/enums.py
  • products/today/backend/feature_flags.py
  • products/today/backend/logic/code_excerpts.py
  • products/today/backend/logic/figure_sources.py
  • products/today/backend/logic/figures.py
  • products/today/backend/logic/jev.py
  • products/today/backend/logic/key_clauses.py
  • products/today/backend/logic/report_page.py
  • products/today/backend/logic/sentences.py
  • products/today/backend/presentation/serializers.py
  • products/today/backend/presentation/views.py
  • products/today/backend/tests/test_api.py
  • products/today/backend/tests/test_figure_sources.py
  • products/today/backend/tests/test_figures.py
  • products/today/backend/tests/test_jev.py
  • products/today/backend/tests/test_key_clauses.py
  • products/today/mcp/tools.yaml
  • services/mcp/src/api/generated.ts
💤 Files with no reviewable changes (1)
  • frontend/src/scenes/project-homepage/today/TodayInlineMarkdown.tsx

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.

@puemos
puemos force-pushed the posthog/today-report-stack-4-jev branch from 8ac6bc2 to d339e9b Compare October 4, 2026 06:05
puemos added a commit that referenced this pull request Oct 4, 2026
… review fixes

Keeps this backup in step with the stack (#111470 to #111473):

- The report page, key clauses and figure marks now require task:read as well as today:read for a scoped key.
- Picking PostHog in the Implement with menu sends copy_implementation_prompt.
- An evidence row with nothing to open gets no click handler.
- The feedback note's focus-on-open state moves into todayReportLogic.

Generated-By: PostHog Desktop
Task-Id: 96e23785-93a4-43ea-b38e-492346d09a90

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (1)
frontend/src/scenes/project-homepage/today/todayReportLogic.ts-306-317 (1)

306-317: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Prevent duplicate figure-mark loads before dispatch.

loadPageSuccess and feature-flag updates both dispatch loadFigureMarks. If a setFeatureFlags event arrives while the first GET is pending, kea-loaders@3.1.1 starts a second request; it does not cancel or coalesce the first. Check figureMarksLoading at the dispatch sites. Do not add the guard inside the loader: returning null is treated as success and clears the loading flag while the first request is still pending. A failed request leaves figureMarks null, so a later flag callback can retry.

Suggested fix
@@
-        return {
+        const loadFigureMarksIfNeeded = (): void => {
+            if (values.asksJev && values.figureMarks === null && !values.figureMarksLoading) {
+                actions.loadFigureMarks()
+            }
+        }
+        return {
@@
             loadPageSuccess: () => {
                 actions.loadKeyClauses()
-                actions.loadFigureMarks()
+                loadFigureMarksIfNeeded()
             },
             [featureFlagLogic.actionTypes.setFeatureFlags]: () => {
                 actions.loadKeyClauses()
-                actions.loadFigureMarks()
+                loadFigureMarksIfNeeded()
             },

ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
  • Review profile: QUIET
  • Plan: Enterprise
  • Run ID: 9669b8e5-107c-45f3-bc75-b5173943ecf7
📥 Commits

Reviewing files that changed from the base of the PR and between 8ac6bc2 and d339e9b.

⛔ Files ignored due to path filters (3)
  • products/today/frontend/generated/api.schemas.ts is excluded by !**/generated/**
  • products/today/frontend/generated/api.ts is excluded by !**/generated/**
  • products/today/frontend/generated/api.zod.ts is excluded by !**/generated/**
📒 Files selected for processing (4)
  • frontend/src/scenes/project-homepage/today/todayReportLogic.ts
  • products/today/backend/presentation/views.py
  • products/today/backend/tests/test_api.py
  • services/mcp/src/api/generated.ts

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 3 remain after this review.

Comment thread frontend/src/scenes/project-homepage/today/todayReportLogic.ts
@puemos
puemos removed this pull request from stack #111474 October 4, 2026 06:25
@puemos
puemos added this pull request to stack #111485 October 4, 2026 06:25
@puemos
puemos force-pushed the posthog/today-report-stack-4-jev branch from d339e9b to 0ce8760 Compare October 4, 2026 06:37

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (2)
frontend/src/scenes/project-homepage/today/todayReportLogic.ts-462-465 (1)

462-465: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Keep over-limit text from rejecting the key-clause batch.

renderedText(lead) can exceed 4,000 characters because the report-page producer does not truncate the lead. The API rejects the entire batch when any KeyClauseRequestSerializer.text exceeds its limit. The catch then stores empty clauses for every missing text, so shorter impact and proposal texts can lose their clauses too. Exclude over-limit text before batching; if it still needs marks, split it and map its offsets and lookup key back to the full displayed text.

Skip over-limit requests
-                ].filter((request): request is KeyClauseRequestApi => !!request.text),
+                ].filter(
+                    (request): request is KeyClauseRequestApi =>
+                        !!request.text && request.text.length <= 4000
+                ),
frontend/src/scenes/project-homepage/today/todayReportLogic.ts-524-525 (1)

524-525: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Keep file identities through Jev selection.

When values.asksJev is true and two cited files each produce one candidate, findCodeQuote returns candidates from only one file. readCode then skips Jev because the returned candidate list has fewer than two entries. Collect candidates with their file identities before requesting a choice, then store the selected file and excerpt together.


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
  • Review profile: QUIET
  • Plan: Enterprise
  • Run ID: 512d41bc-219c-4748-ba01-10c009ce8355
📥 Commits

Reviewing files that changed from the base of the PR and between d339e9b and 0ce8760.

⛔ Files ignored due to path filters (3)
  • products/today/frontend/generated/api.schemas.ts is excluded by !**/generated/**
  • products/today/frontend/generated/api.ts is excluded by !**/generated/**
  • products/today/frontend/generated/api.zod.ts is excluded by !**/generated/**
📒 Files selected for processing (1)
  • frontend/src/scenes/project-homepage/today/todayReportLogic.ts

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 3 remain after this review.

@puemos
puemos force-pushed the posthog/today-report-stack-4-jev branch from 0ce8760 to 3c65c5c Compare October 4, 2026 07:38
puemos added 2 commits October 4, 2026 09:44
Behind today-report-jev, the report page asks Jev, PostHog's decision model, two things:

- Key clauses: POST today/reports/<id>/key_clauses marks the clauses that state the problem, the cause and the fix, with the report sentences that explain each.
- Figure marks: GET today/reports/<id>/figure_marks marks a number in the lead or impact sentence only when a signal or the agent's research states the same result. The hover card underlines the exact number with offsets from the backend.

Code only finds numbers and compares exact values. Jev (jevk5-fp8-0.2, pinned) answers four Choice questions, each with its own threshold, and a number gets a mark only when every answer passes. Code excerpts with several candidates also ask Jev which one the finding describes. Answers are cached per team for 30 days. The endpoints answer 404 without the flag and 503 when Jev is unavailable. Sample reports never ask Jev.

Generated-By: PostHog Desktop
Task-Id: 96e23785-93a4-43ea-b38e-492346d09a90
Each excerpt candidate keeps its file, so a pick from another file shows that file. Texts over the key clause limit stay out of the request, so one long lead does not fail the whole batch.

Generated-By: PostHog Desktop
Task-Id: 96e23785-93a4-43ea-b38e-492346d09a90
@puemos
puemos force-pushed the posthog/today-report-stack-4-jev branch from 5cfada4 to 1e9419d Compare October 4, 2026 07:44

puemos commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor Author

Review outcomes for this round. All fixes are in 6570aec, 1e9419d and 75b3078.

Finding Outcome
loadKeyClauses race and cached failures Fixed. Pending set and no store on failure.
Duplicate figure mark loads Fixed. One shared request per page, cleared on failure so a later load asks again.
Evidence age, segment split, date parity Fixed.
Over-limit text rejects the key clause batch Fixed. A text over the serializer limit stays out of the request.
Keep file identities through Jev selection Fixed. Each candidate keeps its file, and the pick shows that file. A new todayQuotedCode test covers two files.
Hide Jev marks when the flag is off Fixed in 75b3078. Marks and key clauses show only while the page asks Jev.

Other changes from the review of this layer:

  • Jev requests send up to 32 rows each, at most 4 at a time, under one 45 second deadline. Answers from batches that succeed stay cached when another batch fails.
  • A gateway 402 now returns 402 with the out-of-credits message. Other gateway failures and timeouts return 503.
  • Figure questions run in stages. Jev gets the source question only for measured numbers, and the relation and named questions only for the agreed source.
  • Key clause and figure offsets count UTF-16 units, the same as the browser.
  • The three Jev actions have burst and sustained throttles, plus the default API throttles.

Key clauses and figure marks show only while the page asks Jev, so marks loaded before the flag turned off stop showing.

Generated-By: PostHog Desktop
Task-Id: 96e23785-93a4-43ea-b38e-492346d09a90
puemos added a commit that referenced this pull request Oct 4, 2026
Brings this branch to the same Today files as the top of the stack (#111471, #111472, #111473): report access checks, scoped key rules, the popover marks, the free trial rule for investigations, batched Jev requests, UTF-16 offsets, throttles and the flag gating.

Generated-By: PostHog Desktop
Task-Id: 96e23785-93a4-43ea-b38e-492346d09a90
@puemos
puemos marked this pull request as ready for review October 4, 2026 08:01
@parameterai

parameterai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Risk: No findings

The delta since the last review tightens report_summary in products/signals/backend/facade/api.py to exclude DELETED reports, matching the existing behavior of report_page_source, and extends the test to cover it. It is a pure access restriction with no new attack surface, and it closes the path where a deleted report's summary could reach the today/reports/<id>/key_clauses endpoint.

Sentinel reviewed 494abdb · Review settings

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

🦔 Hogbox preview · ✅ ready

▶ Open the preview

🔑 Login test@posthog.com / 12345678 (demo data)
🧩 Running this PR's backend and frontend, on the PostHog :master base
🔗 Link stable across rebuilds — a re-push swaps the box underneath, the URL stays
🔒 Access tailnet only (PostHog VPN)
🛠️ Admin inspect & debug state in hogland
💤 Idle sleeps after ~30 min idle (snapshot to S3, zero node cost) and wakes on your next visit in ~30s, behind a brief "waking up" screen

commit 494abdb · box box-2c61b358e7b1 · ready in 668s (push → usable) · build log · rebuilds on every push, torn down on close

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested review from a team October 4, 2026 08:02
Comment thread products/signals/backend/facade/api.py Outdated
@veria-ai

veria-ai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

Generated-By: PostHog Desktop
Task-Id: 96e23785-93a4-43ea-b38e-492346d09a90
@github-actions
github-actions Bot requested a deployment to preview-pr-111473 October 4, 2026 08:25 In progress
The summary lookup behind the key clause expansions now skips deleted reports, like the page and artefact reads.

Generated-By: PostHog Desktop
Task-Id: 96e23785-93a4-43ea-b38e-492346d09a90

This branch was successfully deployed

1 active deployment
preview-pr-111473 — 494abdb9 Deployed Oct 4, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant