Conversation
An output names its producer in an environment variable, so a slot name has to parse. Only declared slots parse; anything else fails boot and lists the valid names.
Every v0 destination is now an output with a topic and a named producer, read
from CAPTURE_OUTPUT_<OUTPUT>_TOPIC and CAPTURE_OUTPUT_<OUTPUT>_PRODUCER, the
same shape as Node ingestion's INGESTION_OUTPUT_<OUTPUT>_{TOPIC,PRODUCER}.
Moving one output to another cluster is now configuration.
Explicit migration: capture stops reading KAFKA_TOPIC, KAFKA_*_TOPIC and
CAPTURE_ANALYTICS_AI_EVENTS_*_TOPIC. charts#16337 sets the new names first.
Topic defaults are unchanged; producers default to INGESTION.
KAFKA_TOPIC splits in two. Session replay's main topic is its own output
instead of sharing the analytics main topic; recordings pods only reach replay
outputs, so each deployment maps its KAFKA_TOPIC to exactly one of them.
TopicTable becomes OutputTable. PreparedPayload carries the Destination, as
v1's PreparedEvent does, and the Kafka sink resolves it to a topic and producer
at enqueue. Topics are Arc<str>, so the serial enqueue allocates once per
record for the ProduceRecord topic. Custom redirects publish through
CAPTURE_OUTPUT_CUSTOM_PRODUCER.
docker-compose keeps the legacy names beside the new ones until the new image
is the published one; a stale capture:master would otherwise send replay to
the analytics default topic.
Step 17's fallback follows the per-output shape: a secondary target on each output, as in Node ingestion's dual-write outputs.
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
🤖 CI report🚨 Trunk lane — universal laneThis PR is assigned to the universal lane. It cannot merge in parallel with other PRs, so it can take longer to merge. Ask dev-ex if you think this is wrong. 🚨 Comment density — 11% of added code lines are comments (70 of 662)This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out. Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See Files with the most added comment lines:
This check does not block merging. It updates on every push and clears when the share drops.
|
| File | Size | Δ vs base |
|---|---|---|
render-query/src/render-query/render-query.js |
20.23 MiB | 🔺 +9.1 KiB (+0.0%) |
posthog-app/_parent/products/engineering_analytics/frontend/scenes/EngineeringAnalyticsAuthorScene.js |
23.4 KiB | 🔺 +3.6 KiB (+18.4%) |
posthog-app/_parent/products/engineering_analytics/frontend/scenes/PullRequestDetailScene.js |
31.2 KiB | 🔺 +2.5 KiB (+8.8%) |
posthog-app/_parent/products/engineering_analytics/frontend/scenes/EngineeringAnalyticsScene.js |
78.1 KiB | 🔺 +2.3 KiB (+3.1%) |
posthog-app/_parent/products/workflows/frontend/Broadcasts/BroadcastScene.js |
42.4 KiB | 🔺 +1.8 KiB (+4.4%) |
posthog-app/_parent/products/engineering_analytics/frontend/scenes/EngineeringAnalyticsTeamScene.js |
16.6 KiB | 🔺 +1.5 KiB (+10.3%) |
posthog-app/_parent/products/workflows/frontend/Broadcasts/BroadcastsScene.js |
14.8 KiB | 🔺 +1.2 KiB (+8.5%) |
posthog-app/_parent/products/ai_observability/frontend/evaluations/AIObservabilityEvaluation.js |
80.9 KiB | 🔺 +1.1 KiB (+1.4%) |
Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report
✅ Eager graph — within budget
How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.
| Root | Eager (shipped) | Δ vs base | Budget |
|---|---|---|---|
entry (logged-out pages, app bootstrap)src/index.tsx |
1.57 MiB · 22 files | 🔺 +916 B (+0.1%) | █████████░ 85.1% of 1.84 MiB |
logged-out boot: index + App + bootApp (preloaded by every page, including /login)src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts |
3.57 MiB · 628 files | 🔺 +1.2 KiB (+0.0%) | █████████░ 88.6% of 4.03 MiB |
authenticated shell (every logged-in page)src/scenes/AuthenticatedShell.tsx |
7.27 MiB · 2,301 files | 🔺 +9.1 KiB (+0.1%) | █████████░ 87.2% of 8.34 MiB |
🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/layout/navigation-3000/navigationLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/dashboard/dashboardLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/lemon-ui/LemonMarkdown/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/RichContentEditor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/CodeSnippet/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/taxonomy/core-filter-definitions-by-group.json stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/player/sessionRecordingPlayerLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
Largest files eagerly shipped from src/index.tsx
| Size | File |
|---|---|
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 24.6 KiB | ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js |
| 6.3 KiB | ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js |
| 4.5 KiB | ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js |
| 3.9 KiB | ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js |
| 1.4 KiB | ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js |
| 1.3 KiB | src/index.tsx |
| 1.3 KiB | src/RootErrorBoundary.tsx |
| 912 B | ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js |
| 854 B | src/scenes/ChunkLoadErrorBoundary.tsx |
Largest files eagerly shipped from src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
| Size | File |
|---|---|
| 301.8 KiB | ../node_modules/.pnpm/posthog-js@1.434.13_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs |
| 267.6 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 100.4 KiB | src/lib/api.ts |
| 84.9 KiB | src/products.tsx |
| 69.1 KiB | src/lib/lemon-ui/icons/icons.tsx |
| 63.9 KiB | src/lib/utils/eventUsageLogic.ts |
| 38.7 KiB | ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js |
| 33.9 KiB | ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js |
| 28.3 KiB | src/scenes/scenes.ts |
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
| Size | File |
|---|---|
| 301.8 KiB | ../node_modules/.pnpm/posthog-js@1.434.13_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs |
| 271.7 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 267.6 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 153.7 KiB | ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 100.4 KiB | src/lib/api.ts |
| 98.5 KiB | ../packages/quill/packages/quill/dist/index.js |
| 93.3 KiB | ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js |
| 90.6 KiB | ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js |
| 84.9 KiB | src/products.tsx |
Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479
✅ Toolbar bundle — eager 2.37 MiB within budget
What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.
| Metric | Size | Δ vs base | Budget |
|---|---|---|---|
| Eager (shipped) entry + static imports |
2.37 MiB · 19 files | 🔺 +556 B (+0.0%) | ████░░░░░░ 41.4% of 5.72 MiB |
| Deferred (lazy) | 2.10 MiB · 44 files | no change | n/a — loads on demand |
Loader dist/toolbar.js |
1.2 KiB | no change | █░░░░░░░░░ 6.0% of 19.5 KiB |
Largest eagerly-shipped chunks
| Size | File |
|---|---|
| 790.2 KiB | dist/toolbar/toolbar-app-ZNXS4H6W.css |
| 650.4 KiB | dist/toolbar/chunk-chunk-DPTOFHTZ.js |
| 483.6 KiB | dist/toolbar/chunk-chunk-DPLA2GAY.js |
| 138.3 KiB | dist/toolbar/chunk-chunk-2EHPTPPA.js |
| 131.8 KiB | dist/toolbar/chunk-chunk-FDH2IBXT.js |
| 75.2 KiB | dist/toolbar/toolbar-app-FOEGQ3V3.js |
| 69.0 KiB | dist/toolbar/chunk-chunk-TSAL54PB.js |
| 35.6 KiB | dist/toolbar/chunk-chunk-U5CQIMDB.js |
| 21.0 KiB | dist/toolbar/chunk-chunk-OYNHVBHM.js |
| 6.8 KiB | dist/toolbar/chunk-chunk-DV7IWQNF.js |
Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile
✅ Dist folder size — 🟢 -871.16 MiB (-48.0%)
Total size of the built frontend/dist folder (all assets), compared against the base branch.
Total: 943.47 MiB · 🟢 -871.16 MiB (-48.0%)
|
[High risk] Refactors how the capture service routes events to Kafka topics. The PR appears safe to merge, subject to the stated charts-first deployment order. Reviews (1) · Last reviewed commit: "docs(capture): record steps 9 and 10 as ..." |
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository: PostHog/posthog/.coderabbit.yaml Review profile: QUIET Plan: Enterprise Run ID: 📒 Files selected for processing (8)
💤 Files with no reviewable changes (2)
Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review. 📝 WalkthroughWalkthroughCapture now configures topics and producer names per output. Priority: ➖ Normal Merge Risk: ⚪ Minimal · up to The shutdown flush handles the currently configured producer. No actionable merge-blocking issue remains after normal checks. Security Architecture ReviewSecurity architecture risk: 🟡 Moderate · up to Per-output routing is a meaningful design change. The main risk is a deployment that still supplies customized legacy topic settings: the new capture process ignores them and may publish to default topics instead. The production configuration change and deployment order could not be verified here. Retained concerns
Security review detailsSecurity Blast Radius
Security Findings and Attack Paths
Trust Boundaries and Controls
Resilience and Maintainability Implications
Hardening Proposals
🚥 Pre-merge checks | ✅ 1✅ Passed checks (1 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
Review profile: QUIET
Plan: Enterprise
Run ID: c3582124-8ff0-49f6-9f53-248c54a61228
📒 Files selected for processing (26)
bin/start-rust-servicedocker-compose.base.ymlrust/capture/OUTPUTS_REFACTOR_PLAN.mdrust/capture/src/config.rsrust/capture/src/events/analytics.rsrust/capture/src/events/overflow_stamping.rsrust/capture/src/events/recordings.rsrust/capture/src/outputs.rsrust/capture/src/overflow_parity.rsrust/capture/src/producers.rsrust/capture/src/router.rsrust/capture/src/setup.rsrust/capture/src/sinks/kafka.rsrust/capture/src/sinks/registry.rsrust/capture/src/sinks/sink.rsrust/capture/src/v0_request.rsrust/capture/src/v1/analytics/process.rsrust/capture/src/v1/quota_limiter_shim.rsrust/capture/src/v1/sinks/kafka/config.rsrust/capture/src/v1/sinks/types.rsrust/capture/src/v1/test_utils.rsrust/capture/tests/common/utils.rsrust/capture/tests/events.rsrust/capture/tests/integration_person_processing_matrix.rsrust/capture/tests/recordings.rsrust/capture/tests/routing_e2e.rs
Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.
…topics Hobby and local dev pull capture:master with compose files that can predate the CAPTURE_OUTPUT_* variables. With only the legacy names set, the new image falls back to its defaults, and two of them differed from what those compose files set: replay would land in events_plugin_ingestion and exceptions in error_tracking_events, which nothing locally consumes. The defaults are now session_recording_snapshot_item_events and ingestion-errortracking-main, so an old compose file routes as before. Production is unaffected: every deployment that reaches these outputs sets them in charts.
OutputTable is generic over how a target names its producer: a ProducerName in config, the producer handle in the Kafka sink. The sink maps names to handles once at construction, so enqueue publishes through the target it resolved, with no second lookup. The Producers wrapper is gone; the sink keeps the distinct producers only to flush them.
Producers are created once by the ProducerRegistry and shared by every output that names them, so flushing them at shutdown is the registry's job. The server now flushes the registry setup hands it, instead of walking the output tree down to the Kafka leaf. The Kafka sink then holds only its output table, each target carrying its producer handle, and flush comes off PublishEvents, Output, OutputRegistry and the Sink trait. S3, print and noop already flushed nothing, and failover only ever reached the Kafka primary.
This branch has not been deployed
1 participant
Problem
Moving one capture output to another Kafka cluster needs a code change. Every v0 output publishes through the one
INGESTIONproducer, and the topics come from sharedKAFKA_*_TOPICvariables. This is Step 10 of the capture outputs plan, and the emergency fallback (Step 17) builds on it.Changes
Destinationis an output with a topic and a named producer:CAPTURE_OUTPUT_<OUTPUT>_TOPICandCAPTURE_OUTPUT_<OUTPUT>_PRODUCER. This is the same shape as Node ingestion'sINGESTION_OUTPUT_<OUTPUT>_{TOPIC,PRODUCER}.ANALYTICS_MAIN,ANALYTICS_OVERFLOW,ANALYTICS_HISTORICAL,SESSION_REPLAY_MAIN,SESSION_REPLAY_OVERFLOW,HEATMAPS,CLIENT_WARNINGS,ERROR_TRACKING,DLQ,AI_MAIN,AI_OVERFLOW. Custom redirects useCAPTURE_OUTPUT_CUSTOM_PRODUCER.KAFKA_TOPIC,KAFKA_*_TOPIC, orCAPTURE_ANALYTICS_AI_EVENTS_*_TOPIC. charts#16337 sets the new names first and must be merged before this deploys.INGESTION. Unknown producer names fail boot.--pull always) and local dev pullcapture:masterwith compose files that can predate these variables. With only the legacy names set, the new image uses the defaults, so they must route like the old compose:SESSION_REPLAY_MAINdefaults tosession_recording_snapshot_item_events, notevents_plugin_ingestion. Otherwise recordings would land in the analytics events topic.ERROR_TRACKINGdefaults toingestion-errortracking-main, noterror_tracking_events. That's the topic the Node consumer reads.KAFKA_TOPICwith analytics. Recordings pods mount only/s, so each deployment'sKAFKA_TOPICmaps to exactly one of the two outputs.TopicTablebecomesOutputTable, generic over the producer: names in config, handles in the sink. The sink maps names to handles once, at construction.PreparedPayloadcarries theDestination, as v1'sPreparedEventdoes. At enqueue, the sink publishes through that target's own producer, with no second lookup.ProducerRegistrydirectly, and flush comes offPublishEvents,Output,OutputRegistryandSink. Only the Kafka leaf ever flushed through that path.Arc<str>, so enqueue allocates once per record, down from two allocations across prep and enqueue before.docker-compose.base.ymlkeeps the legacy names beside the new ones. A stale localcapture:masterimage would otherwise send replay to the analytics default topic. The cleanup PR removes the legacy names.bin/start-rust-servicebuilds from source and switches to the new names outright.How did you test this code?
cargo test -p capture --lib: all pass except 5event_restrictions::repositorytests that need a local Redis.cargo clippy -p capture --tests -- -D warningsandcargo fmtare clean.rust/capture/tests/compile but were not run locally (they need Kafka and Redis). CI runs them. Their assertions are unchanged; only config field names changed.legacy_topic_env_var_is_not_read)producer_name_parses_only_declared_slots,output_producers_default_to_ingestion_and_parse_their_env_var)session_replay_main_does_not_share_analytics_main)each_output_resolves_to_its_own_producer)output_topic_defaults_match_the_local_stack)docker-compose.base.ymlagainst the new defaults. No hobby stack was run.Release status
Automatic notifications
Docs update
None. The outputs plan doc is updated in this PR.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Agent: Claude Code, Claude Opus 5.5