chore(warehouse-sources): remove the legacy cdc conversion

[danielcarletti]

Problem

• Anyone who works on CDC capture still reads, tests and reasons about roughly 600 lines that no running source can reach.
• feat(warehouse-sources): remove the legacy cdc lane #105912 removed the legacy CDC lane and left cdc/legacy_conversion.py to convert each source's leftover state on its next capture run.
• Every source that captures has converted. The sources still on the legacy mode cannot capture, because their slot is gone or their connection fails.
• Most of those come back through Repair CDC, which resets every table and saves the source as buffered.
• A few are stopped without a broken marker. They would come back through Resume CDC, which is the one path that still needs the conversion.

Changes

Note

Precondition, done on 2026-10-02 in both regions: the legacy sources that were stopped without a broken marker are now marked broken, by a one-off operation outside this PR. A read-only check afterwards showed a broken marker on every source that still reads as legacy. A legacy source left unmarked would, after Resume CDC, capture into a buffer that its paused table schedules never read.

• Deletes cdc/legacy_conversion.py and the call capture made to it before every read.
• Nothing reads cdc_ingest_mode anymore. The table sync no longer waits for a conversion, the re-snapshot gate no longer checks the mode, and CDCConfig.ingest_mode is gone.
• The buffer expiry check loses its 14-day exemption for tables stamped at conversion. Production logs show that every stamped table has completed a buffer read since.
• Two cdc_deferred_runs guards go. They held a table back for the reset that the conversion staged, so without the conversion they would keep such a table out of the buffer for good.
• The tables that still hold deferred runs have their sync off or sit on a broken source. Every way back for them, re-enabling the table or Repair CDC, resets the table.
• Still written: CDC setup and Repair CDC keep saving cdc_ingest_mode: buffered, and the API keeps the key on a PATCH. A rollback to the previous release would read a source without the key as legacy and empty its unconsumed buffer. A later PR can drop the writes.
• The lag sweep no longer replaces a broken marker with the self-managed lag marker. That marker allows Resume CDC and clears itself once the lag drops. On a self-managed source it could lift a stop that only Repair CDC may lift, the legacy marker included.
• Side effect: _close_stranded_capture_jobs goes with the module. It closed job rows that a legacy capture run left Running, up to 14 days old. The last legacy capture run was on 2026-09-30, and the sources that captured since have run it.
• Side effect: the resync and the table sync no longer parse the ingest mode, so they no longer raise CDCJobInputsUnreadableError for unreadable job_inputs. Capture still does, through the CDC config.
• Rejected alternative: convert the stopped sources offline. The conversion rebuilds the table schedules, which would then run against an empty buffer until the customer fixes the connection. Skipping that step leaves the tables paused after Resume CDC.

How did you test this code?

• No new tests. The tests of the deleted module go, with the cases that described legacy states: an unconverted source, the conversion stamp, and deferred runs handed to a reset.
• test_patch_cdc_table_mode_adding_target_triggers_resnapshot keeps its stale cdc_deferred_runs case. A reset still removes that key, and the table now stays in the buffer.
• test_critical_lag_self_managed_marks_broken_without_drop_or_pause gains two cases. A table that already holds another marker keeps it through a critical-lag sweep, and a marker on a table whose sync is off does not stop the sweep from marking the tables that sync. Each fails without its part of the guard.
• Run locally: the CDC, Postgres source, pipeline sync and warehouse API suites, and repo-wide mypy.
• Not run: this branch on a devbox.

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Release status

• No feature flag controls this change
• This change is behind a feature flag and is not available to users
• This change makes a previously flagged feature available to everyone

Automatic notifications

• Publish to changelog?

Docs update

Rewrote "Leftover legacy state" in docs/internal/cdc-buffered-ingress-runbook.md: legacy sources come back through Repair CDC only, and the two leftover keys do nothing.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5

• Follow-up to feat(warehouse-sources): remove the legacy cdc lane #105912. A read-only check in both regions listed the sources whose ingest mode still read as legacy, and each one was stopped: most with a broken marker, a few without.
• The decision was to make Repair CDC the only way back for a legacy source, by marking the unmarked ones, and then delete the conversion instead of keeping it for sources nobody may ever resume.
• Duplicate search (legacy cdc conversion, cdc_ingest_mode): no open PR.
• Skills: /writing-tests, /writing-code-comments, /reviewing-with-coderabbit, /writing-pr-descriptions.
• CodeRabbit CLI: no findings.
• A fresh-eyes review found one blocker, two should-fix items and three nits:
  • Blocker, fixed: the self-managed lag sweep could replace the legacy marker and reopen Resume CDC. The guard in the second commit closes it.
  • Not applicable: a table sync could replay legacy copies from a legacy source's buffer. The legacy lane wrote buffer copies only under a retired validation flag, and no source that is still legacy was under it.
  • Checked in production data: every table stamped at conversion has completed buffer reads since, so removing the stamp's exemption forces no re-snapshot.
  • Nits, fixed: an always-true gate in the ad hoc sync, two duplicate test cases, leftover test fixtures and one runbook sentence.
  • Nit, left for the follow-up that drops the cdc_ingest_mode writes: the clear_deferred_runs flag now only decides whether a dead key is removed.
• Review bots after opening:
  • Fixed: the comments on the places that keep cdc_ingest_mode described an earlier release. They now state the rollback constraint.
  • Fixed: the sweep's marker check counted tables whose sync is off, which mark_cdc_broken never marks. It now looks at the same tables.

🤖 Generated with Claude Code

[trunk-io]

😎 Merged successfully - details.

[github-actions]

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

🚨 Comment density — 9% of added code lines are comments (9 of 98)

This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out.

Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See .agents/skills/writing-code-comments/SKILL.md for the house rules.

Files with the most added comment lines:

File	Comment lines	Added lines
products/warehouse_sources/backend/temporal/data_imports/sources/postgres/cdc/adapter.py	3	3
products/warehouse_sources/backend/ad_hoc_sync.py	2	3
products/warehouse_sources/backend/presentation/views/external_data_source/helpers.py	2	2
products/warehouse_sources/backend/temporal/data_imports/cdc/activities.py	2	16

This check does not block merging. It updates on every push and clears when the share drops.

[greptile-apps]

[Medium risk] Removes legacy CDC data ingestion pathway and conversion logic.

Satisfy the repository’s code-comment requirement before merging; no blocking CDC behavior failure was established.

_{Reviews (1) · Last reviewed commit: "chore(warehouse-sources): remove the leg..."}

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

🧰 Additional context used

📚 Code guidelines (8)

.agents/security.md — configured
.agents/skills/sending-notifications/SKILL.md — configured
docs/published/handbook/engineering/type-system.md — configured
.agents/skills/writing-tests/SKILL.md — configured
docs/internal/person-data-access.md — configured
.agents/skills/adopting-generated-api-types/SKILL.md — configured
.claude/commands/conventions.md — configured
.agents/skills/writing-code-comments/SKILL.md — configured

📝 Walkthrough

Walkthrough

The CDC runtime no longer converts legacy state or gates buffered processing on cdc_ingest_mode. Streaming tables can qualify for buffer snapshots without requiring the source to be marked buffered or deferred runs to be absent. The unread-buffer check no longer uses a legacy-conversion timestamp. The runbook and code comments describe remaining metadata and rollback behavior.

Priority: ➖ Normal

Merge Risk: 🟡 Moderate · up to 9f18b

Resolve the marker race and prevent unrepaired legacy tables from consuming buffered rows before merging. Otherwise, a broken reason can be lost or legacy rows delivered twice.

Security Architecture Review

Security architecture risk: 🟡 Moderate · up to 9f18b

The retirement changes how stopped sources recover and how old change data is handled. A failed pause or an already-running sync can bypass the intended full repair, potentially undermining data integrity. Existing ownership controls remain, but the production rollout preconditions were not independently verified.

Retained concerns

• Medium · reliability · inferred: Retirement removes the legacy consumer gate and pre-read conversion without making repair-required broken markers enforceable at capture and consumption boundaries. A failed schedule pause or already-running sync can therefore execute marked legacy state without Repair's purge and reset, weakening recovery containment and potentially replaying or mishandling residual changes. Public Resume refusal and unread-buffer expiry are important countercontrols, but neither universally blocks these execution paths. The affected production state and any resulting data damage remain unverified.

Security review details

Security Blast Radius

• inferred — The supported failure scenario affects a marked legacy source's active schemas, their buffers, and output tables. The implementation is shared across PostgreSQL-compatible source types, but the inspected selection and marker writes target individual sources and schemas rather than granting fleet-wide authority to a source owner.

Trust Boundaries and Controls

• observed — The deployed source viewset inherits team/organization and access-control mixins, and Repair and Resume obtain the source through get_object. Marker writes lock the identified source with its team ID and merge schema state using team-scoped updates. These inspected controls do not turn a broken marker into a capture or consumption execution gate.

Resilience and Maintainability Implications

• inferred — The critical-lag preservation check is outside the marker-write transaction, so a competing marker can still be overwritten between the query and locked write. Base handling marked unconditionally, making this a residual concurrency limitation rather than a separately demonstrated PR regression. It matters because marker reasons distinguish recoverable lag from Repair-only recovery.

Hardening Proposals

• proposed — Enforce repair-required markers at capture and consumption boundaries while preserving self-managed lag recovery. Validate stopped and in-flight execution states, not only marker presence, before relying on the retirement precondition.

🚥 Pre-merge checks | ✅ 1

✅ Passed checks (1 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description is complete and self-contained. It includes the problem, user-visible and mechanical changes, testing scope and limitations, release status, documentation impact, agent context, duplic…

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

---

_{Comment @coderabbitai help to get the list of available commands.}

[parameterai]

Risk: No findings

This delta tightens the self-managed lag-sweep guard (broken_for_another_reason) to match exactly the schema set mark_cdc_broken would mark, updates its tests, and simplifies PostgresSource.resume_covers_run to grant the resumable retry budget to all full loads. No security surface is touched: the guard covers the full marker-overwrite path, the recovery check keeps the source in ERROR while any legacy marker remains, and the retry-budget change only affects Temporal retry policy selection.

_{Sentinel reviewed 9f18b5b · Review settings}

[trunk-io]

   

View Full Report ↗︎ ⋅ Docs

[coderabbitai]

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (1)

products/warehouse_sources/backend/temporal/data_imports/cdc/activities.py-1699-1699 (1)

1699-1699: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Filter inactive schemas from broken_for_another_reason.

broken_for_another_reason scans disabled and deleted schemas, but mark_cdc_broken updates only enabled, non-deleted CDC schemas. Disabling a schema does not clear its cdc_broken marker. Therefore, a marker on an inactive schema can suppress marking an active schema at critical lag.

🐛 Suggested fix

-        ExternalDataSchema.objects.filter(team_id=source.team_id, source=source, sync_type_config__has_key="cdc_broken")
+        ExternalDataSchema.objects.filter(
+            team_id=source.team_id,
+            source=source,
+            sync_type=ExternalDataSchema.SyncType.CDC,
+            should_sync=True,
+            sync_type_config__has_key="cdc_broken",
+        )
+        .exclude(deleted=True)
         .exclude(sync_type_config__cdc_broken__reason=reason)
         .exists()

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: PostHog/posthog/.coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: cab1e941-bfd1-4205-a4aa-d1d78eb45a5d

📥 Commits

Reviewing files that changed from the base of the PR and between a18fbfc and dca4e70.

📒 Files selected for processing (10)

• docs/internal/cdc-buffered-ingress-runbook.md
• products/warehouse_sources/backend/ad_hoc_sync.py
• products/warehouse_sources/backend/temporal/data_imports/cdc/activities.py
• products/warehouse_sources/backend/temporal/data_imports/cdc/broken.py
• products/warehouse_sources/backend/temporal/data_imports/cdc/tests/test_cleanup_orphan_slots.py
• products/warehouse_sources/backend/temporal/data_imports/cdc/tests/test_extract_activity.py
• products/warehouse_sources/backend/temporal/data_imports/cdc/tests/test_source_manager.py
• products/warehouse_sources/backend/temporal/data_imports/pipelines/test_pipeline_sync.py
• products/warehouse_sources/backend/tests/api/test_cdc_table_mode_patch.py
• products/warehouse_sources/backend/tests/test_stalled_schedules.py

💤 Files with no reviewable changes (2)

• products/warehouse_sources/backend/temporal/data_imports/cdc/tests/test_source_manager.py
• products/warehouse_sources/backend/temporal/data_imports/pipelines/test_pipeline_sync.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.

A new stamphog review started for this PR — the fresh verdict replaces this approval.

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

🟠 Major · Restore the legacy conversion before buffered consumption. · source.py:1894-1900

products/warehouse_sources/backend/temporal/data_imports/sources/postgres/source.py:1894-1900
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Restore the legacy conversion before buffered consumption.

source_for_pipeline routes every seeded streaming CDC schema here, regardless of cdc_ingest_mode. The PR also removes the conversion that purged the legacy buffer and marked the source as buffered. A persisted legacy source can therefore read buffer rows that the legacy lane already delivered, which can load CDC changes twice.

Restore the conversion gate and perform the purge, schedule handoff, and buffered-state update before this branch can read a legacy source.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: PostHog/posthog/.coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: d3348da7-81ba-492a-ba36-7696b83d3b54

📥 Commits

Reviewing files that changed from the base of the PR and between dca4e70 and 66f2f27.

📒 Files selected for processing (2)

• products/warehouse_sources/backend/presentation/views/external_data_source/helpers.py
• products/warehouse_sources/backend/temporal/data_imports/sources/postgres/cdc/adapter.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

A new stamphog review started for this PR — the fresh verdict replaces this approval.

[stamphog]

Approved.

This is a removal of dead legacy CDC conversion code by an author on the owning team with STRONG familiarity. No dangling references to the removed symbols remain, and the earlier CodeRabbit inline concern about inactive schemas is already handled in the current helper. The one real risk is a double-load of a still-legacy source if the out-of-PR "mark broken" precondition was not actually done, which I could not verify from the diff.

• Author wrote 98% of the modified lines and has 30 merged PRs in these paths (familiarity STRONG).
• 👍 on the PR from greptile-apps[bot].
• Safety depends on the one-off operation, done outside this PR, that marked every still-legacy source as broken. The diff cannot confirm it.
• The CodeRabbit comment on an older commit says a persisted legacy source could re-read buffer rows the legacy lane already delivered. The author's argument for why this cannot happen rests on production checks, not on code in the diff.

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✓	no deny categories matched
size	✓	361L, 11F substantive, 774L/24F incl. docs/generated/snapshots — within ceiling
tier	✓	T1-agent / T1d-complex (774L, 24F, two-areas, chore)
stamphog 2.3.1		.stamphog/policy.yml @ 9f18b5b · reviewed head 9f18b5b

[danielcarletti]

/trunk merge

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

🟠 Major · Skip buffered consumption for legacy_lane_retired schemas. · source.py:1893-1899

products/warehouse_sources/backend/temporal/data_imports/sources/postgres/source.py:1893-1899
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Skip buffered consumption for legacy_lane_retired schemas.

A persisted cdc_broken marker can remain while a table schedule or run is active. _buffered_cdc_source() can then consume rows already delivered by the legacy lane and merge them again. Check the marker before buffered consumption. This preserves mode-less buffered routing because it does not inspect or parse job_inputs. repair_cdc clears the marker and restores operation.

Suggested fix

+        cdc_broken = (schema.sync_type_config or {}).get("cdc_broken") or {}
+        if cdc_broken.get("reason") == "legacy_lane_retired":
+            return no_op_tick()
+
         # Defense in depth for the v3-forcing invariant: a run that resolved its pipeline version

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: PostHog/posthog/.coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: 86a01e3d-2416-4a32-983a-042aa43083ae

📥 Commits

Reviewing files that changed from the base of the PR and between 66f2f27 and 9f18b5b.

📒 Files selected for processing (3)

• products/warehouse_sources/backend/temporal/data_imports/cdc/broken.py
• products/warehouse_sources/backend/temporal/data_imports/cdc/tests/test_cleanup_orphan_slots.py
• products/warehouse_sources/backend/temporal/data_imports/sources/postgres/source.py

💤 Files with no reviewable changes (1)

• products/warehouse_sources/backend/temporal/data_imports/sources/postgres/source.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

[danielcarletti]

Merge-queue failure: both failures are unrelated flakes

The queue removed this PR after Django Tests Pass failed on the bisection PR #110961 (72601a4b, this branch merged with master 6d5d628a). I read that job's log rather than this branch's checks. It reports exactly two failing tests, and neither is in a file this PR adds or edits — the diff is confined to products/warehouse_sources/** and docs/internal/cdc-buffered-ingress-runbook.md:

Test	Failure	Verdict
posthog/api/test/test_ingestion_warnings_v2.py::TestIngestionWarningsV2API::test_time_range_bounds_results	assert 3 == 4	Known repo-wide flake
products/growth/backend/tests/test_enrichment_tools.py::test_a_paced_call_waits_for_the_budget_and_retries_a_denial_0_web_search	Expected: sleep(1.5) Actual: sleep(1.4999999999999716)	Real-clock float flake

Evidence for the first one: the identical failure (assert 3 == 4, same test) hit #110795 and #110804 in the same window, both of which likewise leave that file untouched — see #110795 and #110804. The test inserts four message_size_too_large rows into ClickHouse in setUp and asserts the -7d window counts all four, so a count of 3 is an insert-visibility race in the fixture, not a behaviour change.

The second is a float-equality assertion against a wall clock: sleep_mock.assert_called_with(1.5) compares exactly, and the pacer derives the wait from time.monotonic arithmetic, so it lands a few hundred attoseconds low under load.

No code change on this branch: the root causes sit in posthog/api/test/ and products/growth/, owned by other teams and in other CI lanes. Pulling either fix in here would add unrelated files to a deletion PR's diff. The failure is a merge-queue coincidence, not a semantic conflict between this branch and master.

State of the PR right now:

• CI on the head commit 9f18b5b is green — 175 success, 126 skipped, 0 failures.
• All review threads are resolved; the last open one (CodeRabbit on the lag-sweep marker check) is answered in-thread.
• mergeable: true, no conflicts, not behind master, stamphog approval in place.

What's needed from a human: re-enter the merge queue (check the Trunk box above or comment /trunk merge). I did not re-queue it myself, since that command merges to master.

_{🦉 via talyn.dev}

[danielcarletti]

Correction to the comment above: one evidence link was wrong (comment editing is not available to me, hence this follow-up). The two PRs that hit the identical test_time_range_bounds_results / assert 3 == 4 failure in the same window are:

• feat(data-warehouse): implement the aws compute optimizer import source #110795 — feat(data-warehouse): implement the aws compute optimizer import source #110795 (comment)
• feat(data-warehouse): implement the aws sagemaker import source #110804 — feat(data-warehouse): implement the aws sagemaker import source #110804 (comment)

Everything else in that comment stands.

_{🦉 via talyn.dev}

[danielcarletti]

/trunk merge

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-10-02 18:03 UTC	Run
prod-us	✅ Deployed	2026-10-02 18:16 UTC	Run
prod-eu	✅ Deployed	2026-10-02 18:16 UTC	Run