Conversation
Sync conversations, end users, knowledge sources, integrations, and activity. Import completed analytics periods and their question and coverage-gap clusters. Use project API keys and resumable full refresh through the shared REST client.
Gilbert09
added
the
stamphog
Talyn App
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
Hey @Gilbert09! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
🦔 Hogbox preview · ✅ ready▶ Open the preview
commit |
![stamphog[bot]](https://avatars.githubusercontent.com/in/3000332?s=60&v=4){kind=small}
There was a problem hiding this comment.
Approved.
This is an additive, alpha-flagged warehouse import source written by an owning-team author with strong familiarity, and it has thorough tests. It follows the established source pattern: the API key is sent only to the fixed kapa.ai host, redirects are off, and the project ID is checked as a UUID before it goes into a URL.
- Author wrote 88% of the modified lines and has 116 merged PRs in these paths (familiarity STRONG).
- copilot-pull-request-reviewer[bot] reviewed the current head.
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 441L, 5F substantive, 1 binary; 778L/10F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1d-complex (778L, 10F, two-areas, feat) |
| stamphog 2.3.1 | .stamphog/policy.yml @ 0edc35c · reviewed head 0edc35c |
🤖 CI report
|
| First copy | Second copy | Lines | Tokens |
|---|---|---|---|
products/warehouse_sources/backend/temporal/data_imports/sources/companycam/source.py:1 |
products/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/source.py:1 |
21 | 159 |
products/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/source.py:1 |
products/warehouse_sources/backend/temporal/data_imports/sources/papersign/source.py:1 |
21 | 157 |
✅ Duplication (TypeScript) — clean
New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.
✅ Bundle size — no change
Uncompressed size of every built .js bundle, compared against the base branch.
Total: 68.97 MiB · no change
No file changed by more than 1000 B.
Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report
✅ Eager graph — within budget
How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.
| Root | Eager (shipped) | Δ vs base | Budget |
|---|---|---|---|
entry (logged-out pages, app bootstrap)src/index.tsx |
1.58 MiB · 22 files | no change | █████████░ 85.8% of 1.84 MiB |
logged-out boot: index + App + bootApp (preloaded by every page, including /login)src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts |
3.52 MiB · 629 files | no change | █████████░ 87.4% of 4.03 MiB |
authenticated shell (every logged-in page)src/scenes/AuthenticatedShell.tsx |
7.35 MiB · 2,339 files | no change | █████████░ 88.1% of 8.34 MiB |
🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/layout/navigation-3000/navigationLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/dashboard/dashboardLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/lemon-ui/LemonMarkdown/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/RichContentEditor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/CodeSnippet/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/taxonomy/core-filter-definitions-by-group.json stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/player/sessionRecordingPlayerLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
Largest files eagerly shipped from src/index.tsx
| Size | File |
|---|---|
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 24.6 KiB | ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js |
| 6.3 KiB | ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js |
| 4.5 KiB | ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js |
| 3.9 KiB | ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js |
| 1.4 KiB | ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js |
| 1.3 KiB | src/index.tsx |
| 1.3 KiB | src/RootErrorBoundary.tsx |
| 912 B | ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js |
| 854 B | src/scenes/ChunkLoadErrorBoundary.tsx |
Largest files eagerly shipped from src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
| Size | File |
|---|---|
| 301.8 KiB | ../node_modules/.pnpm/posthog-js@1.434.14_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs |
| 216.9 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 100.5 KiB | src/lib/api.ts |
| 88.5 KiB | src/products.tsx |
| 69.4 KiB | src/lib/lemon-ui/icons/icons.tsx |
| 40.1 KiB | src/lib/utils/eventUsageLogic.ts |
| 38.7 KiB | ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js |
| 33.9 KiB | ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js |
| 28.4 KiB | src/scenes/scenes.ts |
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
| Size | File |
|---|---|
| 301.8 KiB | ../node_modules/.pnpm/posthog-js@1.434.14_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs |
| 272.3 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 216.9 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 153.7 KiB | ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 100.5 KiB | src/lib/api.ts |
| 98.8 KiB | ../packages/quill/packages/quill/dist/index.js |
| 93.3 KiB | ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js |
| 90.6 KiB | ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js |
| 88.5 KiB | src/products.tsx |
Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479
✅ Toolbar bundle — eager 2.16 MiB within budget
What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.
| Metric | Size | Δ vs base | Budget |
|---|---|---|---|
| Eager (shipped) entry + static imports |
2.16 MiB · 19 files | no change | ████░░░░░░ 37.8% of 5.72 MiB |
| Deferred (lazy) | 2.10 MiB · 44 files | no change | n/a — loads on demand |
Loader dist/toolbar.js |
1.2 KiB | no change | █░░░░░░░░░ 6.0% of 19.5 KiB |
Largest eagerly-shipped chunks
| Size | File |
|---|---|
| 805.5 KiB | dist/toolbar/toolbar-app-ZATDHZQQ.css |
| 651.7 KiB | dist/toolbar/chunk-chunk-7ZORLPSY.js |
| 259.4 KiB | dist/toolbar/chunk-chunk-2FGAEFTI.js |
| 138.3 KiB | dist/toolbar/chunk-chunk-UA3V2PCC.js |
| 131.8 KiB | dist/toolbar/chunk-chunk-FDH2IBXT.js |
| 75.2 KiB | dist/toolbar/toolbar-app-SD4VQXWA.js |
| 69.0 KiB | dist/toolbar/chunk-chunk-TSAL54PB.js |
| 35.6 KiB | dist/toolbar/chunk-chunk-63EWNJRZ.js |
| 21.0 KiB | dist/toolbar/chunk-chunk-F5GDSM5D.js |
| 6.8 KiB | dist/toolbar/chunk-chunk-DV7IWQNF.js |
Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile
✅ Dist folder size — 🔺 +57.8 KiB (+0.0%)
Total size of the built frontend/dist folder (all assets), compared against the base branch.
Total: 947.85 MiB · 🔺 +57.8 KiB (+0.0%)
⚠️ Backend coverage — 99.0% of changed backend lines covered — 3 uncovered
🧪 Backend test coverage
Patch coverage — changed backend lines (products + core): ████████████████████ 99.0% (309 / 312)
| File | Patch | Uncovered changed lines |
|---|---|---|
products/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/source.py |
91.7% | 71, 74 |
products/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/kapa_ai.py |
98.7% | 120 |
🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 242765029253128 -n patch-coverage), or the coverage-data block at the end of this comment.
Per-product line coverage (touched products)
| Product | Coverage | Lines |
|---|---|---|
demo |
███████████░░░░░░░░░ 52.8% |
1,411 / 2,673 |
batch_exports |
████████████████░░░░ 81.2% |
21,526 / 26,502 |
cdp |
██████████████████░░ 88.2% |
4,545 / 5,155 |
mcp_analytics |
██████████████████░░ 89.2% |
5,038 / 5,651 |
product_tours |
██████████████████░░ 89.3% |
1,331 / 1,491 |
dashboards |
██████████████████░░ 89.5% |
6,904 / 7,714 |
data_warehouse |
██████████████████░░ 90.0% |
14,027 / 15,589 |
notebooks |
██████████████████░░ 90.2% |
15,297 / 16,964 |
signals |
██████████████████░░ 90.2% |
57,838 / 64,097 |
cohorts |
██████████████████░░ 90.4% |
8,420 / 9,316 |
streamlit_apps |
██████████████████░░ 90.8% |
2,684 / 2,956 |
managed_warehouse |
██████████████████░░ 91.0% |
10,252 / 11,263 |
tasks |
██████████████████░░ 91.1% |
75,525 / 82,874 |
data_modeling |
██████████████████░░ 91.4% |
10,554 / 11,543 |
exports |
██████████████████░░ 91.6% |
9,680 / 10,562 |
engineering_analytics |
██████████████████░░ 91.7% |
11,032 / 12,030 |
business_knowledge |
██████████████████░░ 92.2% |
7,684 / 8,330 |
conversations |
███████████████████░ 92.5% |
28,734 / 31,062 |
early_access_features |
███████████████████░ 92.6% |
1,332 / 1,439 |
stamphog |
███████████████████░ 92.8% |
8,109 / 8,742 |
canvas |
███████████████████░ 92.8% |
6,873 / 7,405 |
approvals |
███████████████████░ 93.0% |
3,974 / 4,271 |
mcp_registry |
███████████████████░ 93.1% |
1,670 / 1,794 |
notifications |
███████████████████░ 93.2% |
1,145 / 1,229 |
error_tracking |
███████████████████░ 93.2% |
16,359 / 17,547 |
surveys |
███████████████████░ 93.3% |
6,571 / 7,040 |
slack_app |
███████████████████░ 93.4% |
13,995 / 14,989 |
autoresearch |
███████████████████░ 93.6% |
8,481 / 9,061 |
context_layer |
███████████████████░ 93.9% |
3,415 / 3,638 |
web_analytics |
███████████████████░ 94.0% |
21,815 / 23,218 |
alerts |
███████████████████░ 94.0% |
8,570 / 9,114 |
billing_alerts |
███████████████████░ 94.1% |
2,094 / 2,226 |
mcp_store |
███████████████████░ 94.4% |
8,940 / 9,472 |
ai_observability |
███████████████████░ 94.5% |
24,473 / 25,896 |
workflows |
███████████████████░ 94.6% |
15,222 / 16,093 |
wizard |
███████████████████░ 94.7% |
6,150 / 6,496 |
reminders |
███████████████████░ 94.8% |
760 / 802 |
review_hog |
███████████████████░ 95.0% |
11,507 / 12,119 |
endpoints |
███████████████████░ 95.1% |
9,206 / 9,681 |
annotations |
███████████████████░ 95.1% |
817 / 859 |
customer_analytics |
███████████████████░ 95.2% |
25,636 / 26,938 |
marketing_analytics |
███████████████████░ 95.3% |
19,566 / 20,528 |
posthog_ai |
███████████████████░ 95.3% |
2,488 / 2,610 |
experiments |
███████████████████░ 95.4% |
32,457 / 34,020 |
actions |
███████████████████░ 95.5% |
756 / 792 |
logs |
███████████████████░ 95.5% |
15,399 / 16,130 |
data_catalog |
███████████████████░ 95.6% |
4,402 / 4,606 |
tracing |
███████████████████░ 95.6% |
3,536 / 3,699 |
replay_vision |
███████████████████░ 95.6% |
27,675 / 28,939 |
growth |
███████████████████░ 95.7% |
11,228 / 11,734 |
messaging |
███████████████████░ 95.8% |
3,798 / 3,963 |
skills |
███████████████████░ 95.8% |
6,972 / 7,274 |
product_analytics |
███████████████████░ 96.0% |
28,470 / 29,647 |
access_control |
███████████████████░ 96.3% |
7,113 / 7,386 |
revenue_analytics |
███████████████████░ 96.4% |
1,876 / 1,946 |
user_interviews |
███████████████████░ 96.5% |
2,867 / 2,971 |
feature_flags |
███████████████████░ 96.5% |
25,588 / 26,509 |
warehouse_sources |
███████████████████░ 97.3% |
458,756 / 471,611 |
data_quality |
████████████████████ 97.6% |
7,587 / 7,774 |
metrics |
████████████████████ 98.0% |
4,084 / 4,166 |
analytics_platform |
████████████████████ 98.3% |
2,784 / 2,833 |
pulse |
████████████████████ 98.5% |
2,046 / 2,078 |
live_debugger |
████████████████████ 99.2% |
626 / 631 |
Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. 📝 WalkthroughWalkthroughThe change adds Kapa AI endpoint definitions and a source that validates credentials, retrieves data through REST resources, and supports resumable pagination. The source configuration includes API-key and project-ID fields and is marked alpha. Tests cover pagination, resume state, credential validation, and error handling. The source inventory now lists Kapa AI as implemented. Priority: ➖ Normal Merge Risk: 🟡 Moderate · up to A failed finalization on a Kapa AI import can leave a saved completed marker, so the retry imports nothing and the refresh may be skipped or receive empty input. A type-check error in the tests is also failing CI. Fix both before merging. Security Architecture ReviewSecurity architecture risk: 🟡 Moderate · up to The integration adds a new path for importing project data. Its outbound credential controls are supported by the implementation, but safe checkpoint ownership depends on how import jobs are isolated and scheduled; that guarantee remains unverified. Retained concerns
Security review detailsSecurity Blast Radius
Trust Boundaries and Controls
Resilience and Maintainability Implications
Hardening Proposals
🚥 Pre-merge checks | ✅ 1✅ Passed checks (1 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Note
Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.
🟡 Other comments (1)
products/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/tests/test_kapa_ai.py-246-246 (1)
246-246: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winFix the mypy failure on
RESTClient._send_request.retry.CI fails at Line 246 with
Callable has no attribute "retry". Mypy types the tenacity-decorated method as a plainCallable. Add a targeted ignore, or cast the method toAnybeforepatch.object.Proposed fix
- with patch.object(RESTClient._send_request.retry, "wait", return_value=0): + with patch.object(RESTClient._send_request.retry, "wait", return_value=0): # type: ignore[attr-defined]Source: Pipeline failures
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
Review profile: QUIET
Plan: Enterprise
Run ID: 82b8f9e0-e166-48cc-892f-6d8e5f853776
⛔ Files ignored due to path filters (1)
frontend/public/services/kapa_ai.pngis excluded by!**/*.png
📒 Files selected for processing (9)
products/warehouse_sources/backend/temporal/data_imports/sources/SOURCES.mdproducts/warehouse_sources/backend/temporal/data_imports/sources/generated_configs/kapaai.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/canonical_descriptions.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/kapa_ai.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/settings.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/source.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/tests/test_kapa_ai.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/tests/test_kapa_ai_source.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/tests/test_source_catalog_invariants.py
💤 Files with no reviewable changes (1)
- products/warehouse_sources/backend/temporal/data_imports/sources/tests/test_source_catalog_invariants.py
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 3 remain after this review.
| def get_rows() -> Iterator[list[dict[str, Any]]]: | ||
| resume = resumable_source_manager.load_state() if resumable_source_manager.can_resume() else None | ||
| if resume and resume.completed: | ||
| return | ||
| resource = build_resource(api_key, project_id, endpoint, team_id, job_id, resumable_source_manager, resume) | ||
| yield from resource | ||
| resumable_source_manager.save_state(KapaResumeConfig(completed=True)) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
#!/bin/bash
rg -nP -C3 'completed=True|\.completed\b' products/warehouse_sources/backend/temporal/data_imports/sources --type=py -g '!**/tests/**' | head -60
rg -nP -C3 '\bclear_state\s*\(' products/warehouse_sources/backend/temporal/data_imports --type=py -g '!**/tests/**' | head -60Repository: PostHog/posthog
Length of output: 14940
🏁 Script executed:
#!/bin/bash
set -o pipefail
printf '%s\n' '--- Kapa source and diff ---'
sed -n '1,180p' products/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/kapa_ai.py
git diff --unified=25 cbf0c3d7ec8a7504143ddf796aed559bb532a30c9 0edc35c19f53d788edb346269512961074ddb3c9 -- products/warehouse_sources/backend/temporal/data_imports/sources/kapa_ai/kapa_ai.py
printf '%s\n' '--- manager declarations/usages ---'
rg -n -P -C5 'class ResumableSourceManager|def (save_state|load_state|clear_state|can_resume)\b|resumable_source_manager' products/warehouse_sources/backend/temporal/data_imports posthog --type=py | head -240
printf '%s\n' '--- extraction/load orchestration ---'
rg -n -P -C6 'yield from .*get_rows|get_rows\(\)|replace|load.*final|extract.*load|SourceResponse' products/warehouse_sources/backend/temporal/data_imports --type=py | head -260Repository: PostHog/posthog
Length of output: 41483
🏁 Script executed:
#!/bin/bash
set -o pipefail
printf '%s\n' '--- resumable manager files ---'
fd -i 'resumable' products/warehouse_sources/backend/temporal/data_imports
printf '%s\n' '--- manager definitions ---'
rg -n -P -C8 'class ResumableSourceManager|def (save_state|load_state|clear_state|can_resume|commit)\b' products/warehouse_sources/backend/temporal/data_imports --type=py
printf '%s\n' '--- pipeline resume boundary ---'
rg -n -P -C14 'def _commit_resume_state|_commit_resume_state\(|staged.*resume|resume.*staged|release.*batch|held.*batch' products/warehouse_sources/backend/temporal/data_imports/pipelines --type=py
printf '%s\n' '--- row consumption and load failure boundary ---'
rg -n -P -C10 'source_response\.items|resource\.items|items\(\)|for .*batch|async for .*batch|load.*batch|consume.*batch' products/warehouse_sources/backend/temporal/data_imports/pipelines products/warehouse_sources/backend/temporal/data_imports --type=py | head -240Repository: PostHog/posthog
Length of output: 45665
🏁 Script executed:
#!/bin/bash
set -o pipefail
printf '%s\n' '--- resumable manager ---'
cat -n products/warehouse_sources/backend/temporal/data_imports/sources/common/resumable.py
printf '%s\n' '--- pipeline resume methods ---'
rg -n -C10 '_commit_resume_state|resumable_source_manager|resume_state|source_response.items|finalize_run|prepare_run|write_batch' products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/pipeline.py products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/destinations_load --type=py | head -360
printf '%s\n' '--- V2 manager commit path ---'
rg -n -C12 'manager\.commit|resumable_source_manager\.commit|commit\(\)' products/warehouse_sources/backend/temporal/data_imports --type=py | head -220Repository: PostHog/posthog
Length of output: 42345
🏁 Script executed:
#!/bin/bash
set -o pipefail
printf '%s\n' '--- manager implementation ---'
cat -n products/warehouse_sources/backend/temporal/data_imports/sources/common/resumable.py
printf '%s\n' '--- focused resume references ---'
rg -n -C16 'commit_resume|save_state|clear_state|load_state|can_resume|finalize_run' products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/pipeline.py products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/destinations_load products/warehouse_sources/backend/temporal/data_imports/sources/common/resumable.py --type=py | head -500Repository: PostHog/posthog
Length of output: 42173
Clear the resumable state after the walk completes.
save_state stages completed=True, and the pipeline persists it after the rows are written. Finalization can still fail afterward. The retry then reads completed=True and yields no rows, so the replace refresh can be skipped or receive empty input.
Suggested fix
def get_rows() -> Iterator[list[dict[str, Any]]]:
resume = resumable_source_manager.load_state() if resumable_source_manager.can_resume() else None
- if resume and resume.completed:
- return
resource = build_resource(api_key, project_id, endpoint, team_id, job_id, resumable_source_manager, resume)
yield from resource
- resumable_source_manager.save_state(KapaResumeConfig(completed=True))
+ resumable_source_manager.clear_state()📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def get_rows() -> Iterator[list[dict[str, Any]]]: | |
| resume = resumable_source_manager.load_state() if resumable_source_manager.can_resume() else None | |
| if resume and resume.completed: | |
| return | |
| resource = build_resource(api_key, project_id, endpoint, team_id, job_id, resumable_source_manager, resume) | |
| yield from resource | |
| resumable_source_manager.save_state(KapaResumeConfig(completed=True)) | |
| def get_rows() -> Iterator[list[dict[str, Any]]]: | |
| resume = resumable_source_manager.load_state() if resumable_source_manager.can_resume() else None | |
| resource = build_resource(api_key, project_id, endpoint, team_id, job_id, resumable_source_manager, resume) | |
| yield from resource | |
| resumable_source_manager.clear_state() |
Problem
PostHog users can now import their kapa.ai data into the data warehouse.
Changes
X-API-KEYheader.updated_sincefiltering is verified against a live account.How did you test this code?
👉 Stay up-to-date with PostHog coding conventions for a smoother review.
Ran the source and shared source registry suites with
python -m pytest -q -p no:cacheprovider.Nothing was run against a live kapa.ai account. These documentation-based request shapes remain unverified against a live account:
X-API-KEYauthentication.include,sort=asc,page_size, and cursor parameters.nextURLs.Release status
Automatic notifications
Docs update
The posthog.com source page is a follow-up.