feat(warehouse-sources): add dub analytics and partner program tables - #106633
Conversation
Closes the Dub endpoint-coverage gaps recorded in COVERAGE_GAPS_APPENDIX.md.
Adds ten GET /analytics breakdown tables (timeseries, continents, countries,
regions, cities, devices, browsers, os, referers, triggers), partner_applications
from GET /partners/applications, and partner_analytics_timeseries from
GET /partners/analytics.
GET /bounties/{bountyId}/submissions is skipped: the Dub spec exposes no endpoint
that lists bounties, so the parent IDs cannot be enumerated.
Generated-By: PostHog Desktop
Task-Id: 48ddd15f-4520-4053-9858-ce0827d4b607
|
😎 Merged successfully - details. |
Gilbert09
added
the
stamphog
Talyn App
🤖 CI report
|
| First copy | Second copy | Lines | Tokens |
|---|---|---|---|
products/warehouse_sources/backend/temporal/data_imports/sources/cloudbeds/cloudbeds.py:12 |
products/warehouse_sources/backend/temporal/data_imports/sources/dub/dub.py:10 |
11 | 87 |
✅ Duplication (TypeScript) — clean
New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.
🚨 Comment density — 8% of added code lines are comments (31 of 396)
This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out.
Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See .agents/skills/writing-code-comments/SKILL.md for the house rules.
Files with the most added comment lines:
| File | Comment lines | Added lines |
|---|---|---|
products/warehouse_sources/backend/temporal/data_imports/sources/dub/tests/test_dub.py |
14 | 105 |
products/warehouse_sources/backend/temporal/data_imports/sources/dub/settings.py |
11 | 63 |
products/warehouse_sources/backend/temporal/data_imports/sources/dub/dub.py |
6 | 84 |
This check does not block merging. It updates on every push and clears when the share drops.
⚠️ Backend coverage — 96.0% of changed backend lines covered — 4 uncovered
🧪 Backend test coverage
Patch coverage — changed backend lines (products + core): ███████████████████░ 96.0% (113 / 117)
| File | Patch | Uncovered changed lines |
|---|---|---|
products/warehouse_sources/backend/temporal/data_imports/sources/dub/dub.py |
91.8% | 235, 240, 273, 278 |
🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 363392025794264 -n patch-coverage), or the coverage-data block at the end of this comment.
Per-product line coverage (touched products)
| Product | Coverage | Lines |
|---|---|---|
warehouse_sources |
███████████████████░ 97.2% |
442,183 / 454,906 |
Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.
![stamphog[bot]](https://avatars.githubusercontent.com/in/3000332?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. 📝 WalkthroughWalkthroughDub coverage expands from 11 to 23 imported tables, including workspace analytics breakdowns, partner analytics time series, and partner applications. Endpoint configuration adds single-response pagination and composite primary keys. Runtime handling updates request parameters and probe intervals. Tests and coverage documentation reflect the added endpoints and behavior. Priority: ➖ Normal Merge Risk: 🟡 Moderate · up to The new partner analytics table requests data without the partner identifier that Dub requires, so its syncs are likely to fail even for workspaces with a partner program. Several new tables also link to documentation pages that no longer exist. Fix or remove the partner analytics table before merging, and update the documentation links. Security Architecture ReviewSecurity architecture risk: 🟡 Moderate · up to The new tables include potentially sensitive application answers and depend on full-refresh behavior. Existing credential checks and team-scoped storage paths limit the apparent exposure, but data-handling and failure recovery need design-level validation. Retained concerns
Security review detailsSecurity Blast Radius
Trust Boundaries and Controls
Resilience and Maintainability Implications
Hardening Proposals
🚥 Pre-merge checks | ✅ 1✅ Passed checks (1 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Note
Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.
🟡 Other comments (1)
products/warehouse_sources/backend/temporal/data_imports/sources/dub/canonical_descriptions.py-192-192 (1)
192-192: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winUpdate the new Dub API-reference paths. Both paths return 404, so users cannot open the endpoint documentation from these tables. Dub’s documentation index identifies replacement pages. ()
products/warehouse_sources/backend/temporal/data_imports/sources/dub/canonical_descriptions.py#L192-L192: use the current analyticsretrievepage for all new workspace analytics descriptions.products/warehouse_sources/backend/temporal/data_imports/sources/dub/canonical_descriptions.py#L320-L320: use the current partnersapplications/listpage.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
Review profile: QUIET
Plan: Enterprise
Run ID: 5f6e9a6b-e636-46fb-9981-38294e88467d
📒 Files selected for processing (6)
products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdproducts/warehouse_sources/backend/temporal/data_imports/sources/dub/canonical_descriptions.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/dub/dub.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/dub/settings.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/dub/tests/test_dub.pyproducts/warehouse_sources/backend/temporal/data_imports/sources/dub/tests/test_dub_source.py
Included review availability: Your plan provides up to 12 included reviews per hour; 3 remain after this review.
Dub's GET /partners/analytics handler calls throwIfNoPartnerIdOrTenantId before it reads anything, so the endpoint has no program-wide mode and the previous table would have failed on every import. The published OpenAPI spec marks both identifiers optional, which does not match the handler. Build the table by reading the endpoint once per enrolled partner and stamping the partner id onto each row, so the bucket timestamps of two partners no longer collide on the primary key. Generated-By: PostHog Desktop Task-Id: 48ddd15f-4520-4053-9858-ce0827d4b607
A new stamphog review started for this PR — the fresh verdict replaces this approval.
There was a problem hiding this comment.
Approved.
Contained feature addition to a single warehouse-sources connector by a STRONG-familiarity author on the owning team, well covered by new tests; the one unresolved CodeRabbit concern about the partner analytics endpoint is already addressed by the per-partner walk and tests in this diff.
- Author wrote 95% of the modified lines and has 20 merged PRs in these paths (familiarity STRONG).
- CodeRabbit flagged two dead docs_url links in canonical_descriptions.py (cosmetic, not blocking).
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 314L, 3F substantive, 460L/6F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1d-complex (460L, 6F, single-area, feat) |
| stamphog 2.1.0 | .stamphog/policy.yml @ f902fcc · reviewed head f902fcc |
|
/trunk merge |
Problem
/eventsstream needs a Dub Business plan.GET /analyticsdoes not, so for every workspace below that plan it is the only path to any performance numbers at all.COVERAGE_GAPS_APPENDIX.md, not a new Dub release.Changes
groupBycut ofGET /analytics.partner_applicationslists the applications pending approval, with the applicant's profile and their answers to the program's application form.partner_analytics_timeseriesreports each enrolled partner's clicks, leads, sales and earnings over time, one row per partner per interval. No other Dub table carriesearnings./analyticsrecomputes its aggregates per request and exposes no row-level cursor.interval=allandevent=composite. Dub otherwise answers a 24 hour window of clicks only, which returns a well-formed but near-empty table.DubEndpointConfig.primary_keybecomesprimary_keys, and a"single"pagination mode selectsSinglePagePaginatorand drops the page-size param that the aggregate endpoints reject.Endpoints verified but not implemented
GET /bounties/{bountyId}/submissions: the spec exposes no endpoint that lists bounties, and no other response carries abountyId, so there is no way to enumerate the parent IDs to fan out over.top_linksandtop_urlscuts of/analytics:linksalready carries each link's destination URL alongside its lifetime clicks, leads and sales./partners/analytics: the handler callsthrowIfNoPartnerIdOrTenantIdbefore it reads anything, so the endpoint only answers for one named partner. The table walks every enrolled partner instead.How did you test this code?
Automated only. No Dub account was available, so nothing in this PR was run against the live API. Every endpoint path, parameter name, enum value and response shape was checked against Dub's published OpenAPI spec, which is the same reference the coverage audit diffed against.
Three tests added, each for a failure that stays silent:
test_analytics_endpoints_widen_both_dub_defaults— leaving either Dub default in place still yields a well-formed table, just a near-empty one, so no other test would go red.test_aggregate_endpoints_stop_after_one_request— these responses carry no next-page marker, so a paginator that kept asking would re-import the same rows until the run was killed, and a leaked page-size param makes the endpoint 422.test_composite_primary_keys_all_reach_the_source_response— a geo table keyed on its leaf dimension alone merges same-named cities in different countries on every merge.TestPartnerAnalyticsWalk— a request that names no partner is rejected outright, and two partners share bucket timestamps, so unstamped rows collide on the key. Also covers resume and a workspace with no program.Ran locally: the Dub suite and the
sources/invariant suite (categories, versions, schema), plusruffand the.semgrep/rules/security/set over the changed files.Repo-wide
mypy --cache-fine-grained .reports no issues across 21588 source files. The only type-shaped change is theprimary_keytoprimary_keysrename, and a grep found no reference to that field outside the Dub package.Release status
Automatic notifications
Docs update
No repo docs change. The Supported tables section of the Dub docs page renders from
get_documented_tables(), which this PR extends throughcanonical_descriptions.py.🤖 Agent context
Autonomy: Fully autonomous
Agent: Claude Code, Opus 5
dub,bounties,partner applicationsandpartners/analytics, and listed every open PR by the maintainer who authors both the hand-written and the automated warehouse-source PRs. Nothing addresses Dub endpoint coverage./implementing-warehouse-sources,/writing-tests,/writing-code-comments,/writing-pr-descriptions.POSTHOG_TASK_RUN_IDset).partner_analytics_timeseriessent no partner identifier. Dub's published spec marks both identifiers optional, but the handler rejects the request without one, so the table would have failed on every import. Reading Dub's route source confirmed it, and the table now walks each partner.Created with PostHog Desktop