Skip to content

fix(setup-detection): stop filing issues for 5xx and network failures - #108139

Draft
posthog[bot] wants to merge 1 commit into
masterfrom
posthog-self-driving/fixsetup-detection-stop-transient-500s-3e7ffb
Draft

posthog[bot] wants to merge 1 commit into
masterfrom
posthog-self-driving/fixsetup-detection-stop-transient-500s-3e7ffb

Conversation

@posthog

@posthog posthog Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Problem

  • The team gets a new error tracking issue each time a product setup check hits a one-off backend 500. The user still sees the normal page.
  • The shared detectStatus loader throws on every failed detect, so each product and each failure opens its own "A server error occurred." issue. The data warehouse and tracing setup checks each did this.
  • The gate already fails open, so these reports carry no user impact. They are triage noise only.

Origin

  • Error tracking: issue 1, issue 2
  • First signal: 2026-09-25
  • Inbox report: open
  • Likely cause: e44a950
  • Task started by: auto-start, after the report was rated P4 and ready to fix

Changes

  • The shared loader now answers null when detect fails with a 5xx ApiError or a browser network failure. The gate already treats null as "cannot answer". It shows the scene if nothing has answered yet, and it keeps an existing answer.
  • Users see no change, except that the "Detect status failed" toast no longer shows for these failures.
  • Other failures still throw. A dead-scope 404 still stops the poll, and 4xx responses or application bugs still reach error tracking.
  • The fix is in the shared loader, not in each product's detect, so all adopting products get it. It does not touch has_spans, so it does not conflict with #103402.

How did you test this code?

  • Extended the existing "fails open" test in setupDetectionLogic.test.ts with 500, dropped-connection, 400, and application-bug cases. Without the fix, the 500 and dropped-connection cases fail.
  • Ran the jest suites for setupDetectionLogic and all products/*/frontend/emptyState setup logics locally.
  • Not checked: manual testing in a running app.

Release status

  • No feature flag controls this change
  • This change is behind a feature flag and is not available to users
  • This change makes a previously flagged feature available to everyone

Automatic notifications

  • Publish to changelog?

Docs update

None.

🤖 Agent context

Autonomy: Fully autonomous

Agent: PostHog Desktop (Claude Code), claude-opus-5-5

  • Skills invoked: /writing-tests, /writing-pr-descriptions.
  • Duplicate search: #107333 covers detection that hangs, not failed requests. No open PR covers this.
  • Plain 500s stay reportable elsewhere on purpose (shouldReportApiFailure). This change excuses them only under setup detection, where the gate already degrades.

Created with PostHog Desktop from this inbox report.

🤖 Generated with Claude Code

A backend 5xx or a dropped connection under a product setup check filed a new error tracking issue per product, even though the gate already failed open. The shared loader now returns null for these, which the gate treats as "cannot answer". Other failures still throw, so a dead-scope 404 still stops the poll and real defects still report.

Generated-By: PostHog Desktop
Task-Id: 74992ab0-1bc4-41a2-a2f7-ee005e466593
@posthog posthog Bot added the self-driving label Sep 29, 2026
@trunk-io

trunk-io Bot commented Sep 29, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

Contained frontend fix in the setup-detection loader, outside risky territory. It is covered by extended tests, and the helpers it uses exist and match their intended semantics. The existing gate already treats null as fail-open.

Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✓ no deny categories matched
size ✓ 22L, 1F substantive, 35L/2F incl. docs/generated/snapshots — within ceiling
tier ✓ T1-agent / T1b-small (35L, 2F, single-area, fix)
stamphog 2.3.0 .stamphog/policy.yml @ 435d928 · reviewed head 435d928

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

✅ Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

✅ Complexity (TypeScript) — clean

Cyclomatic complexity above the limit in changed typescript files (10 for production files, 15 for test files). Warn only: worth simplifying when you next touch these functions.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

⚠️ Bundle size — 🔺 +137 B (+0.0%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 68.88 MiB · 🔺 +137 B (+0.0%)

No file changed by more than 1000 B.

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1.57 MiB · 22 files no change █████████░ 85.5% of 1.84 MiB
logged-out boot: index + App + bootApp (preloaded by every page, including /login)
src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
3.51 MiB · 629 files no change █████████░ 87.2% of 4.03 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
7.34 MiB · 2,339 files no change █████████░ 88.0% of 8.34 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/layout/navigation-3000/navigationLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/dashboard/dashboardLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/lemon-ui/LemonMarkdown/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/RichContentEditor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/CodeSnippet/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/taxonomy/core-filter-definitions-by-group.json stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/player/sessionRecordingPlayerLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB src/index.tsx
1.3 KiB src/RootErrorBoundary.tsx
912 B ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
854 B src/scenes/ChunkLoadErrorBoundary.tsx
Largest files eagerly shipped from src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
Size File
301.8 KiB ../node_modules/.pnpm/posthog-js@1.434.14_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
216.0 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
100.5 KiB src/lib/api.ts
88.4 KiB src/products.tsx
69.4 KiB src/lib/lemon-ui/icons/icons.tsx
40.1 KiB src/lib/utils/eventUsageLogic.ts
38.7 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
33.9 KiB ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js
28.4 KiB src/scenes/scenes.ts
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
301.8 KiB ../node_modules/.pnpm/posthog-js@1.434.14_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
271.7 KiB src/taxonomy/core-filter-definitions-by-group.json
216.0 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
100.5 KiB src/lib/api.ts
98.5 KiB ../packages/quill/packages/quill/dist/index.js
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
90.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
88.4 KiB src/products.tsx

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

✅ Toolbar bundle — eager 2.16 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric Size Δ vs base Budget
Eager (shipped)
entry + static imports
2.16 MiB · 19 files no change ████░░░░░░ 37.7% of 5.72 MiB
Deferred (lazy) 2.10 MiB · 44 files no change n/a — loads on demand
Loader dist/toolbar.js 1.2 KiB no change █░░░░░░░░░ 6.0% of 19.5 KiB
Largest eagerly-shipped chunks
Size File
800.5 KiB dist/toolbar/toolbar-app-QUJ43CJ4.css
651.6 KiB dist/toolbar/chunk-chunk-ZPCK2O6G.js
259.4 KiB dist/toolbar/chunk-chunk-CV2VU6SQ.js
138.3 KiB dist/toolbar/chunk-chunk-DYPTRYMF.js
131.8 KiB dist/toolbar/chunk-chunk-FDH2IBXT.js
75.2 KiB dist/toolbar/toolbar-app-4HYNQ5KU.js
69.0 KiB dist/toolbar/chunk-chunk-TSAL54PB.js
35.6 KiB dist/toolbar/chunk-chunk-HOKNU4ZL.js
21.0 KiB dist/toolbar/chunk-chunk-Z5ELNJKM.js
6.8 KiB dist/toolbar/chunk-chunk-DV7IWQNF.js

Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile

✅ Dist folder size — 🔺 +4.4 KiB (+0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 946.26 MiB · 🔺 +4.4 KiB (+0.0%)

@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

📝 Walkthrough

Walkthrough

Detection now treats API errors with a defined status of 500 or higher and browser network failures as unanswered results (null). Other errors continue to loader failure handling. Tests verify dispatch outcomes for these cases and confirm that the resulting status remains unknown.

Priority: ⬇️ Low

Merge Risk: 🔵 Low · up to 435d9

An unusual 4xx response can be treated as a temporary outage, so polling may continue when it should stop. The localized fix should be made before merging, but the impact is narrow.

Architecture Summary

Architecture risk: 🟡 Medium · up to 435d9

The change affects 1 system.

Changed systems: frontend

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — frontend (ui) was modified; 2 changed files map to changed impact.

Before / after behavior

  • observed — Modified behavior in frontend/src/lib/components/ProductEmptyState/setupDetectionLogic.test.ts: Replaced the single generic “network down” rejection test with cases for HTTP 500, dropped connections, HTTP 400, and an application error. The test now checks the dispatched outcome, expecting detectStatusSuccess for the 500 and dropped connection and detectStatusFailure for the 400 and application error; the unknown status assertion remains.
  • observed — Modified behavior in frontend/src/lib/components/ProductEmptyState/setupDetectionLogic.ts: The API-error import now includes ApiError and isBrowserNetworkFailure for outage classification.
  • observed — Modified behavior in frontend/src/lib/components/ProductEmptyState/setupDetectionLogic.ts: detectStatus now calls an outage-aware wrapper instead of calling detect directly; qualifying outages return null, while other errors continue to propagate.
  • observed — Modified behavior in frontend/src/lib/components/ProductEmptyState/setupDetectionLogic.ts: Adds detectOrNullOnOutage: API errors with a defined status of at least 500 and browser network failures return null; all other errors are rethrown.

Reliability and maintainability

  • inferred — Risk-relevant change factors for frontend: blast_radius_1; direct_dependents_1
🚥 Pre-merge checks | ✅ 1
✅ Passed checks (1 passed)
Check name Status Explanation
Description check ✅ Passed The description is complete and follows the required structure. It explains the problem, user-visible changes, preserved error behavior, tests, release status, documentation status, agent context, dup…
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (1)
frontend/src/lib/components/ProductEmptyState/setupDetectionLogic.ts-283-283 (1)

283-283: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Restrict browser network matching to statusless errors.

A 4xx ApiError with a browser fetch-failure message can enter this branch. This does not block setup detection: both a null success and a failure set a loading gate to unknown, and neither changes an existing answer. It does bypass the failure handler, so a scope-not-found 404 can keep polling instead of disposing the poll.

Use the status check below. Do not exclude every ApiError, because NetworkError extends ApiError and has no status.

Suggested fix
-            isBrowserNetworkFailure(error)
+            isBrowserNetworkFailure(error) &&
+                (!(error instanceof ApiError) || error.status === undefined)

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: PostHog/posthog/.coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: ed8dab79-1270-42e8-a77f-b9316d7b14e6

📥 Commits

Reviewing files that changed from the base of the PR and between 147ca7b and 435d928.

📒 Files selected for processing (2)
  • frontend/src/lib/components/ProductEmptyState/setupDetectionLogic.test.ts
  • frontend/src/lib/components/ProductEmptyState/setupDetectionLogic.ts

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 7 remain after this review.

@trunk-io

trunk-io Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

Failed Test Failure Summary Logs
Scenes-App/SidePanels SidePanelNotebooks smoke-test The test timed out while waiting for a loading indicator or spinner to disappear. Logs ↗︎

View Full Report ↗︎ ⋅ Docs

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant