Skip to content

README.md

Latest commit

 

History

History
177 lines (137 loc) · 9.83 KB

README.md

File metadata and controls

177 lines (137 loc) · 9.83 KB
Raw

PostHog dashboards-as-code

Part of Growth architecture and operations.

Spec: analytics-foundation-1a-dashboards-as-code.

PostHog is configured via a Public-API-driven sync script — not through the PostHog UI. Every dashboard, insight, and cohort the GTM motion depends on is a JSON file in this directory. The sync tool reconciles JSON ↔ PostHog. Git is the source of truth.

Current Growth dashboards

Dashboard Answers
Growth · Quick overview Which acquisition, docs, demo and public runtime signals are present? Are runtime events missing transport?
Growth · Acquisition and demo engagement Which landing pages, install-dialog actions, forms, docs capabilities and independent demo milestones receive activity?
Growth · Runtime diagnostics Which transports produce runtime instances, requests, stream starts, ends and errors?

These are event counts, not a joined developer conversion funnel. Install command copy attempts are recorded before the clipboard operation; they are not completed npm installs. Form acceptance is observed by the browser, not proof of a unique, qualified or email-authorized contact. Runtime counts include the canonical demo and public, unauthenticated SDK telemetry. Missing and unknown transport are not real adapters. Historical malformed events remain visible.

Install observations, development announcements, activation, enrichment, email eligibility and delivery outcomes remain authoritative in Neon. Use npm run growth:report -- funnel --from <UTC> --to <UTC> and npm run growth:report -- journey --contact <UUID> through the Growth operations guide. No Neon export is added by these dashboards. The untracked legacy package dashboard uses the retired tplane:postinstall event and does not measure the current install flow.

Directory layout

tools/posthog/
├── project.json                    # Nx project (posthog-tools)
├── env.ts                          # zod-validated env parsing
├── client.ts                       # openapi-fetch wrapper
├── schema.ts                       # zod schemas for local JSON
├── sync.ts                         # CLI: plan / apply / writeback
├── report.ts                       # CLI: pull insights → markdown
├── live-quality.ts                 # CLI: sample recent events and validate payload quality
├── *.spec.ts                       # tests
├── types/posthog-api.gen.ts        # generated from PostHog OpenAPI spec
├── scripts/generate-types.ts       # regenerate the above
├── dashboards/*.json               # one JSON per dashboard
├── insights/*.json                 # reusable insight specs
└── cohorts/                        # currently empty; populated post-1A

CLI

All commands wrap nx run posthog-tools:*. Root-package aliases:

npm run posthog:sync       # → nx run posthog-tools:sync:plan
npm run posthog:apply      # → nx run posthog-tools:sync:apply
npm run posthog:report     # → nx run posthog-tools:report
npm run posthog:quality    # → nx run posthog-tools:quality:live
npm run posthog:generate-types  # → regenerate types/posthog-api.gen.ts

Direct Nx invocations work too:

nx run posthog-tools:sync:plan
nx run posthog-tools:sync:apply
nx run posthog-tools:sync:apply --args="--delete-orphans"
nx run posthog-tools:quality:live -- --days 7 --limit-per-event 25
nx run posthog-tools:test
nx run posthog-tools:lint

Auth

Requires a Personal API Key with dashboard:write, insight:write, cohort:write, project:read scopes. Create one at https://us.posthog.com/me/settings#personal-api-keys.

Env vars (see .env.example at repo root):

Variable Purpose
POSTHOG_PERSONAL_API_KEY Write-scoped Personal API Key for local --apply and --report
POSTHOG_PERSONAL_API_KEY_READONLY Read-only Personal API Key for CI --plan and live quality checks
POSTHOG_HOST https://us.i.posthog.com (default) or your region
POSTHOG_PROJECT_ID Numeric project id (visible in PostHog URL)

CI maps POSTHOG_PERSONAL_API_KEY_READONLY into the tool's POSTHOG_PERSONAL_API_KEY environment variable for read-only --plan and live quality checks. Local development continues using the write-scoped POSTHOG_PERSONAL_API_KEY for --apply and --report.

JSON contract

// tools/posthog/dashboards/developer-funnel.json
{
  "slug": "developer-funnel", // local id, stable across syncs
  "posthog_id": null, // assigned on first sync; do not edit
  "name": "Growth · Acquisition and demo engagement",
  "description": "Independent acquisition and demo signals; lifecycle outcomes live in Neon.",
  "tags": ["gtm", "growth", "acquisition"],
  "tiles": [
    { "insight": "pageviews-by-landing" },
    { "insight": "growth-install-intent" }
  ]
}
// Example trend definition
{
  "slug": "install-command-clicks",
  "posthog_id": null,
  "name": "Install command copy attempts",
  "kind": "trends",
  "interval": "day",
  "events": [
    { "event": "marketing:cta_click", "math": "total",
      "properties": [{ "key": "cta_id", "value": "hero_install" }] }
  ]
}

Event names must match docs/gtm/taxonomy.md. The taxonomy.spec.ts test enforces this on every CI run.

Data quality checks

telemetry-contract.ts is the machine-readable event/property contract used by tests and live checks.

  • taxonomy.spec.ts and telemetry-contract.spec.ts guard committed dashboard JSON against undocumented events, unsupported breakdowns, unsupported filters, runtime dashboard coverage drift, and forbidden sensitive runtime fields.
  • npm run posthog:quality -- --days 7 --limit-per-event 25 samples recent live PostHog events and validates observed payloads against the same contract. It exits non-zero for missing required properties or forbidden sensitive properties, and prints warnings for non-contract fields.
  • npm run posthog:quality -- --days 7 --limit-per-event 100 --require-critical-coverage also requires recent samples for critical browser and runtime events. The scheduled PostHog telemetry quality workflow runs this thresholded seven-day check daily and supports manual dispatch.
  • The live workflow requires Actions secrets named POSTHOG_PERSONAL_API_KEY_READONLY and POSTHOG_PROJECT_ID.
  • Counts are bounded samples per event name, not a traffic census. A zero count does not establish healthy collection. Use --days 1 for a current-day comparison without replacing the seven-day audit. Historical malformed events can continue to fail that audit after a fix ships; do not delete them or infer a lifecycle outage from this alone.
  • Public SDK admission uses libs/telemetry/src/shared/ingest.ts at Node capture and website ingest: known events, required runtime transport/browser-chat surface, bounded primitive metadata, and a 16 KiB HTTP body limit. Unknown properties are dropped and malformed known values rejected. This validates shape, not caller identity. Legacy Node stream helpers report unknown when transport is omitted.

Sync semantics

  • --plan — diff against PostHog, no writes. Outputs [create] [update] [orphan] per artifact. CI runs this on every PR that affects posthog-tools.
  • --apply — upsert via PATCH. Re-running preserves object IDs but still writes managed metadata. Membership reconciliation detaches stale tiles from managed dashboards and preserves memberships in unrelated dashboards. Wiring failures make the command fail.
  • --apply --delete-orphans — explicit deletion of remote artifacts that have no local JSON. Never automatic.
  • posthog_id writeback — first successful create writes the assigned PostHog id back into the JSON. Include these IDs with the finished dashboard change.

The weekly report fetches details for repository-managed dashboards and separates additive daily trend series over the last 28 complete UTC days, excluding today's partial bucket. Funnels, unique counts, breakdowns and missing/stale/incomplete results render as Unavailable with a reason; they are never silently converted to zero. Inspect those insights in PostHog.

Renaming an artifact

To rename without losing the PostHog id:

  1. Edit the slug field in the JSON, keeping posthog_id unchanged.
  2. Do not move the file — the file path is the slug source.
  3. npm run posthog:sync will detect this as an update, not a create + orphan.

Regenerating types

PostHog publishes their full Public API as OpenAPI 3 at https://us.posthog.com/api/schema/. We commit the generated TypeScript types to avoid network calls at build time. Refresh quarterly:

npm run posthog:generate-types

Review the diff carefully — field renames in PostHog's API will surface here.

Why dashboards-as-code

  • git blame answers "who changed this metric and why."
  • No clicking through the PostHog UI ("api/cli-first" actually delivers).
  • Reproducible on a fresh PostHog project for staging/test envs.
  • Reviewable in PRs like any other change.
  • taxonomy.spec.ts prevents dashboards from referencing events the taxonomy doesn't document.

See also