feat(posthog): add per-metric summary reports (#3824)

[sanskar-singh-2403]

Fixes #3824

Describe the changes you have made in this PR -

Adds PostHog analytics per-metric summary reports, mirroring the existing
Sentry morning-digest ("Viktor-style") pattern so the surface, output shape,
and scheduling behaviour stay consistent across integrations.

• posthog-summary skill (integrations/posthog/tools/skills/posthog-summary/SKILL.md):
  pulls metrics via the existing PostHog MCP tools (list_posthog_tools /
  call_posthog_tool) and formats a per-metric report with current vs. previous
  deltas, percent change, and trend direction.
• report_runner (integrations/posthog/report_runner.py): runs one headless
  posthog-summary turn and returns the assistant report text; guards on PostHog
  being configured.
• report_prerequisites (integrations/posthog/report_prerequisites.py):
  PostHog integration + delivery-provider readiness checks.
• CLI (surfaces/cli/commands/posthog_report.py):
  opensre posthog report run (on-demand) and
  opensre posthog report schedule add/list/remove/run (scheduled), registered
  in the CLI command list.
• Scheduler wiring: new POSTHOG_METRIC_REPORT TaskKind, a message builder
  in platform/scheduler/tasks.py (strips credential keys, defaults to a 7d
  window), and a route in integrations/scheduled_agent_bootstrap.py.
• Assistant prompt: a PostHog report output-shape rule so the per-metric
  table format stays deterministic.
• Tests: builder coverage in tests/scheduler/test_tasks.py and runner
  coverage in tests/integrations/posthog/test_report_runner.py.

Acceptance criteria from the issue: connect to PostHog data source ✅,
per-metric summary report ✅, Viktor-style output format ✅, on-demand and
scheduled triggering ✅.

Demo/Screenshot for feature changes and bug fixes -

---

Code Understanding and AI Usage

Did you use AI assistance (ChatGPT, Claude, Copilot, etc.) to write any part of this code?

• No, I wrote all the code myself
• Yes, I used AI assistance (continue below)

If you used AI assistance:

• I have reviewed every single line of the AI-generated code
• I can explain the purpose and logic of each function/component I added
• I have tested edge cases and understand how the code handles them
• I have modified the AI output to follow this project's coding standards and conventions

Explain your implementation approach:

The issue asks for automated per-metric PostHog reports "like Viktor reports."
The codebase already had that exact pattern for Sentry (the morning digest), so
rather than inventing a new mechanism I mirrored it: a bundled skill defines the
report workflow/format, a headless runner dispatches a single agent turn driven
by that skill, and the scheduler multiplexes the task onto the shared
agent-runner seam via a new TaskKind. PostHog connectivity already existed
(integrations/posthog REST config + posthog_mcp tool surface), so no new
transport code was needed — the runner just reuses those tools.

Alternative considered: querying the PostHog REST API directly and templating a
report in Python. I rejected that because it would duplicate query logic the MCP
tool surface already exposes, wouldn't reuse the shared skill/prompt formatting,
and would diverge from the Sentry precedent that reviewers already know.

Key components: run_posthog_report (headless turn + report extraction),
_build_posthog_metric_report (scheduler builder, credential stripping,
default window), the posthog report CLI group (on-demand + schedule
management), and the posthog-summary skill/prompt rule that pin the output
shape to a Metric | Current | Previous | Change | % | Trend table.

---

Checklist before requesting a review

• I have added proper PR title and linked to the issue
• I have performed a self-review of my code
• I can explain the purpose of every function, class, and logic block I added
• I understand why my changes work and have tested them thoroughly
• I have considered potential edge cases and how my code handles them
• If it is a core feature, I have added thorough tests
• My code follows the project's style guidelines and conventions

---

Note: Please check Allow edits from maintainers if you would like us to assist in the PR.

[github-actions]

Greptile code review

This repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md.

Run a review — add a PR comment with:

@greptile review

Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5.

Optional: automate with the greploop skill.

[greptile-apps]

Greptile Summary

This PR adds PostHog per-metric summary reports by mirroring the existing Sentry morning-digest pattern: a bundled posthog-summary skill, a headless runner, a new POSTHOG_METRIC_REPORT TaskKind, and a posthog report CLI group with run and schedule add/list/remove/run subcommands.

• New skill + runner: SKILL.md defines the per-metric table format; report_runner.py dispatches a single headless agent turn and extracts the LLM response.
• Scheduler integration: _build_posthog_metric_report in tasks.py strips credential keys, defaults to a 7-day window, and routes via the shared invoke_agent_runner → run_scheduled_agent_digest → run_posthog_report chain.
• CLI surface: posthog_report.py exposes on-demand and scheduled execution with prerequisite guards (require_posthog_integration, require_report_delivery_provider) and the shared _validate_cron_and_timezone helper.

Confidence Score: 5/5

Safe to merge; the implementation faithfully mirrors the established Sentry digest pattern and all new code paths are covered by tests.

The changes introduce no data loss, auth bypass, or scheduler correctness issues. Credential stripping is applied in the task builder, the PostHog availability guard fires early in both the CLI and the headless dispatch path, and the routing logic in scheduled_agent_bootstrap correctly identifies PostHog payloads. The one ordering quirk in posthog_report_schedule_run is cosmetic and does not affect correctness under normal operation.

surfaces/cli/commands/posthog_report.py — the schedule run subcommand installs runners before validating the task ID, a minor ordering inconsistency worth tidying.

Important Files Changed

Filename	Overview
surfaces/cli/commands/posthog_report.py	New CLI command group; the schedule run subcommand calls _install_scheduler_runners() before validating the task ID, unlike every other guard pattern in the file which validates first.
integrations/posthog/report_runner.py	New headless runner — mirrors Sentry morning_digest_runner; prerequisite check, harness construction, and report extraction all follow the established pattern cleanly.
integrations/posthog/report_prerequisites.py	New shared prerequisites module; centralises POSTHOG_SERVICE_NAMES, stats_period_to_hours, and both guard functions; well-tested and clearly documented.
platform/scheduler/tasks.py	Adds _build_posthog_metric_report; correctly strips credential keys, defaults stats_period, and wires into the existing builder dispatch table.
integrations/scheduled_agent_bootstrap.py	Routes posthog source payloads to run_posthog_report; consistent with existing github_pr and uptime_watch routing patterns.

_{Reviews (2): Last reviewed commit: "feat(posthog): add per-metric summary re..." | Re-trigger Greptile}

[sanskar-singh-2403]

@greptile review

[sanskar-singh-2403]

@muddlebee can you kindly let me know your thoughts on this?

[muddlebee]

There's no demo, need screen recording?

[sanskar-singh-2403]

@muddlebee

Demo -

** Recording 2026-07-20 212111.mp4 - Google Drive
** github.com/user-attachments/files/30195791/posthog_demo_log.txt
Verified end-to-end against a live PostHog project via the hosted PostHog MCP server, with the headless agent turn driven by the posthog-summary skill:

integrations verify posthog_mcp — connection confirmed, event schema fetched
posthog report run --period 7d — runs schema discovery, then one bounded HogQL query per metric, and renders the per-metric table (Metric | Current | Previous | Change | % | Trend)
The project I connected is a fresh PostHog project with no ingested events, so the counts are 0 — this exercises the skill's "genuine no-data vs. query-error" handling (it reports a real zero-ingestion result, not a failure). Happy to seed sample events and re-record with non-zero movement if you'd prefer a data-rich table.

[muddlebee]

thank you will take a look shortly

[Davidson3556]

Nice work . One correctness issue I'd want fixed, left inline, plus two process gaps that have no line to anchor to:

Docs — this adds a user-facing opensre posthog report group with no docs change. docs/posthog.mdx and docs/posthog-mcp.mdx already exist and docs/sentry.mdx documents the Sentry digest, so there's a clear home for it. Needs the docs/docs.json entry too or Mintlify won't surface the page.

[sanskar-singh-2403]

@Davidson3556 I have made all the changes suggested by you, kindly review this and let me know your thoughts.

[Devesh36]

@Davidson3556 can you please check this again

[Devesh36]

lot of ci failurs too

[sanskar-singh-2403]

lot of ci failurs too

hey @Devesh36 can we re-run the pipeline i have fixed the test

[sanskar-singh-2403]

Hey @muddlebee @Davidson3556, just a friendly reminder whenever you get a chance to review this PR. Thanks!

[Davidson3556]

ci is failing.

[Davidson3556]

1. report_prerequisites.py pulls delivery helpers from integrations.sentry.digest_delivery, but that module is provider-generic. A second consumer is the cue to move it somewhere neutral (platform/scheduler/) instead of coupling posthog to sentry.

2. _install_scheduler_runners() is copy-pasted from sentry_digest.py, and the "7d" default is repeated in a few spots.

[sanskar-singh-2403]

@Davidson3556 i added this for python.exe path to be taken as well in case of windows OS

[sanskar-singh-2403]

1. report_prerequisites.py pulls delivery helpers from integrations.sentry.digest_delivery, but that module is provider-generic. A second consumer is the cue to move it somewhere neutral (platform/scheduler/) instead of coupling posthog to sentry.
2. _install_scheduler_runners() is copy-pasted from sentry_digest.py, and the "7d" default is repeated in a few spots.

Done both addressed in the latest push.

1. Moved the provider-generic delivery helpers out of sentry/ into platform/scheduler/delivery.py; both the sentry and posthog prerequisites now import downward from there instead of posthog→sentry. Old integrations/sentry/digest_delivery.py deleted, test moved to tests/platform/scheduler/.

2. Extracted the duplicated _install_scheduler_runners() into one shared install_scheduler_runners(), and replaced the repeated "7d" with a DEFAULT_POSTHOG_PERIOD constant.

Tested locally lint-imports --strict passes (1 kept, 0 broken) and the affected suites are green. The only local failure is tests/config/test_runtime_metadata_perf.py::test_baseline_stability, which is unrelated to this change: it's a wall-clock threshold test (asserts build_runtime_metadata runs in <5ms) and it blows the budget only because I'm running under pytest -n auto on a WSL2 box where CPU contention across workers pushes it to ~240ms. It should pass on CI's dedicated runner 🤞 .

[sanskar-singh-2403]

Hey @Davidson3556 can we re-run the CI? if you are ok with the changes? Thanks!