feat(warehouse_sources): add hetzner metrics, pricing and zones tables

[Gilbert09]

Problem

• Hetzner Cloud users can sync their servers and load balancers, but not how those resources perform, what they cost, or their DNS zones.
• Why: the endpoint coverage audit listed these as long-standing gaps in the Hetzner source.

Changes

• Four new tables appear in the Hetzner source's schema picker: server_metrics, load_balancer_metrics, pricing and zones.
• server_metrics and load_balancer_metrics hold one row per sample: resource id, series name (for example cpu or bandwidth.in), timestamp and value.
  • The source lists servers or load balancers, then reads each one's metrics in windows over the API's 30-day retention.
  • A fixed 5-minute step keeps every run on the same timestamp grid, so a re-read sample merges onto its row.
  • Each window stays under the API's cap of 500 samples per series.
  • The tables support incremental sync on timestamp, because start/end is a real server-side filter. They are merge-only, because each run re-reads the newest sample.
  • The tables use sort_mode="desc", because rows are ordered per resource but not across resources. The watermark then commits once, at the end of a successful run, and not after one resource's newest sample.
  • A resource deleted between the list call and its metrics call returns 404, and the source skips it.
• pricing is a single row keyed by currency. It uses a single-page paginator, because /pricing has no pagination metadata and ignores page.
• zones is a plain list table, like the existing ones (id:asc, partitioned on created).
• Mechanical: canonical column descriptions for the four tables, and ticked boxes in COVERAGE_GAPS_APPENDIX.md.

Verified against the Cloud API spec: all four endpoints exist on v1, the version the source calls.

Note

The audit says /pricing is "the only way to cost the servers we already sync". That overstates it: server_types and load_balancer_types already carry per-location prices. The new table adds the other prices (volumes, IPs, images, backups), plus the currency and VAT rate.

Not done: GET /zones/{id_or_name}/rrsets, which is in the appendix but was not part of this request. Its box stays open.

How did you test this code?

Unit tests only. Nothing ran against a live Hetzner account, because the sandbox has no Hetzner token.

Test rationale:

• TestMetrics in test_hetzner.py catches four regressions: wrong flattening of the series map into rows, gaps or overlaps between retention windows, an incremental run that ignores the watermark, and a deleted resource that aborts the sync.
• test_pricing_is_one_row_from_one_request catches a regression to the page-number paginator, which would request /pricing forever.
• test_hetzner_source.py now checks that only the metrics tables advertise incremental sync, replacing the "all endpoints are full refresh" test. It also checks that source_for_pipeline routes metrics schemas to the metrics transport.

Release status

• No feature flag controls this change
• This change is behind a feature flag and is not available to users
• This change makes a previously flagged feature available to everyone

Automatic notifications

• Publish to changelog?

Docs update

None in this repo. The posthog.com Hetzner doc builds its table list from the public source config API, so the new tables appear there on their own.

🤖 Agent context

Autonomy: Fully autonomous

Agent: Claude Code (PostHog Desktop cloud task), Claude Opus 5.5 (claude-opus-5-5)

• Skills invoked: /implementing-warehouse-sources, /writing-pr-descriptions.
• No duplicate: searches for open PRs on Hetzner, the module path and each endpoint name found nothing. The maintainer's open PRs contain no Hetzner work.
• Public artifact: the PR holds no session material. All test data is invented.

---

🤖 Generated with Claude Code

Created with PostHog Desktop

[trunk-io]

😎 Merged successfully - details.

[parameterai]

Risk: No findings

This increment replaces the plain tuple yielded by _metric_windows with a frozen MetricsWindow dataclass and updates the call site to read window.start/window.end. It is a behavior-neutral refactor with no new inputs, outputs, or trust boundaries.

_{Sentinel reviewed 4386189 · Review settings}

[github-actions]

🤖 CI report

⚠️ Trunk lane — backend Python lane (py:product:warehouse_sources)

This PR is assigned to the backend Python lane (py:product:warehouse_sources). It runs backend Python tests and may merge in parallel with PRs in other lanes.

⚠️ Duplication (Python) — 1 new duplicated block (worst 121 tokens)

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

First copy	Second copy	Lines	Tokens
products/warehouse_sources/backend/temporal/data_imports/sources/descope/descope.py:8	products/warehouse_sources/backend/temporal/data_imports/sources/hetzner/hetzner.py:9	12	121

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

🚨 Comment density — 8% of added code lines are comments (36 of 438)

This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out.

Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See .agents/skills/writing-code-comments/SKILL.md for the house rules.

Files with the most added comment lines:

File	Comment lines	Added lines
products/warehouse_sources/backend/temporal/data_imports/sources/hetzner/hetzner.py	16	151
products/warehouse_sources/backend/temporal/data_imports/sources/hetzner/settings.py	9	57
products/warehouse_sources/backend/temporal/data_imports/sources/hetzner/source.py	4	17
products/warehouse_sources/backend/temporal/data_imports/sources/hetzner/tests/test_hetzner.py	4	139
products/warehouse_sources/backend/temporal/data_imports/sources/hetzner/tests/test_hetzner_source.py	3	22

This check does not block merging. It updates on every push and clears when the share drops.

⚠️ Backend coverage — 96.0% of changed backend lines covered — 6 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ███████████████████░ 96.0% (185 / 191)

File	Patch	Uncovered changed lines
products/warehouse_sources/backend/temporal/data_imports/sources/hetzner/hetzner.py	92.4%	162–163, 167, 198–199, 264

🤖 Agents: add a test only if an uncovered line exposes a realistic regression that existing tests miss. Otherwise explain why no new test is needed under "How did you test this code?". Gap list: the patch-coverage artifact on this run (gh run download 47789257735369 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)

Product	Coverage	Lines
warehouse_sources	███████████████████░ 97.3%	469,840 / 482,775

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

[Gilbert09]

/trunk merge

A new stamphog review started for this PR — the fresh verdict replaces this approval.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

🧰 Additional context used

📚 Code guidelines (7)

.agents/security.md — configured
.agents/skills/sending-notifications/SKILL.md — configured
docs/published/handbook/engineering/type-system.md — configured
docs/internal/person-data-access.md — configured
.agents/skills/adopting-generated-api-types/SKILL.md — configured
.claude/commands/conventions.md — configured
.agents/skills/writing-code-comments/SKILL.md — configured

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: PostHog/posthog/.coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: ee497260-f4a9-4d66-b2ae-cc84199a85cf

📥 Commits

Reviewing files that changed from the base of the PR and between b370acd and 4386189.

📒 Files selected for processing (1)

• products/warehouse_sources/backend/temporal/data_imports/sources/hetzner/hetzner.py

Limit details: You’ve used all 12 included reviews currently available.

---

📝 Walkthrough

Walkthrough

The Hetzner source adds zone and pricing endpoints and adds server and load-balancer metrics extraction. Metrics requests use bounded five-minute windows and produce flattened rows. Metrics schemas support incremental sync and route to the metrics source. The coverage checklist marks pricing, zones, and both metrics endpoints complete; zone RRsets remains incomplete.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 43861

Metrics sync requests now time out rather than waiting indefinitely, and no actionable merge-blocking risk remains in the reviewed change.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 43861

The new requests retain the existing fixed provider address, bearer authentication, and redirect blocking. No introduced security vulnerability was established. Remaining uncertainty concerns end-to-end recovery, concurrent ingestion, and shared pagination behavior.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The new extraction increases data read from the connected Hetzner project, without evidence of broader provider privileges. Source configuration describes a read token scoped to one project. Shared REST changes have wider potential consumer scope than the Hetzner extractor alone; their complete consumer exposure was not traced.

Trust Boundaries and Controls

• inferred — The examined entrypoint does not provide an arbitrary credentialed destination: schema selection is static, resource IDs originate in authenticated provider responses, and requests use fixed parent paths with redirects disabled. Resource-ID validation was not independently established, but no caller-controlled URL authority was found in this flow.

Resilience and Maintainability Implications

• inferred — Per-window sample limits, finite request timeouts, stable row identity, and deferred watermark advancement support bounded failure recovery. Total work still scales with project resources and time windows; per-request bounds do not establish a whole-run resource budget or complete recovery guarantee.

🚥 Pre-merge checks | ✅ 1

✅ Passed checks (1 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description is complete and follows the repository template. It explains the problem, user-visible changes, testing scope and rationale, release status, documentation impact, and agent context. It…

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

Usage-based review receipt

• Mode: Continue automatically
• Reviewed files: 1
• Waived: $0.25 (charged $0.00)
• View usage details

Note

This review exceeded your plan’s limits and used usage-based reviews—free during trial. After your trial, your Enterprise plan’s existing billing terms apply. Manage usage-based reviews.

---

_{Comment @coderabbitai help to get the list of available commands.}

A new stamphog review started for this PR — the fresh verdict replaces this approval.

[stamphog]

Approved.

This is an additive Hetzner warehouse-source change, outside risky territory. The author is on the owning team with STRONG familiarity, tests cover the new paths, and the helper APIs it relies on (merge_only, request_timeout) exist.

• Author wrote 100% of the modified lines and has 50 merged PRs in these paths (familiarity STRONG).

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✓	no deny categories matched
size	✓	330L, 4F substantive, 540L/7F incl. docs/generated/snapshots — within ceiling
tier	✓	T1-agent / T1d-complex (540L, 7F, single-area, feat)
stamphog 2.3.1		.stamphog/policy.yml @ 4386189 · reviewed head 4386189

[trunk-io]

   

View Full Report ↗︎ ⋅ Docs

[Gilbert09]

All code checks passed on 4386189, but GitHub has left Determine need to run storybook checks pending for over 20 minutes. Please rerun or cancel that stuck workflow/check so the PR can reach a fully completed CI state; the API rate limit also prevents this runner from inspecting it further.

_{🦉 via talyn.dev}

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-10-02 15:25 UTC	Run
prod-us	✅ Deployed	2026-10-02 15:47 UTC	Run
prod-eu	✅ Deployed	2026-10-02 15:49 UTC	Run