Skip to content

Design and implement webhook replay/backfill reconciliation after downtime #262

Description

@Lakes41

Difficulty: Expert
Type: Feature

Background
GuildPass core delivers events (member.joined, member.left, pass.activated, pass.expired, guild.updated) to the dashboard via signed webhooks, verified with @guildpass/webhook-utils and deduplicated via the idempotency store discussed in Issue #8. Webhooks are inherently a best-effort, at-least-once (or worse, at-most-once if the sender doesn't retry) delivery mechanism — if the dashboard is down, mid-deploy, or the idempotency store itself is unavailable during an incident, events can be permanently missed with no built-in recovery path.

Problem
There is currently no mechanism for the dashboard to detect that it has missed a window of webhook events (e.g. after an outage) or to reconcile its local state (durable storage, if Issue #6 lands, or mock/in-memory state otherwise) against the source of truth once it comes back online. This is a correctness gap: a guild's member/pass state can silently drift from reality with no automated recovery.

Expected outcome
On startup (or on a configurable interval / manual trigger from /settings or /integrations), the dashboard can request a reconciliation pass: fetch the authoritative current state (or an event log, if GuildPass core exposes one) for a bounded time window via @guildpass/integration-client, diff it against local state, and apply corrective updates while emitting synthetic activity-log entries flagging the reconciliation (so admins can see "this data was corrected via reconciliation, not a live webhook" in the /activity feed).

Suggested implementation

  • Define the reconciliation contract clearly: what does the dashboard ask GuildPass core for (a point-in-time snapshot per guild vs. an event-log range), and design the integration-client methods needed to support it, acknowledging this may require coordinating on an interface that GuildPass core would need to support (call this out explicitly in the PR/issue discussion, since guildpass-core is a separate repository per SECURITY.md's scope notes).
  • Implement a reconciliation job (triggerable manually first, then optionally on an interval) that: pulls current authoritative state, diffs against local storage, and applies only the deltas.
  • Tag every reconciliation-driven change in the activity log distinctly from webhook-driven changes (e.g. a source: "reconciliation" field surfaced in the UI).
  • Guard against reconciliation itself causing duplicate/incorrect activity entries by routing applied deltas through the same idempotency-aware write path used for webhooks.
  • Include a dry-run mode that reports what a reconciliation pass would change without applying it, for safe operational use.

Acceptance criteria

  • A reconciliation pass can be triggered (manually, at minimum) and produces a clear diff/report of proposed changes before (dry-run) or alongside (applied) the update.
  • Applied reconciliation changes are visually distinguishable from webhook-driven changes in /activity.
  • Reconciliation does not create duplicate activity entries for state that hasn't actually changed.
  • The design explicitly documents its dependency on (or workaround for) GuildPass core's available APIs, since full implementation may be blocked on core-side support — a partial implementation behind a mock/stubbed core response is acceptable to unblock this repo's side of the work.
  • Tests cover: no-drift (no-op) reconciliation, partial drift, and full resync scenarios against a stubbed core client.

Likely affected files/directories
packages/integration-client/, new reconciliation module in apps/dashboard/lib/, apps/dashboard/app/settings/ or apps/dashboard/app/integrations/ (trigger UI), activity log write path.

Metadata

Metadata

Assignees

Labels

GrantFox OSSGrantFox Open Source Sponsorship program tagMaybe RewardedIssue may qualify for a reward upon successful completion per campaign rulesOfficial Campaign | FWC26Official FWC26 campaign issue — eligible for campaign scoring and rewardsarchitectureStructural / design-level changeenhancementImprovement or extension to existing project functionality, architecture, or developer experiencehelp wantedExtra attention is neededsecuritySecurity-related fix, hardening, audit, or vulnerability remediation

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions