Skip to content

feat(maintainer): reconcile stored author_association to live GitHub roles#190

Merged
entrius merged 2 commits into
testfrom
feat/maintainer-role-reconciler
Jun 16, 2026
Merged

feat(maintainer): reconcile stored author_association to live GitHub roles#190
entrius merged 2 commits into
testfrom
feat/maintainer-role-reconciler

Conversation

@anderdc

@anderdc anderdc commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

author_association is snapshotted at mirror-ingest time and never refreshed. If a contributor becomes (or stops being) a maintainer after a row is ingested, every stored PR/issue/review/comment row keeps the stale role forever. Scoring that reads the per-row snapshot (the issue-bonus tier in entrius/gittensor) is then permanently wrong for those authors.

Concrete miss on phase-rs/phase: matthewevans (id 1388610, currently MEMBER) and plind-junior (id 59729252, currently COLLABORATOR) have every stored row stuck at CONTRIBUTOR — they were private collaborators when the rows were ingested. The /repos/:repo/maintainers view already reports the correct current role (it derives latest-known role), and the maintainer_cut carve-out already trusts it, but the raw per-row columns scoring reads do not.

Fix

A new MaintainerRoleReconcileService (hourly @Cron) that, for registered + installed repos only (registered = true AND installation_id IS NOT NULL):

  1. Pulls the live maintainer set from GitHub — direct collaborators (GET /repos/{owner}/{repo}/collaborators?affiliation=direct) + org members (GET /orgs/{org}/members).
  2. Promotes: rewrites each current maintainer's rows to their live association across the four association-bearing tables (pull_requests, issues, comments, reviews) — the same set contributor_repo_roles reads.
  3. Demotes: anyone still flagged OWNER/MEMBER/COLLABORATOR who is no longer in the live set is reset to CONTRIBUTOR.

After this runs, the stored snapshot reflects current role, so the downstream validator check stays the trivial author_association IN (...) — no validator-side workaround needed.

Safety

  • Fetches both sets before any write, and fails closed per-repo: a fetch/DB error skips that repo (caught, logged) so a partial fetch can never read as a mass demotion; the next sweep retries.
  • An empty maintainer set from a non-throwing GitHub response is treated as suspicious and skipped (a real repo always has its owner), rather than demoting every contributor.
  • RETURNING 1 row-counts make promote/demote logging version-independent.
  • Reuses GitHubFetcherService's existing installation-token cache + rate-limit-aware githubFetch.

What this does NOT change

  • No schema change. Columns are rewritten in place (author_association is a scoring-support field, not an audit record).
  • Near-real-time member/membership/organization webhook triggers are intentionally out of scope — the hourly sweep is the v1; webhooks can tighten the window later.

Dependency

Adds @nestjs/schedule@^4.1.2 (Nest 10-compatible) + ScheduleModule.forRoot(). The existing reconcilers (PrReconcileService, etc.) self-schedule with setInterval; this introduces @nestjs/schedule as the cleaner standard going forward.

Lockfile note

package-lock.json shows a large delete count because npm install also pruned pre-existing extraneous entries that were stale on test (redis/@redis/*, generic-pool, chokidar deps — not referenced by the current package.json tree; verified keyv/@keyv/redis still resolve). The only additions are @nestjs/schedule, cron, @types/luxon. Happy to split the pruning into a separate hygiene PR if preferred.

Testing

  • npm run build — clean (tsc via nest build).
  • npm run lint — clean (eslint + prettier).
  • No test files included, per request.

anderdc added 2 commits June 16, 2026 13:02
…roles

author_association is snapshotted at mirror-ingest time and never refreshed.
A contributor who becomes (or stops being) a maintainer after filing keeps a
stale role on every historical PR/issue/review/comment row — e.g. on
phase-rs/phase, matthewevans (MEMBER) and plind-junior (COLLABORATOR) have all
historical rows stuck at CONTRIBUTOR because they were private collaborators at
ingest. Scoring that reads the per-row snapshot therefore denies solvers the
maintainer issue-bonus tier indefinitely.

Add an hourly MaintainerRoleReconcileService that, for registered + installed
repos only, pulls the live collaborator + org-member set from GitHub and
rewrites the four association columns (pull_requests, issues, comments,
reviews) to the current role. Promotes current maintainers and demotes departed
ones to CONTRIBUTOR. Fetches both sets before any write and fails closed per
repo, so a partial/failed fetch never reads as a mass demotion.

Wires in @nestjs/schedule (@Cron) for the sweep. Two new GitHubFetcherService
methods (fetchRepoCollaborators, fetchOrgMembers) reuse the existing
installation-token + rate-limit plumbing.
The previous lockfile was regenerated with npm 11, which deduped chokidar
to v4 and pruned the chokidar@3 transitive deps (readdirp@3.6.0,
picomatch@2.3.2, binary-extensions@2.3.0) still required by
@angular-devkit/core@17 via @nestjs/schematics@10. CI runs npm 10 (Node
20), which strictly rejects the missing entries, failing 'npm ci' before
lint/build could run. Regenerated with npm 10 to match CI.
@entrius entrius merged commit 1a6f2ca into test Jun 16, 2026
2 checks passed
@entrius entrius deleted the feat/maintainer-role-reconciler branch June 16, 2026 22:39
entrius pushed a commit that referenced this pull request Jun 17, 2026
…s table (#192)

PR #190 reconciled author_association by rewriting the stored column across
4 tables hourly — write-amplification, an hour staleness window, and it
corrupts the event-time meaning of the snapshot. Separately, the per-row
label/review association resolution went through contributor_repo_roles, a
plain view re-derived per row (~30ms each: sort + DISTINCT ON over ~8k
rows), which times the validator out on high-volume miners.

Both collapse into one primitive: a live, indexed maintainers table.

- Add maintainers (repo_full_name, github_id, login, association, refreshed_at),
  PK (repo_full_name, github_id); repo_full_name stored lowercased so reads
  join as m.repo_full_name = LOWER(src.repo_full_name) and still hit the PK.
- Convert the reconcile service into MaintainerPopulateService: same fetcher
  (direct collaborators + org members), same safety rules (fetch-before-write,
  fail-closed-per-repo, skip-on-empty), but atomically upserts the maintainers
  table per repo instead of rewriting 4 tables. Runs hourly + once on boot.
- Repoint pr_labels_by_actor / issue_labels_by_actor actor_association and
  pr_linked_issues issue_author_association and pr_review_summary's maintainer
  filter off contributor_repo_roles onto maintainers (indexed lookup; ~10-20x).
- Resolve PR/issue author_association at serve time in miners.service via
  COALESCE(maintainers.association, stored) — no new payload field, so the
  validator needs no change.
- Repoint /repos/:repo/maintainers to the table. Mark stored association
  columns as ingest snapshots; leave contributor_repo_roles in place (no
  remaining hot-path consumers).

Co-authored-by: anderdc <me@alexanderdc.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants