fix(sync): remint access token in-process on events.list 401 - #2778
Merged
Conversation
Stale cached tokens were retried as job-engine failures, reopening the PostHog fingerprint and burning the ladder on a dead grant. Force-refresh the page read once; drop when refresh fails or the fresh token is still rejected. Co-authored-by: Tyler Dane <tyler-dane@users.noreply.github.com>
Pull stored the provider token as const, so the in-process 401 retry threw when assigning the reminted value. Co-authored-by: Tyler Dane <tyler-dane@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Staging
incrementalPulljobs on connection6a7250be…were failing Googleevents.listwith HTTP 401authError. PostHog reused the old “Sync job engine failed” fingerprint (calendarListSync/subscriptionMaintain ladders already cleared after #2635/#2696/#2712). #2729 classified those 401s asProviderEventReadError(transient), so the worker retried the same cached access token up to 20 times and logged an error every attempt. #2754 later classified 401 asauthExpiredand invalidated the cache, but recovery still waited for the next job attempt — so a stale token still hit PostHog, and a dead/invalid grant still burned a retry before dropping.This remints the access token in-process on the same
events.listpage (pull, import, and repair):invalid_grant/authorizationRevoked→ dispatch drops the job and discards the credential so the connection asks for reconnect.Ops for the current staging connection still applies: reconnect Google (or
manage-failed-jobsif a row already exhausted). Deploying this stops the retry storm and the fingerprint reopen; it does not revive an already-revoked grant. #2748 (provider event identity collisions) is unrelated.Simplicity
One helper (
listEventPageWithAuthRetry) used by the three event-read engines. Reuses existing custodyinvalidateAccessToken+getValidAccessTokenand the existingProviderAuthErrordrop path. No new job states or health classes. No further simplification after the base-to-head pass — extracting the helper is the DRY boundary; inlining would triplicate the remint/drop contract.Automated validation
bun test:sync:fast -- packages/sync/src/domain/list-event-page-with-auth-retry.test.ts— 5 passbun test:sync --pull, dispatch, worker db tests — 62 passbun run lint— no new issues in touched filesIndependent review
Fresh read-only review of
main...HEAD: no confirmed findings.Test plan
onError