Build resumable, idempotent receipt batch processing with bounded-memory pagination

[grantfox-oss]

Context

The receipt batch pipeline in src/services/receipt.service.ts and the worker in src/workers/receipt.worker.ts currently process a single page of donations, update progress after each receipt, and mark the entire run as complete once the in-memory loop finishes. This is a real durability gap: a crash or restart leaves the batch stuck in PROCESSING, prevents resumption from the next pending donation, and can also undercount the true batch size because createBatchJob caps the job to config.receipts.maxBatchSize before the work begins.

Problem statement

Implement a resumable receipt batch processor that can process arbitrarily large donation sets without redoing completed work, persists enough state to resume safely after interruption, and guarantees idempotent receipt creation even when the same job is re-run or retried.

Current behavior

• src/services/receipt.service.ts creates a batch job with a totalCount based on a count capped by config.receipts.maxBatchSize.
• src/services/receipt.service.ts fetches an initial donation page, loops through it sequentially, generates a receipt for each donation, sends email, and updates progress per row.
• The current implementation has no durable checkpoint for the next pending donation, no explicit recovery path for interrupted batches, and no mechanism to guarantee that a restarted run will not re-create receipts for work that already completed.

Required behavior

• Batch jobs must persist a durable checkpoint describing which donation IDs or pagination cursor have already been processed.
• A restarted worker must resume from that checkpoint instead of restarting the entire job from the beginning.
• The implementation must make receipt generation idempotent so that a retry or resume does not create duplicate receipts or duplicate storage objects for a donation that was already processed.
• totalCount must reflect the full matching donation set rather than the capped subset used for the first page.
• The processor must support large queues without loading the entire result set into memory at once.

Constraints

• The fix must preserve the existing receipt storage, email, and job model semantics.
• The implementation must work within the current Prisma/PostgreSQL environment and must not require a new heavy dependency.
• The solution must remain safe under worker crashes, retries, and duplicate job execution.
• Do not rely on repository snapshots or point-in-time repo states; work against the live default branch only.

Acceptance criteria

• A batch job can be interrupted and resumed without generating duplicate receipts for donations already processed.
• A restarted worker picks up from the last durable checkpoint rather than restarting from the first donation.
• The batch job reports the true total number of matching donations rather than a capped subset.
• Unit tests cover checkpoint persistence, resume behavior, and duplicate prevention; integration tests cover job restart and replay scenarios.
• A performance regression test proves that a 10,000-donation batch can be processed without exhausting memory or timing out on a local PostgreSQL-backed test setup.

Out of scope

• Reworking the receipt PDF template or email design.
• Changing the public receipt API contract.
• Implementing a completely separate queueing system outside the existing worker architecture.

Hints and references

• Think about cursor-based pagination and durable checkpointing rather than a single large findMany call.
• Consider how to make receipt generation idempotent at the storage and database layers, not just at the application layer.
• Review how the current worker retries and the batch-job state machine interact so the resume path is safe under partial failure.

[oyinade247]

can i work on this pleaseeeeeee

[grantfox-oss]

🦊 GrantFox — @oyinade247 has been assigned to this issue as part of the Official Campaign | FWC26 campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #136)
2. Your PR will be reviewed by the aid-linkk maintainers

Good luck! Track your progress on GrantFox.