fix: claim only delivery types supported by each worker

Add route-aware claims and a matching index for PostgreSQL and MySQL. Rotate
the first route between batches, reject unsafe custom stores, and prevent a
single deliverer from processing an entry of another type.

Cover heterogeneous workers, concurrent claims, migrations, and a million-row
Docker benchmark. Document the custom-store and rolling-upgrade requirements.

Closes #116

Author: endrju19

README.md

@@ -85,14 +85,16 @@ Runnable, self-contained applications live in [okapi-examples](https://github.co
 ## How it works
 
 1. `publish()` writes a `PENDING` row to `okapi_outbox` in your transaction.
-2. A background scheduler polls for pending rows (every second by default), claiming them with `FOR UPDATE SKIP LOCKED` so workers do not process the same row concurrently.
+2. A background scheduler polls for pending rows (every second by default), claiming only the delivery types configured on that worker with `FOR UPDATE SKIP LOCKED` so workers do not process the same row concurrently.
 3. Each row goes to the transport matching its delivery type. Success marks it `DELIVERED`; a retriable failure leaves it `PENDING` while retry attempts remain; an exhausted retry budget or permanent failure marks it `FAILED`.
 4. A purger deletes delivered rows after a retention period.
 
 ### Guarantees and limits
 
 - **Duplicate delivery is possible.** A crash between a successful delivery and the status update means the message may be sent again after restart. If processing a message more than once would cause unwanted effects, make the consumer idempotent — for example, deduplicate on a business key in the payload or a header set in the `DeliveryInfo`. okapi sends your payload and configured headers, but not the `OutboxId` returned by `publish()`; that identifier stays on the publisher side for correlation and logging.
-- **Best-effort ordering.** Rows are claimed by `created_at`, oldest first. However, parallel delivery and retries mean messages may reach consumers in a different order. Strict delivery ordering is not guaranteed.
+- **Best-effort ordering.** Rows are claimed by `created_at, id` within each delivery type. The processor rotates which type it checks first between batches. Parallel delivery and retries can also change arrival order. Strict delivery ordering across types or within a type is not guaranteed.
+- **Different workers can handle different delivery types.** Each worker claims only its registered types. A custom `OutboxStore` used for processing must implement `RouteAwareOutboxStore`; otherwise processing fails before an unsafe claim. The older `claimPending(limit)` method remains available for callers that need it, but Okapi's processor does not use it. Apply the new index migration before starting updated workers if you manage the schema yourself.
+- **Rolling upgrades need sequencing.** Workers running an older Okapi version can still claim rows of a newly introduced delivery type. Upgrade or stop all old processors before publishing that type. An upgrade does not automatically retry rows already marked `FAILED` by older workers.
 - **Failure classification is the transport's job.** Each deliverer decides what is retriable. HTTP: 5xx, 429, 408 and connection errors are retriable; other responses and TLS errors are permanent. Kafka: broker-side retriable exceptions are retried; authorization and configuration errors are not.
 - **Retry budget.** `okapi.processor.max-retries` (default 5) counts retries *after* the first attempt — six attempts in total before a row becomes `FAILED`. `FAILED` is terminal. Retriable messages become eligible again on the next processor poll; there is no per-message backoff.
 

benchmarks/route-aware-claim/README.md

@@ -0,0 +1,23 @@
+# Route-aware claiming on a large outbox
+
+Local exploratory measurements with PostgreSQL 16 and MySQL 8.0 in Docker. The setup SQL files create a table with the Okapi columns and 1,000,000 `PENDING` rows. The oldest 900,000 use `delivery_type='unhandled'`; the newer 100,000 are split evenly across 100 `handled_00` through `handled_99` routes. Both the old `(status, created_at)` index and the new `(status, delivery_type, created_at, id)` index are present.
+
+Run `okapi-116-pg-setup.sql` with `psql` or `okapi-116-my-setup.sql` with `mysql`, then run the matching `okapi-116-*-queries.sql` file. The query files compare one route, an `IN` query for 100 present routes, and an `IN` query for 100 absent routes. All use `FOR UPDATE SKIP LOCKED` and a batch limit of 100. The `okapi-116-*-client-100.sql` files send 100 separate empty claims through a SQL client.
+
+| Case | PostgreSQL observed execution | MySQL observed execution |
+| --- | ---: | ---: |
+| One present route, first 100 rows | 0.12 ms | 0.49 ms |
+| `IN` with 100 absent routes, ordered by type and creation time | 165 ms | 0.21 ms |
+| 100 separate empty route claims, server-side loop | 1.57 ms | 2.04–2.37 ms |
+| 100 separate empty route claims through one local SQL client process | 24 ms | 15 ms |
+
+The PostgreSQL plan for the absent `IN` case scanned all 1,000,000 rows and removed them by filter, despite the new index. The one-route plan on both databases used the new composite index and stopped at the limit. This is why the processor claims one route at a time and stops once its batch is full. At 100 empty routes it still performs 100 small queries per poll; this cost grows with the number of configured routes.
+
+The `okapi-116-*-write.sql` files insert 20,000 rows and update about 20,000 statuses inside rolled-back transactions. One run with the new index and one after dropping only that index gave:
+
+| Operation | PostgreSQL with / without new index | MySQL with / without new index |
+| --- | ---: | ---: |
+| Insert 20,000 | 125 / 81 ms | 177 / 141 ms |
+| Update about 20,000 statuses | 158 / 135 ms | 363 / 304 ms |
+
+These write figures are **single local runs**, sensitive to cache state and test order. They show a real write-cost tradeoff, not a stable throughput estimate. The read measurements are also exploratory; the query plan and scanned-row counts are the main evidence. Integration tests separately cover route isolation, concurrent processors, migrations, and delivery behavior on both databases.