Skip to content

Commit 38e064e

Browse files
rawoclaude
andcommitted
fix(mysql): pin READ COMMITTED only on MySQL and only where the PTM accepts it (#129)
Addresses review on #139: - Spring: pin READ COMMITTED only when the store is MysqlOutboxStore. Postgres already defaults to it, so other stores keep the template verbatim (no extra round trip, no behaviour change). Skip pinning with a startup WARN for PTMs that reject custom isolation levels (JtaTransactionManager, JpaTransactionManager without the Hibernate/EclipseLink JpaDialect); otherwise every tick would throw InvalidIsolationLevelException. - Exposed: ExposedTransactionRunner(database) uses READ COMMITTED only on MySQL/MariaDB dialects; ExposedTransactionRunner(database, level) sets it explicitly. - MysqlClaimDoesNotBlockPublishersTest drives a real OutboxProcessor tick with a blocking deliverer instead of calling claimPending directly, so it follows whatever claim query the processor runs (e.g. after #120). Adds Hibernate-JPA and DefaultJpaDialect cases. - New OutboxAutoConfigurationIsolationTest, including a JtaTransactionManager case. - CHANGELOG: behaviour-change note for Exposed overriding defaultIsolationLevel on MySQL. README: binlog_format=ROW/MIXED requirement and the JTA/JPA fallback. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1 parent 192abf5 commit 38e064e

11 files changed

Lines changed: 421 additions & 132 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 21 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -27,19 +27,30 @@ Until `1.0.0`, breaking changes may appear in any release and are flagged with *
2727
default `REPEATABLE READ`, the claim's `FOR UPDATE SKIP LOCKED` took gap locks on
2828
`(status, created_at)`; with a backlog smaller than the batch, the gap ran to the end of the index,
2929
exactly where every `publish()` inserts, so business transactions waited for the whole HTTP/Kafka
30-
delivery and could fail with `Lock wait timeout exceeded`. The scheduler transaction now runs at
31-
`READ COMMITTED`, which takes no gap locks:
32-
- Spring Boot: the autoconfigured `TransactionRunner` pins `READ COMMITTED` when its
33-
`TransactionTemplate` leaves isolation at `DEFAULT`. It uses a copy, so a user's or Boot's
34-
template bean is not mutated and its timeout/propagation/read-only settings carry over; a
35-
template with an explicit isolation level is still used as-is.
36-
- `ExposedTransactionRunner` gains a `transactionIsolation` parameter defaulting to
37-
`READ_COMMITTED` (`null` restores the database default).
38-
39-
On Postgres `READ COMMITTED` is already the default, so behaviour there is unchanged. (#129)
30+
delivery and could fail with `Lock wait timeout exceeded`. On MySQL and MariaDB the scheduler
31+
transaction now runs at `READ COMMITTED`, which takes no gap locks:
32+
- Spring Boot: the autoconfigured `TransactionRunner` pins `READ COMMITTED` when the store is
33+
`MysqlOutboxStore` and its `TransactionTemplate` leaves isolation at `DEFAULT`. It uses a copy,
34+
so a user's or Boot's template bean is not mutated and its timeout/propagation/read-only
35+
settings carry over; a template with an explicit isolation level is still used as-is. Pinning is
36+
skipped, with a startup WARN, for PTMs that reject custom isolation levels — a
37+
`JtaTransactionManager`, or a `JpaTransactionManager` without the Hibernate/EclipseLink
38+
`JpaDialect` — since every tick would otherwise throw `InvalidIsolationLevelException`.
39+
- `ExposedTransactionRunner(database)` uses `READ COMMITTED` when the database's dialect is MySQL or
40+
MariaDB. A new `ExposedTransactionRunner(database, transactionIsolation)` constructor sets the
41+
level explicitly (`null` = the database default).
42+
43+
Other stores and databases are unchanged: Postgres already defaults to `READ COMMITTED`. MySQL at
44+
`READ COMMITTED` needs `binlog_format=ROW` or `MIXED` (see the README). (#129)
4045

4146
### Changed
4247

48+
- **`ExposedTransactionRunner` on MySQL/MariaDB now overrides `DatabaseConfig.defaultIsolationLevel`.**
49+
Scheduler transactions run at `READ COMMITTED` even when the application configured another default
50+
isolation (e.g. `SERIALIZABLE`) for its `Database`; business transactions keep that default. Pass
51+
the level you want to `ExposedTransactionRunner(database, transactionIsolation)`, or `null` to keep
52+
the database default, but note that `REPEATABLE READ` or stricter brings back the `publish()`
53+
blocking described under *Fixed*. (#129)
4354
- **`CompositeMessageDeliverer.deliverBatch`** now dispatches transport groups concurrently — one
4455
virtual thread per group, with the last group run on the calling thread — instead of blocking on
4556
each group in turn. A batch spanning N transports costs ~`max(Tᵢ)` rather than `sum(Tᵢ)`; a batch

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ Runnable, self-contained applications live in [okapi-examples](https://github.co
9494
- **Duplicate delivery is possible.** A crash between a successful delivery and the status update means the message may be sent again after restart. If processing a message more than once would cause unwanted effects, make the consumer idempotent. okapi sends your payload, your configured headers, and an `x-outbox-id` header to deduplicate on — see [Deduplicating on `x-outbox-id`](#deduplicating-on-x-outbox-id).
9595
- **Best-effort ordering.** Rows are claimed by `created_at`, oldest first. However, parallel delivery and retries mean messages may reach consumers in a different order. Strict delivery ordering is not guaranteed.
9696
- **Failure classification is the transport's job.** Each deliverer decides what is retriable. HTTP: 5xx, 429, 408 and connection errors are retriable; other responses and TLS errors are permanent. Kafka: broker-side retriable exceptions are retried; authorization and configuration errors are not.
97-
- **Scheduler transactions run at `READ COMMITTED`.** The autoconfigured Spring `TransactionRunner` and `ExposedTransactionRunner` both use it. On MySQL this matters: at the default `REPEATABLE READ`, the claim's gap locks would block every concurrent `publish()` until delivery finishes. If you supply your own `TransactionRunner` or a `TransactionTemplate` with an explicit isolation level, keep it at `READ COMMITTED` on MySQL.
97+
- **On MySQL, scheduler transactions run at `READ COMMITTED`.** The autoconfigured Spring `TransactionRunner` (with `MysqlOutboxStore`) and `ExposedTransactionRunner` (on a MySQL/MariaDB dialect) both use it. At MySQL's default `REPEATABLE READ`, the claim's gap locks would block every concurrent `publish()` until delivery finishes. If you supply your own `TransactionRunner`, an `OutboxStore` of your own, or a `TransactionTemplate` with an explicit isolation level, keep it at `READ COMMITTED` on MySQL. With a `JtaTransactionManager`, or a `JpaTransactionManager` without the Hibernate/EclipseLink `JpaDialect`, okapi cannot set it (startup logs a WARN): configure `transaction_isolation = 'READ-COMMITTED'` on the server or in the JDBC URL instead. `READ COMMITTED` requires `binlog_format=ROW` (the default since MySQL 5.7.7) or `MIXED`; with `STATEMENT`, InnoDB rejects the scheduler's writes with `ERROR 1665`.
9898
- **Retry budget.** `okapi.processor.max-retries` (default 5) counts retries *after* the first attempt — six attempts in total before a row becomes `FAILED`. `FAILED` is terminal. Retriable messages become eligible again on the next processor poll; there is no per-message backoff.
9999

100100
### Deduplicating on `x-outbox-id`

‎gradle/libs.versions.toml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ slf4j = "2.0.20"
1717
logback = "1.6.5"
1818
assertj = "3.27.7"
1919
h2 = "2.5.252"
20+
jakartaTransaction = "2.0.1"
2021
micrometer = "1.17.2"
2122
jmh = "1.37"
2223
jmhGradlePlugin = "0.7.3"
@@ -66,6 +67,8 @@ slf4jApi = { module = "org.slf4j:slf4j-api", version.ref = "slf4j" }
6667
slf4jSimple = { module = "org.slf4j:slf4j-simple", version.ref = "slf4j" }
6768
logbackClassic = { module = "ch.qos.logback:logback-classic", version.ref = "logback" }
6869
h2 = { module = "com.h2database:h2", version.ref = "h2" }
70+
# Test-only: a fake UserTransaction for JtaTransactionManager in okapi-spring-boot tests.
71+
jakartaTransactionApi = { module = "jakarta.transaction:jakarta.transaction-api", version.ref = "jakartaTransaction" }
6972
jmhCore = { module = "org.openjdk.jmh:jmh-core", version.ref = "jmh" }
7073
jmhGeneratorAnnprocess = { module = "org.openjdk.jmh:jmh-generator-annprocess", version.ref = "jmh" }
7174

Lines changed: 25 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package com.softwaremill.okapi.exposed
22

33
import com.softwaremill.okapi.core.TransactionRunner
4+
import org.jetbrains.exposed.v1.core.vendors.MysqlDialect
45
import org.jetbrains.exposed.v1.jdbc.Database
56
import org.jetbrains.exposed.v1.jdbc.transactions.transaction
67
import java.sql.Connection
@@ -12,18 +13,36 @@ import java.sql.Connection
1213
* Used by the outbox scheduler/processor when running outside of
1314
* an existing transactional context (e.g., background processing thread).
1415
*
15-
* Runs at [transactionIsolation], READ COMMITTED by default (issue #129). Under MySQL's default
16+
* On MySQL and MariaDB it runs at READ COMMITTED by default (issue #129). Under MySQL's default
1617
* REPEATABLE READ the claim's `FOR UPDATE SKIP LOCKED` takes gap locks at the end of the
1718
* `(status, created_at)` index -- where every `publish()` inserts -- so publishers block until the
18-
* claiming transaction, and with it delivery, finishes. On Postgres READ COMMITTED is already the
19-
* default. Pass `null` to use the [database]'s default isolation instead.
19+
* claiming transaction, and with it delivery, finishes. This overrides a
20+
* `DatabaseConfig.defaultIsolationLevel` set for the rest of the application. Other databases keep
21+
* their default (Postgres already defaults to READ COMMITTED).
2022
*
2123
* @param database The [Database] instance where the outbox table resides.
22-
* @param transactionIsolation a `java.sql.Connection.TRANSACTION_*` level, or `null` for the database default.
2324
*/
24-
class ExposedTransactionRunner @JvmOverloads constructor(
25+
class ExposedTransactionRunner private constructor(
2526
private val database: Database,
26-
private val transactionIsolation: Int? = Connection.TRANSACTION_READ_COMMITTED,
27+
explicitIsolation: Int?,
28+
isolationIsExplicit: Boolean,
2729
) : TransactionRunner {
30+
constructor(database: Database) : this(database, null, isolationIsExplicit = false)
31+
32+
/**
33+
* @param transactionIsolation a `java.sql.Connection.TRANSACTION_*` level, or `null` for the
34+
* [database]'s default isolation -- on every database, MySQL included.
35+
*/
36+
constructor(database: Database, transactionIsolation: Int?) : this(database, transactionIsolation, isolationIsExplicit = true)
37+
38+
// Lazy: resolving the dialect may open a connection, which the constructor shouldn't do.
39+
private val transactionIsolation: Int? by lazy {
40+
when {
41+
isolationIsExplicit -> explicitIsolation
42+
database.dialect is MysqlDialect -> Connection.TRANSACTION_READ_COMMITTED // MariaDBDialect extends it
43+
else -> null
44+
}
45+
}
46+
2847
override fun <T> runInTransaction(block: () -> T): T = transaction(database, transactionIsolation = transactionIsolation) { block() }
2948
}

‎okapi-exposed/src/test/kotlin/com/softwaremill/okapi/exposed/ExposedTransactionRunnerTest.kt‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ package com.softwaremill.okapi.exposed
22

33
import io.kotest.core.spec.style.FunSpec
44
import io.kotest.matchers.shouldBe
5+
import org.jetbrains.exposed.v1.core.DatabaseConfig
56
import org.jetbrains.exposed.v1.jdbc.Database
67
import org.jetbrains.exposed.v1.jdbc.transactions.TransactionManager
78
import org.jetbrains.exposed.v1.jdbc.transactions.transactionManager
@@ -18,8 +19,21 @@ class ExposedTransactionRunnerTest : FunSpec({
1819
(TransactionManager.current().connection.connection as Connection).transactionIsolation
1920
}
2021

21-
test("runs at READ COMMITTED by default (no MySQL gap locks on the claim, #129)") {
22-
isolationSeenBy(ExposedTransactionRunner(db)) shouldBe Connection.TRANSACTION_READ_COMMITTED
22+
// The MySQL default (READ COMMITTED, #129) is covered against a real MySQL in
23+
// okapi-integration-tests' MysqlClaimDoesNotBlockPublishersTest.
24+
test("keeps the database's default isolation on a non-MySQL database") {
25+
// H2's own default is already READ COMMITTED, so configure another one to tell them apart.
26+
val serializableDb = Database.connect(
27+
"jdbc:h2:mem:runner_test_serializable_${System.nanoTime()};DB_CLOSE_DELAY=-1",
28+
driver = "org.h2.Driver",
29+
databaseConfig = DatabaseConfig { defaultIsolationLevel = Connection.TRANSACTION_SERIALIZABLE },
30+
)
31+
isolationSeenBy(ExposedTransactionRunner(serializableDb)) shouldBe Connection.TRANSACTION_SERIALIZABLE
32+
}
33+
34+
test("honours an explicit READ COMMITTED") {
35+
isolationSeenBy(ExposedTransactionRunner(db, Connection.TRANSACTION_READ_COMMITTED)) shouldBe
36+
Connection.TRANSACTION_READ_COMMITTED
2337
}
2438

2539
test("honours an explicit isolation level") {

0 commit comments

Comments
 (0)