Skip to content

Commit 91fbe92

Browse files
alcoclaude
andcommitted
Defer materializer subscription until the in-flight transaction commits
Deferring only the write-unit promotion was not enough. A materializer's initial history read is bounded by storage's `last_seen_txn_offset`, which advances only on commit, while a fragment-streaming consumer's `latest_offset` advances with every fragment it writes. Replying to a mid-transaction subscription with that offset let the materializer skip past fragments it could not yet read, silently dropping the head of the transaction from its index: a two-fragment transaction inserting rows 1 and 2 left the materializer with link values `[2]`. Park the subscription instead: when `subscribe_materializer` arrives while a fragmented transaction is pending, stash `{pid, from}` in `deferred_materializer_subscription` and complete it (monitor, promote, reply) from `finish_pending_txn/1` at the commit boundary. The consumer is not marked as subscribed during the wait, so it never calls into the blocked materializer; the materializer then reads a fully committed history and receives every later change as a whole transaction. `State.mark_materializer_subscribed/1` now assumes that invariant and promotes directly; the separate `maybe_promote_write_unit/1` hook is gone. A consumer already buffering whole transactions can still be subscribed mid-transaction, since it has no partial writes on disk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016uMB7eVy1iXSfY8gqcn5A8
1 parent 384696c commit 91fbe92

5 files changed

Lines changed: 117 additions & 66 deletions

File tree

‎.changeset/promote-inner-consumer-write-unit.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22
'@core/sync-service': patch
33
---
44

5-
Promote an already-running standalone shape consumer to whole-transaction writes when a subquery materializer subscribes to it. If a fragmented transaction is already in progress, defer the promotion until its commit boundary.
5+
Promote an already-running standalone shape consumer to whole-transaction writes when a subquery materializer subscribes to it. If a fragmented transaction is already in progress, the subscription is completed at its commit boundary so the materializer never misses the already-written head of that transaction.

‎packages/sync-service/lib/electric/shapes/consumer.ex‎

Lines changed: 45 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -217,10 +217,21 @@ defmodule Electric.Shapes.Consumer do
217217
end
218218
end
219219

220+
# A materializer's initial history read is bounded by the last committed offset in
221+
# storage, while `state.latest_offset` advances with every fragment written in
222+
# `write_unit: :txn_fragment` mode. Replying mid-transaction would hand the materializer an
223+
# offset past what it can read, silently losing the already-written head of the
224+
# transaction. Park the subscription until the commit boundary instead — see
225+
# `complete_deferred_materializer_subscription/1`.
226+
def handle_call({:subscribe_materializer, pid}, from, %State{pending_txn: pending_txn} = state)
227+
when is_write_unit_txn_fragment(state.write_unit) and not is_nil(pending_txn) do
228+
Logger.debug("Deferring materializer subscription for #{state.shape_handle} until commit")
229+
state = %{state | deferred_materializer_subscription: {pid, from}}
230+
{:noreply, state, state.hibernate_after}
231+
end
232+
220233
def handle_call({:subscribe_materializer, pid}, _from, state) do
221-
Logger.debug("Subscribing materializer for #{state.shape_handle}")
222-
Process.monitor(pid, tag: :materializer_down)
223-
state = State.mark_materializer_subscribed(state)
234+
state = subscribe_materializer(pid, state)
224235
{:reply, {:ok, state.latest_offset}, state, state.hibernate_after}
225236
end
226237

@@ -467,8 +478,9 @@ defmodule Electric.Shapes.Consumer do
467478
# ShapeLogCollector holds no pending flush entry for this shape.
468479
defp consumer_can_suspend?(state) do
469480
is_snapshot_started(state) and not Shape.has_dependencies(state.shape) and
470-
not state.materializer_subscribed? and is_nil(state.pending_txn) and
471-
state.txn_offset_mapping == [] and is_nil(state.pending_flush_offset)
481+
not state.materializer_subscribed? and is_nil(state.deferred_materializer_subscription) and
482+
is_nil(state.pending_txn) and state.txn_offset_mapping == [] and
483+
is_nil(state.pending_flush_offset)
472484
end
473485

474486
defp schedule_suspend_timer(%{suspend_after: nil} = state), do: state
@@ -1197,19 +1209,41 @@ defmodule Electric.Shapes.Consumer do
11971209
state
11981210
end
11991211

1200-
# After a pending transaction completes and txn_offset_mapping is populated,
1201-
# process the deferred flushed offset (if any), then promote a consumer that
1202-
# gained a materializer subscriber while the transaction was in progress.
1212+
# After a pending transaction completes and txn_offset_mapping is populated, process the
1213+
# deferred flushed offset (if any), then complete a materializer subscription that arrived
1214+
# while the transaction was in progress.
12031215
#
12041216
# Even if the most recent transaction is skipped or no changes from it end up satisfying the
12051217
# shape's `where` condition, Storage may have signaled a flush offset from the previous transaction
12061218
# while we were still processing fragments of the current one. Therefore this function must
1207-
# `finish_pending_txn/1` must be called any time `state.pending_txn` is reset to nil in a
1208-
# multi-fragment transaction processing setting.
1219+
# be called any time `state.pending_txn` is reset to nil in a multi-fragment transaction
1220+
# processing setting.
12091221
defp finish_pending_txn(state) do
12101222
state
12111223
|> clear_pending_flush_offset()
1212-
|> State.maybe_promote_write_unit()
1224+
|> complete_deferred_materializer_subscription()
1225+
end
1226+
1227+
defp complete_deferred_materializer_subscription(
1228+
%{deferred_materializer_subscription: nil} = state
1229+
),
1230+
do: state
1231+
1232+
# The transaction that was in flight when the materializer called us has now committed, so
1233+
# `state.latest_offset` sits on a commit boundary and everything up to it is readable from
1234+
# storage. Subscribe and release the materializer's blocked call.
1235+
defp complete_deferred_materializer_subscription(
1236+
%{deferred_materializer_subscription: {pid, from}} = state
1237+
) do
1238+
state = subscribe_materializer(pid, %{state | deferred_materializer_subscription: nil})
1239+
GenServer.reply(from, {:ok, state.latest_offset})
1240+
state
1241+
end
1242+
1243+
defp subscribe_materializer(pid, state) do
1244+
Logger.debug("Subscribing materializer for #{state.shape_handle}")
1245+
Process.monitor(pid, tag: :materializer_down)
1246+
State.mark_materializer_subscribed(state)
12131247
end
12141248

12151249
defp clear_pending_flush_offset(%{pending_flush_offset: nil} = state), do: state

‎packages/sync-service/lib/electric/shapes/consumer/state.ex‎

Lines changed: 20 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,11 @@ defmodule Electric.Shapes.Consumer.State do
2626
buffer: [],
2727
txn_offset_mapping: [],
2828
materializer_subscribed?: false,
29+
# `{pid, from}` of a materializer whose `subscribe_materializer` call arrived while a
30+
# fragmented transaction was in progress. The subscription is completed (and the call
31+
# replied to) once that transaction commits, so the materializer's initial history read
32+
# starts from a commit boundary.
33+
deferred_materializer_subscription: nil,
2934
terminating?: false,
3035
buffering?: false,
3136
# Based on the write unit value, consumer will either buffer txn fragments in memory until
@@ -135,26 +140,27 @@ defmodule Electric.Shapes.Consumer.State do
135140
}
136141
end
137142

143+
@doc """
144+
Record that a materializer is subscribed to this consumer and promote the consumer to
145+
whole-transaction writes: a materializer needs transaction-atomic updates.
146+
147+
A fragment-streaming consumer must only be promoted with no transaction in progress
148+
(`pending_txn: nil`), so that already-written fragments are never mixed with a buffered
149+
tail of the same transaction. The consumer guarantees this by deferring subscriptions that
150+
arrive mid-transaction until the commit boundary. A consumer already buffering whole
151+
transactions can be subscribed at any point.
152+
"""
138153
@spec mark_materializer_subscribed(t()) :: t()
139-
def mark_materializer_subscribed(%__MODULE__{} = state) do
140-
maybe_promote_write_unit(%{state | materializer_subscribed?: true})
154+
def mark_materializer_subscribed(%__MODULE__{write_unit: @write_unit_txn} = state) do
155+
%{state | materializer_subscribed?: true}
141156
end
142157

143-
# A materializer needs transaction-atomic updates. If it subscribes while a fragmented
144-
# transaction is in progress, keep writing fragments until that transaction completes.
145-
@spec maybe_promote_write_unit(t()) :: t()
146-
def maybe_promote_write_unit(
147-
%__MODULE__{
148-
materializer_subscribed?: true,
149-
pending_txn: nil,
150-
write_unit: @write_unit_txn_fragment
151-
} = state
158+
def mark_materializer_subscribed(
159+
%__MODULE__{write_unit: @write_unit_txn_fragment, pending_txn: nil} = state
152160
) do
153-
%{state | write_unit: @write_unit_txn}
161+
%{state | materializer_subscribed?: true, write_unit: @write_unit_txn}
154162
end
155163

156-
def maybe_promote_write_unit(%__MODULE__{} = state), do: state
157-
158164
@doc """
159165
After the storage is ready, initialize the state with info from storage and writer state.
160166
"""

‎packages/sync-service/test/electric/shapes/consumer/state_test.exs‎

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -118,21 +118,18 @@ defmodule Electric.Shapes.Consumer.StateTest do
118118
%{state: state}
119119
end
120120

121-
test "promotes a standalone consumer to transaction writes", %{state: state} do
121+
test "promotes a fragment-streaming consumer to transaction writes", %{state: state} do
122+
state = %{state | write_unit: :txn_fragment}
123+
122124
assert %{materializer_subscribed?: true, write_unit: :txn} =
123125
State.mark_materializer_subscribed(state)
124126
end
125127

126-
test "defers promotion until an in-progress transaction completes", %{state: state} do
127-
state = %{state | pending_txn: %{}}
128-
129-
assert %{materializer_subscribed?: true, write_unit: :txn_fragment} =
130-
state = State.mark_materializer_subscribed(state)
128+
test "subscribes a transaction-buffering consumer even mid-transaction", %{state: state} do
129+
state = %{state | write_unit: :txn, pending_txn: %{}}
131130

132-
assert %{write_unit: :txn} =
133-
state
134-
|> Map.put(:pending_txn, nil)
135-
|> State.maybe_promote_write_unit()
131+
assert %{materializer_subscribed?: true, write_unit: :txn, pending_txn: %{}} =
132+
State.mark_materializer_subscribed(state)
136133
end
137134
end
138135

‎packages/sync-service/test/electric/shapes/consumer_test.exs‎

Lines changed: 44 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ defmodule Electric.Shapes.ConsumerTest do
1313
alias Electric.Shapes
1414
alias Electric.Shapes.Shape
1515
alias Electric.Shapes.Consumer
16+
alias Electric.Shapes.Consumer.Materializer
1617

1718
alias Support.StubInspector
1819

@@ -28,7 +29,8 @@ defmodule Electric.Shapes.ConsumerTest do
2829
register_as_replication_client: 1,
2930
complete_txn_fragment: 3,
3031
txn_fragments: 3,
31-
txn_fragment: 4
32+
txn_fragment: 4,
33+
wait_until: 1
3234
]
3335

3436
@receive_timeout 1_000
@@ -1958,8 +1960,7 @@ defmodule Electric.Shapes.ConsumerTest do
19581960
assert [] == :ets.tab2list(table)
19591961
end
19601962

1961-
test "defers write-unit promotion until the current fragmented transaction commits",
1962-
ctx do
1963+
test "materializer subscribing mid-transaction sees the whole transaction", ctx do
19631964
{shape_handle, _} = ShapeCache.get_or_create_shape_handle(@shape1, ctx.stack_id)
19641965
:started = ShapeCache.await_snapshot_start(shape_handle, ctx.stack_id)
19651966

@@ -1969,44 +1970,57 @@ defmodule Electric.Shapes.ConsumerTest do
19691970
xid = 11
19701971
lsn = Lsn.from_integer(10)
19711972

1973+
new_record = fn id, offset ->
1974+
%Changes.NewRecord{
1975+
relation: {"public", "test_table"},
1976+
record: %{"id" => id},
1977+
log_offset: offset
1978+
}
1979+
end
1980+
19721981
begin_fragment =
1973-
txn_fragment(
1974-
xid,
1975-
lsn,
1976-
[
1977-
%Changes.NewRecord{
1978-
relation: {"public", "test_table"},
1979-
record: %{"id" => "1"},
1980-
log_offset: LogOffset.new(lsn, 0)
1981-
}
1982-
],
1983-
has_begin?: true
1984-
)
1982+
txn_fragment(xid, lsn, [new_record.("1", LogOffset.new(lsn, 0))], has_begin?: true)
19851983

19861984
assert :ok = ShapeLogCollector.handle_event(begin_fragment, ctx.stack_id)
19871985
assert %{pending_txn: %{}, write_unit: :txn_fragment} = :sys.get_state(consumer_pid)
19881986

1989-
assert {:ok, _offset} = Consumer.subscribe_materializer(ctx.stack_id, shape_handle, self())
1987+
materializer_opts = %{stack_id: ctx.stack_id, shape_handle: shape_handle}
19901988

1991-
assert %{materializer_subscribed?: true, pending_txn: %{}, write_unit: :txn_fragment} =
1989+
materializer_pid =
1990+
start_link_supervised!(
1991+
{Materializer,
1992+
Map.merge(materializer_opts, %{
1993+
storage: ctx.storage,
1994+
columns: ["id"],
1995+
materialized_type: {:array, :int8}
1996+
})}
1997+
)
1998+
1999+
# The subscription is parked until the in-flight transaction commits.
2000+
assert wait_until(fn ->
2001+
match?(
2002+
%{deferred_materializer_subscription: {^materializer_pid, _}},
2003+
:sys.get_state(consumer_pid)
2004+
)
2005+
end)
2006+
2007+
assert %{materializer_subscribed?: false, write_unit: :txn_fragment} =
19922008
:sys.get_state(consumer_pid)
19932009

19942010
commit_fragment =
1995-
txn_fragment(
1996-
xid,
1997-
lsn,
1998-
[
1999-
%Changes.NewRecord{
2000-
relation: {"public", "test_table"},
2001-
record: %{"id" => "2"},
2002-
log_offset: LogOffset.new(lsn, 1)
2003-
}
2004-
],
2005-
has_commit?: true
2006-
)
2011+
txn_fragment(xid, lsn, [new_record.("2", LogOffset.new(lsn, 1))], has_commit?: true)
20072012

20082013
assert :ok = ShapeLogCollector.handle_event(commit_fragment, ctx.stack_id)
2009-
assert %{pending_txn: nil, write_unit: :txn} = :sys.get_state(consumer_pid)
2014+
2015+
assert %{
2016+
pending_txn: nil,
2017+
write_unit: :txn,
2018+
materializer_subscribed?: true,
2019+
deferred_materializer_subscription: nil
2020+
} = :sys.get_state(consumer_pid)
2021+
2022+
assert :ok = Materializer.wait_until_ready(materializer_opts)
2023+
assert Materializer.get_link_values(materializer_opts) == MapSet.new([1, 2])
20102024
end
20112025

20122026
@tag with_pure_file_storage_opts: [flush_period: 1]

0 commit comments

Comments
 (0)