[cigrainger]

Summary

When a computed %Dux{} (table ref) is distributed to workers, it was going through Elixir rows:

1. table_to_rows → list of maps (expensive materialization)
2. Sent as {:list, rows} in pipeline struct (large Erlang term)
3. Workers re-ingest via rows_to_columns → ADBC (rebuild Arrow from scratch)

Now uses Arrow IPC — the same binary format workers already use for results:

1. table_to_ipc → compact Arrow binary
2. Sent as {:ipc, binary} (binary copy, no term encoding)
3. Workers table_from_ipc → zero-copy ADBC ingest

Changes:

• Add {:ipc, binary} source type in QueryBuilder
• Partitioner.replicate/2 uses IPC instead of rows
• Shuffle.ensure_worker_safe/1 uses IPC instead of rows
• Inspect handles {:ipc, _} source display

Benchmark (2 local workers)

Benchmark	Before (median)	After (median)	Speedup
Replicate 1K rows	16ms	27ms	~same (IPC overhead on small data)
Replicate 10K rows	381ms	24ms	15.9x
Broadcast join (computed right)	260ms	72ms	3.6x

Test plan

• 29 distributed tests pass (0 failures)
• Credo strict clean
• Benchmarks before and after

🤖 Generated with Claude Code