Repository navigation
Normalize shape key generation from where clause #3256
Description
A user spent several days trying to debug why a big shape was frequently getting recreated (409s) when they were navigating around the app and doing other actions. They weren't doing any of the normal things that trigger 409s it seemed like changing the shape def in their proxy or pushing schema updates to their db.
After adding lots of logging to the URLs generated in their proxy, they eventually noticed that under certain conditions, the order of the where in clauses changed, triggering Electric to think it's a new shape and return a 409 to force a reload of the shape.
This was both a very noticeable performance problem w/ their app and a head-scratcher to debug.
We should prevent this issue by normalizing all non-essential parts of the where clauses. I assume we do some already but sorting where in & array operations makes sense. I had Claude come up with a long list of other potential ways to normalize https://claude.ai/share/3230d2fa-95fb-40ed-9565-d50471c44698
Activity
balegas commentedon Feb 16, 2026
Have we done this?
KyleAMathews commentedon Feb 16, 2026
Not afaik
alco commentedon Apr 23, 2026
PR #4051 adds a round-trippable AST-to-SQL deparser (
Electric.Replication.Eval.SqlGenerator). That's the missing piece that makes a clean fix for this tractable — once it merges.The current shape-identity equivalence goes through
Expr.Comparableineval/expr.ex:expr.queryis the raw user-supplied SQL text, which is why whitespace/order/paren changes in the WHERE clause reshape the hash and force a 409. After PR #4051 it should be straightforward to swap the source text for a canonical deparse of the AST:SqlGeneratoris deterministic and precedence-aware, so two WHEREs that parse to the same AST will produce identical normalized SQL — which kills off whitespace, redundant parens, and casing differences for free. Sorting/canonicalization for commutative ops (INlists,AND/ORchains, array operations) would still need to be added at the AST level to fix the original report'sWHERE INorder-churn, but that's a much smaller change to layer on top of the deparser.A secondary bonus: PR #4051 also uses this same
comparableto dedup repeated subqueries within one shape's WHERE (build_shape_dependencies/2inshape.ex), so the same normalization would collapse duplicate-but-textually-different subqueries into a single inner materializer at no extra cost.