Repository navigation
Avoid invoking PublicationManager from SchemaReconciler when there are no diverged relations - #3108
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3108 +/- ##
==========================================
- Coverage 77.53% 77.06% -0.47%
==========================================
Files 169 172 +3
Lines 8931 9145 +214
Branches 242 296 +54
==========================================
+ Hits 6925 7048 +123
- Misses 2004 2095 +91
Partials 2 2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
✅ Deploy Preview for electric-next ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
As the failing integration test has shown, the change in the 1st commit alone wasn't enough. I have pushed another change that moves the responsibility of refreshing the publication with As for the need to fetch the publication state from the database once every minute, it could be debated. One thing it does for us is repair any meddling with the publication in Postgres outside of Electric. For example, if the admin adds a table to the publication that Electric can't read from (or can't update the replica identity off), the periodic |
bce197b to
5491d2c
Compare
|
Rebased the branch on top of #3119 |
…e no diverged relations
So far SchemaReconciler has been responsible both for updating the publication when it detected schema changes as well as when there are no schema changes. This latter periodic update is useful to keep the publication up-to-date with the filters but it's not really the responsibility of SchemaReconciler.
5491d2c to
a29029f
Compare
|
Rebased once more on top of #3152. |
Fixes #3167 We started seeing this error with an update made on 22nd of September, and until then one of the relevant changes included is this: #3108 Since @alco has introduced periodic forced publication updates anyway, and upon reconciling the schema we also trigger dropping relevant shapes (which would remove them from the publication on cleanup), we no longer need to refresh the publication manually IMO - not much to be gained other than unnecessary dependencies between our processes. It's still worth investigating why we're seeing these timeouts (but was harder to debug because we were missing sentry metadata, see #3166) - but ultimately we should simplify things. 1. Schema gets reconciled 2. Shapes using diverged/stale relations get dropped 3. Shape cleanup will involve removing them from the publication which should cause a publication update (as all relevant shapes for a given relation will be removed) 4. Even if (3) doesn't occur, which it will, we now periodically refresh the publication anyway In fact, I would go even further to say that since we're periodically refreshing the publication and thus detecting missing relations there, we should simply add an inspector cleanup and get rid of the schema reconciler in its entirety - what do you think @alco @icehaunter ?
Just getting rid of a no-op function invocation that pollutes the log with periodic log entries.