Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct and tune equinox-sync template progress management #19

Merged
merged 7 commits into from
Apr 12, 2019

Conversation

bartelink
Copy link
Collaborator

@bartelink bartelink commented Apr 11, 2019

The current equinox-sync impl does not deal with the following correctly:

  • if 5 retries fails to save the changes for a batch, it gives up, and CFP cannot be convinced not to write progress in this instance
  • there are bugs in the result processing which cause some items not to be processed
  • logging is extremely noisy compared to sister PR for EventStore

This PR brings over work done in the context of the ingester template in (#16) so will appear slightly overkill...

  • Target Equinox 2.0.0-preview5 in order to avail of manual checkpointing and logging
  • Make all progress writing subject to stream validations from CosmosIngester
  • Make progress writing async and lazy
  • Allow loading to run in advance of stream syncing
  • Add Seq logging support

@bartelink bartelink changed the title New ingester logic from ES sync support Fix equinox-sync template error management Apr 11, 2019
@bartelink bartelink changed the title Fix equinox-sync template error management Correct and tune equinox-sync template progress management Apr 12, 2019
@bartelink bartelink marked this pull request as ready for review April 12, 2019 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant