Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: add internal data system config #185

Draft
wants to merge 71 commits into
base: v7
Choose a base branch
from
Draft

refactor: add internal data system config #185

wants to merge 71 commits into from

Commits on Sep 16, 2024

  1. refactor: add internal data system config

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    0988889 View commit details
    Browse the repository at this point in the history

  2. more docs

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    46814dd View commit details
    Browse the repository at this point in the history

  3. refactor persistent store config

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    0efda7a View commit details
    Browse the repository at this point in the history

  4. make package name shorter in import

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    cfc7c25 View commit details
    Browse the repository at this point in the history

  5. adding a V2 method to existing data sources

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    17a4179 View commit details
    Browse the repository at this point in the history

  6. creating the dataSystem interface

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    ebae2cf View commit details
    Browse the repository at this point in the history

  7. tests pass

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    71e1fb7 View commit details
    Browse the repository at this point in the history

  8. lints

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    e4b2ea7 View commit details
    Browse the repository at this point in the history

  9. add concept of DataStatus, remove need to check initialized

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    e706bd1 View commit details
    Browse the repository at this point in the history

  10. create stub fdv2 data system

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    618609a View commit details
    Browse the repository at this point in the history

  11. add dual-mode store

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    3c6cac4 View commit details
    Browse the repository at this point in the history

  12. refactoring the store component

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    33b0ae8 View commit details
    Browse the repository at this point in the history

  13. comment

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    d814a16 View commit details
    Browse the repository at this point in the history

  14. doc comments

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    8fef482 View commit details
    Browse the repository at this point in the history

  15. copious comments

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    3e93a66 View commit details
    Browse the repository at this point in the history

  16. adding store unit tests

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    a99a329 View commit details
    Browse the repository at this point in the history

  17. use pointer swap to switch stores

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    b4e0e4b View commit details
    Browse the repository at this point in the history

  18. goimports

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    98d660b View commit details
    Browse the repository at this point in the history

  19. more store unit tests

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    6bbf09e View commit details
    Browse the repository at this point in the history

  20. revert changes to StreamingDataSourceBuilder, and make a V2 struct in… 

    …stead
    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    64b8074 View commit details
    Browse the repository at this point in the history

  21. remove now-unnecessary ToSynchronizer converter

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    460a1fb View commit details
    Browse the repository at this point in the history

  22. make v2 data sources implement Synchronizer

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    dac8e9d View commit details
    Browse the repository at this point in the history

  23. ensure closeWhenReady is closed in offline mode

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    bfb1e25 View commit details
    Browse the repository at this point in the history

  24. expose some fdv2 types in datasourcev2, and use them to implement e2e… 

    … unit tests
    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    8579fe4 View commit details
    Browse the repository at this point in the history

  25. break out data destination/status reporter from DataSourceUpdateSink … 

    …implementation:
    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    169df91 View commit details
    Browse the repository at this point in the history

  26. make separate test file for fdv2 e2e tests

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    b6da0da View commit details
    Browse the repository at this point in the history

  27. use top-level cfg.Offline to determine if data system should be enabled

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    7d15862 View commit details
    Browse the repository at this point in the history

  28. add dedicated FDv2 streaming protocol builder for unit tests

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    155b24a View commit details
    Browse the repository at this point in the history

  29. add another unit test

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    543b387 View commit details
    Browse the repository at this point in the history

  30. more fdv2 parity e2e tests

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    d30042d View commit details
    Browse the repository at this point in the history

  31. goimports

    @cwaldren-ld
    cwaldren-ld committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    99cffc2 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. merge dataSystem refactor changes

    @cwaldren-ld
    cwaldren-ld committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    c47eb1b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'v7' into cw/sc-254717/data-system-config

    @cwaldren-ld
    cwaldren-ld committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    7d46d80 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'v7' into cw/sc-254717/data-system-config

    @cwaldren-ld
    cwaldren-ld committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    5ed8607 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. Merge branch 'v7' into cw/sc-254717/data-system-config

    @cwaldren-ld
    cwaldren-ld committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    e260858 View commit details
    Browse the repository at this point in the history

  2. chore: add fdv2 store

    @cwaldren-ld
    cwaldren-ld committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    5f6af74 View commit details
    Browse the repository at this point in the history

  3. remove old comment

    @cwaldren-ld
    cwaldren-ld committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    941fee4 View commit details
    Browse the repository at this point in the history

  4. introduce DataQuality enum

    @cwaldren-ld
    cwaldren-ld committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    d11e039 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. add SetPersist

    @cwaldren-ld
    cwaldren-ld committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    4d03305 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'cw/sdk-557/fdv2-store' into cw/sc-254717/data-system-co… 

    …nfig
    @cwaldren-ld
    cwaldren-ld committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    e28b97f View commit details
    Browse the repository at this point in the history

  3. plumbing the persist bool around the store implementation

    @cwaldren-ld
    cwaldren-ld committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    02f2627 View commit details
    Browse the repository at this point in the history

  4. use persist concept

    @cwaldren-ld
    cwaldren-ld committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    24f380e View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. make store implement FDv2 protocol

    @cwaldren-ld
    cwaldren-ld committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    097429e View commit details
    Browse the repository at this point in the history

  2. remove old data destination interface

    @cwaldren-ld
    cwaldren-ld committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    c72dd6d View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2024

  1. revert chnages to datasource/streaming_data_source_events.go

    @cwaldren-ld
    cwaldren-ld committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    c530451 View commit details
    Browse the repository at this point in the history

  2. use fdv2 types in streaming data source event handler

    @cwaldren-ld
    cwaldren-ld committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    01aa57b View commit details
    Browse the repository at this point in the history

  3. refactor concurrency tests

    @cwaldren-ld
    cwaldren-ld committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    aa67ce4 View commit details
    Browse the repository at this point in the history

  4. optimize comments on store.go

    @cwaldren-ld
    cwaldren-ld committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    2a54265 View commit details
    Browse the repository at this point in the history

  5. add dump method to memory store for atomically getting segments and f… 

    …eatures
    @cwaldren-ld
    cwaldren-ld committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    76da39e View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. inMemoryStore tests for SetBasis/ApplyDelta

    @cwaldren-ld
    cwaldren-ld committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    f4772f6 View commit details
    Browse the repository at this point in the history

  2. add new memorystorev2 package

    @cwaldren-ld
    cwaldren-ld committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    521c1d3 View commit details
    Browse the repository at this point in the history

  3. update tests

    @cwaldren-ld
    cwaldren-ld committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    0d2f8e9 View commit details
    Browse the repository at this point in the history

  4. more unit tests

    @cwaldren-ld
    cwaldren-ld committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    d853f88 View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. chore: introduce memorystorev2

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    3e5a64d View commit details
    Browse the repository at this point in the history

  2. benchmarks

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    84b493c View commit details
    Browse the repository at this point in the history

  3. lints

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    6432563 View commit details
    Browse the repository at this point in the history

  4. use defer pattern for locks

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    746c1a8 View commit details
    Browse the repository at this point in the history

  5. make element matcher consistent

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    b0c8a7c View commit details
    Browse the repository at this point in the history

  6. doc tweaks

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    eaab968 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'cw/sdk-709/memory-store' into cw/sdk-557/fdv2-store

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    fd652d1 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'cw/sdk-557/fdv2-store' into cw/sc-254717/data-system-co… 

    …nfig
    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    4dbfe23 View commit details
    Browse the repository at this point in the history

  9. fix broken tests by refactoring fdv2 data model into single package

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    4392516 View commit details
    Browse the repository at this point in the history

  10. remove some unused interface implementation from fdv2proto.RawEvent

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    47f8748 View commit details
    Browse the repository at this point in the history

  11. refactoring selector

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    c1ae8a4 View commit details
    Browse the repository at this point in the history

  12. add better collection matcher

    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    b755ed9 View commit details
    Browse the repository at this point in the history

  13. Merge branch 'cw/sdk-557/fdv2-store' into cw/sc-254717/data-system-co… 

    …nfig
    @cwaldren-ld
    cwaldren-ld committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    b60b1f9 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. Merge branch 'v7' into cw/sc-254717/data-system-config

    @cwaldren-ld
    cwaldren-ld committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6469c5c View commit details
    Browse the repository at this point in the history

  2. remove obsolete file

    @cwaldren-ld
    cwaldren-ld committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    7008d63 View commit details
    Browse the repository at this point in the history

  3. fixing some lints

    @cwaldren-ld
    cwaldren-ld committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    15168a8 View commit details
    Browse the repository at this point in the history

  4. more lints

    @cwaldren-ld
    cwaldren-ld committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    67a0a7d View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2024

  1. Merge branch 'v7' into cw/sc-254717/data-system-config

    @cwaldren-ld
    cwaldren-ld authored Nov 6, 2024
    Configuration menu
    Copy the full SHA
    d1700f0 View commit details
    Browse the repository at this point in the history