Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Pocket Dimension #63

Open
dariusc93 opened this issue Oct 23, 2022 · 0 comments
Open

Refactor Pocket Dimension #63

dariusc93 opened this issue Oct 23, 2022 · 0 comments
Labels
enhancement New feature or request improvement Improve on existing feature P-Low Low Priority refactoring Major change to features

Comments

@dariusc93
Copy link
Contributor

dariusc93 commented Oct 23, 2022

Pocket Dimension is a interface that handles caching of data encoded with Sata (originally was Data), which can be used to fetch the data later on. The idea was to reduce the need for recurring calls for the same data, and to provide for quick indexing and searching of data. However, the interface is too simple and has become apparent to be missing functions related storing data such as files (with files it uses a struct to hold/maintain the metadata which may not be apparent to outside use), functions allow caching in a K/V (key/value) format while the underlining data is still stored in the format while also containing a function to store raw encoded data (Sata) directly as it does now, while still maintaining a agnostic nature. This would make the interface friendlier and would extend functionality for different implementation, and in theory improve performance (due to other extensions also being async, this interface should probably do the same to prevent it from blocking any task)

Interface should:

  1. Provides or extends to using key/value. The key parameter should be a string while the value should be in bytes. When fetching, it should return the bytes of that value that the user can handle it accordingly.
  2. Support caching files through either a path, buffer, or stream of data of the file. When fetching the file later on, it can be transferred to a specific destination on disk, buffered into bytes, or but possibly return a handle to stream the data.
  3. Allow inserting Sata directly into the cache, as it does now and fetch it later based on a query/filter.

Additional thoughts:

  1. Should we retain the data type in the cache and treat it as a sort of column for anything being cache? Eg if the data is related to accounts, we can retain DataType::Accounts or Module::Accounts so it is only store specifically for that type.

Note:

  1. It may be ideal not use Sata directly on the interface (and may have to rethink using it internally) due to some noticeable overhead.

Propose interface: TBD

Note: This will get updated overtime.

@dariusc93 dariusc93 added enhancement New feature or request improvement Improve on existing feature refactoring Major change to features P-Low Low Priority labels Oct 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request improvement Improve on existing feature P-Low Low Priority refactoring Major change to features
Projects
None yet
Development

No branches or pull requests

1 participant