Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement batch data fetching and error handling in DataConnector #482

Merged

Conversation

MrRoudyk
Copy link
Contributor

Closes #481

Improvements to data fetching:

  • apps/dashboard_app/data_connector.py: Modified the fetch_data method to support batch fetching with a new batch_size parameter. This allows for more efficient data retrieval by paginating the results.
  • apps/dashboard_app/data_connector.py: Added a new method fetch_protocol_last_block_number to fetch the last block number for a specific protocol, enhancing the flexibility of the data connector.

Enhancements to error handling:

  • apps/dashboard_app/helpers/load_data.py: Wrapped the data fetching and processing logic in a try block within the _init_zklend_state method to handle potential exceptions and log errors. This improves the robustness of the data initialization process.

These changes collectively enhance the efficiency, flexibility, and reliability of the data handling processes within the application.

@MrRoudyk MrRoudyk marked this pull request as ready for review February 26, 2025 21:51
@djeck1432 djeck1432 self-requested a review February 27, 2025 07:31
Copy link
Collaborator

@djeck1432 djeck1432 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, add first/last block number method to use instead of while True
so you will go through range of block numbers

Copy link
Collaborator

@djeck1432 djeck1432 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, now looks better

@djeck1432 djeck1432 merged commit 1ca7763 into CarmineOptions:master Feb 28, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Dashboard App] Optimize loading loan_states from DB
2 participants