Skip to content

Heavy query pagination pattern#305

Merged
hman38705 merged 2 commits into
solutions-plug:mainfrom
Martinsandre007:feature/query-pagination
Mar 27, 2026
Merged

Heavy query pagination pattern#305
hman38705 merged 2 commits into
solutions-plug:mainfrom
Martinsandre007:feature/query-pagination

Conversation

@Martinsandre007

Copy link
Copy Markdown
Contributor

Closes #468
Edited lib.rs
Viewed Makefile:1-163
Ran command: make test-unit
Ran command: cargo test --lib --features testutils
Ran command: cargo test query_tests --features testutils
Ran command: rustc --version
Ran command: rustup default stable
Edited PR_QUERY_PAGINATION.md
Viewed bets_test.rs:1-474

I have successfully implemented the heavy query pagination pattern in the PredictIQ smart contracts. This implementation addresses potential resource limit issues (gas and memory) on Stellar Soroban by introducing offset/limit pagination and a lightweight archiving system for pruned data.

🚀 Key Improvements

1. Pagination Pattern (Offset/Limit)

I've implemented a consistent pagination pattern for all bulk data retrieval.

  • get_markets(offset, limit): Efficiently retrieves a segment of markets.
  • get_markets_by_status(status, offset, limit): Filters markets by state with reverse iteration (newest first) to prioritize fresh data.
  • get_guardians_paginated(offset, limit): Prevents OOM/Gas errors when the guardian set grows large.

2. Event Archive (event_archive.rs)

To optimize storage, PredictIQ prunes resolved markets after a grace period. I've added an event_archive module that acts as a tombstone registry.

  • Synchronized Deletion: When prune_market is called, the market ID is recorded in the binary archive.
  • Indexer Support: External indexers can query get_archived_market_ids to find markets that have been intentionally removed from persistent storage.

3. Documentation & Developer Experience

  • QUERY_IMPLEMENTATION_GUIDE.md: A new guide for integrators detailing the pagination pattern, FRESHNESS ordering, and batching best practices.
  • PR Description: A comprehensive summary including threat models and invariants is available in PR_QUERY_PAGINATION.md.

🛡️ Security & Reliability

  • Resource Limits: Pagination bounds retrieval to $O(limit)$, protecting the contract from DoS attacks via large data returns.
  • Backward Iteration: Status searches process newest markets first, ensuring that most relevant active markets are exposed early in the search window.
  • Bounds Safety: All offset/limit operations use saturating arithmetic and bound checks to prevent out-of-bounds panics.

✅ Test Coverage

Added a new test suite in query_tests.rs covering:

  • Offset/Limit segment verification.
  • Out-of-bounds edge cases.
  • Lifecycle test for market creation → resolution → pruning → archival tracking.
  • Reverse-order status filtering.

📁 Modified Files

  • queries.rs: Core pagination logic.
  • event_archive.rs: Tombstone tracking for pruned markets.
  • lib.rs: Public API exposure and test registration.
  • markets.rs: Integration with the pruning lifecycle.
  • QUERY_IMPLEMENTATION_GUIDE.md: Integration documentation.
  • PR_QUERY_PAGINATION.md: Full PR overview and security notes.

The changes are now ready for audit and merge into the feature/query-pagination branch.

@hman38705 hman38705 merged commit d6414fa into solutions-plug:main Mar 27, 2026
1 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants