perf: fix cardano blocks with transactions retrieval in mainnet network

[Alenar]

Content

This PR includes a performance fix for the GetCardanoBlockTransactionsQuery to guarantee that it use indexes in the large mainnet dataset.

We found that the SQLite query planner is not able to compute a fast plan for this query on large dataset, even after running pragma optimize, the available index is ignored and a full scan is performed ... which is extremely slow with a 30GB database, taking seconds to retrieve a single row (and we needs millions of them).

export QUERY="select * from cardano_block left join cardano_tx on cardano_block.block_hash = cardano_tx.block_hash where (block_number >= 4007114 and block_number < 4007129) group by cardano_block.block_hash"
sqlite3 -table ~/data/mainnet/mithril-aggregator/mithril/stores/cardano-transaction.sqlite3 'explain query plan $QUERY'
QUERY PLAN
|--SCAN cardano_block USING INDEX sqlite_autoindex_cardano_block_1
`--SEARCH cardano_tx USING COVERING INDEX block_hash_transaction_hash_index (block_hash=?)

Forcing the planner to use the index fix the issue:

export QUERY="select * from cardano_block indexed by cardano_block_block_number_index left join cardano_tx on cardano_block.block_hash = cardano_tx.block_hash where (block_number >= 4007114 and block_number < 4007129) group by cardano_block.block_hash"
sqlite3 -table ~/data/mainnet/mithril-aggregator/mithril/stores/cardano-transaction.sqlite3 'explain query plan $QUERY'
QUERY PLAN
|--SEARCH cardano_block USING INDEX cardano_block_block_number_index (block_number>? AND block_number<?)
|--SEARCH cardano_tx USING COVERING INDEX block_hash_transaction_hash_index (block_hash=?)
`--USE TEMP B-TREE FOR GROUP BY

Caution

This use the indexed by SQLite clause which is not recommended under normal circumstances to tune performance, but we did not find alternatives, even running a large ANALYZE (with a analysis_limit of 100 000 rows) did not fix the plan.

Pre-submit checklist

• Branch
  • Crates versions are updated (if relevant)
  • Commit sequence broadly makes sense
  • Key commits have useful messages
• PR
  • All check jobs of the CI have succeeded
  • Self-reviewed the diff
  • Useful pull request description
  • Reviewer requested
• Documentation
  • No new TODOs introduced

Issue(s)

Relates to #3050

[github-actions]

Test Results

    5 files  ±0    190 suites  ±0   57m 21s ⏱️ + 1m 25s
2 671 tests ±0  2 671 ✅ ±0  0 💤 ±0  0 ❌ ±0 
9 586 runs  ±0  9 586 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit c09584e. ± Comparison against base commit b114b4e.

♻️ This comment has been updated with latest results.

[Copilot]

Pull request overview

This PR addresses a SQLite query planner regression on large (mainnet) datasets by forcing index usage for the GetCardanoBlockTransactionsQuery, avoiding full scans on a ~30GB database during high-volume block/transaction retrieval.

Changes:

• Force SQLite to use cardano_block_block_number_index via INDEXED BY in the block+transactions retrieval query.
• Document the hard dependency between the query and the index name in the migration SQL.
• Bump mithril-persistence crate version.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

File	Description
internal/mithril-persistence/src/database/query/cardano_block/get_cardano_block_with_transactions.rs	Adds INDEXED BY cardano_block_block_number_index to guarantee indexed access by block number range.
internal/mithril-persistence/src/database/cardano_transaction_migration.rs	Documents that the cardano_block_block_number_index name is coupled to the query and must not be removed/renamed without updating it.
internal/mithril-persistence/Cargo.toml	Bumps crate version to 0.2.68.
Cargo.lock	Updates locked version for mithril-persistence to 0.2.68.

[jpraynaud]

LGTM