Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Chain Indexer Performance #738

Merged
merged 14 commits into from
Nov 7, 2023
Merged

Conversation

jhbertra
Copy link
Contributor

@jhbertra jhbertra commented Oct 27, 2023

  • Add marlowe-chain-copy executable
  • Improve throughput of marlowe-chain-indexer
  • Remove asset table and inline policy IDs and token names in assetOut and assetMint tables
  • Extract database row mapping functionality to reusable modules delineated by era.

How this was tested:

  • tables exported to csv files from qa preprod database using COPY
  • tables exported to csv files from local database using COPY
  • csv files diffed.
  • Issues identified and fixed using this technique:
    • erroneous use of base58 encoding of bootstrap (byron) addresses
    • inconsistent CBOR encoding of transaction metadata

Pre-submit checklist:

  • Branch
    • Tests are provided (if possible - see comment)
    • Test report is updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
    • Relevant tickets are mentioned in commit messages
    • Formatting, PNG optimization, etc. are updated
    • Operables are updated with changes to executable command line options.
    • Deploy charts updated with changes to operables.
  • PR
    • Self-reviewed the diff
    • Useful pull request description
      • Review required
      • Substantial changes to code, test, or documentation
      • Change made to Marlowe validator (@bwbush and @palas must be included as reviewers)
      • Review not required
      • Minor changes to non-critical code, documentation, nix derivations, configuration files, or scripts
      • Formatting, spelling, grammar, or reorganization
    • Reviewer requested

@jhbertra jhbertra force-pushed the plt-8242-chain-insert-performance branch from bf6f325 to fa870fd Compare November 2, 2023 15:33
@jhbertra jhbertra changed the title WIP Improve Chain Indexer Performance Nov 2, 2023
@jhbertra jhbertra force-pushed the plt-8242-chain-insert-performance branch from fa870fd to d7888ea Compare November 6, 2023 15:15
@jhbertra jhbertra marked this pull request as ready for review November 7, 2023 12:37
@jhbertra jhbertra self-assigned this Nov 7, 2023
@jhbertra jhbertra merged commit e1c6c1a into main Nov 7, 2023
5 checks passed
@jhbertra jhbertra deleted the plt-8242-chain-insert-performance branch November 7, 2023 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant