Skip to content

Document skip transactions feature #600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/indexer/build/optimisation.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ If you're looking for advice on how to run high performance SubQuery infrastruct
- Use a convenient `modulo` filter to run a handler only once to a specific block. This filter allows handling any given number of blocks, which is extremely useful for grouping and calculating data at a set interval. For instance, if modulo is set to 50, the block handler will run on every 50 blocks. It provides even more control over indexing data to developers.
- For all major chains we already [provide valid SubQuery dictionaries](https://github.com/subquery/templates/blob/main/dist/dictionary.json), but if you’re indexing a custom chain, you may want to implement your own [dictionary](../academy/tutorials_examples/dictionary.html#how-does-a-subquery-dictionary-work) project to speed up your indexer (we can help create one for your new network). You can see examples of how to create a dictionary in the [dictionary repository](https://github.com/subquery/subql-dictionary).
- Use the strictest possible filter conditions in your mapping handlers (within the project manifest) to reduce the number of events/transactions that need to be processed. Create filters as specific as possible to avoid querying unnecessary data.
- If your project contains only event/log handlers and you don't need the transactions data, you can use the `skipTransactions` flag which will skip fetching the transactions. (Only supported with Ethereum and Substrate SDKs).
- Set the start block in your project manifest to when the contract was initialised or, better yet, when the first event/transaction occurs.
- Use `node worker threads` to move block fetching and block processing into its own worker thread. It could speed up indexing by up to 4 times (depending on the particular project). You can easily enable it using the `-workers=<number>` flag. Note that the number of available CPU cores strictly limits the usage of worker threads. [Read more here](../run_publish/references.html#w-workers).

Expand Down
11 changes: 10 additions & 1 deletion docs/indexer/run_publish/references.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,11 +140,12 @@ subql-node -f . --db-schema=test2

### -f, --subquery

**Boolean** - Use this flag to start the SubQuery project.
**String (default: current working directory)** - The path to the subquery project directory or project manifest.

```shell
subql-node -f . // OR
subql-node --subquery .
subql-node -f ~/Path/To/Your/Project
```

### force-clean
Expand Down Expand Up @@ -300,6 +301,10 @@ subql-node reindex -f /example/subql-project --targetHeight=30
Once the command is executed and the state has been rolled back to the specified height, the application will exit. You can then start up the indexer to proceed again from this height.
:::

### --skip-transactions

**Boolean** - This flag allows you to skip fetching transactions when indexing in order to increase speed. This is useful when you only need to index events/logs. Only supported with Ethereum and Substrate SDKs.

### --scale-batch-size

**Boolean** - Scale the block fetch batch size with memory usage.
Expand All @@ -320,6 +325,10 @@ Once the command is executed and the state has been rolled back to the specified

**Positive Integer (default: `5`)** - The interval, in seconds, at which data is flushed from the cache. This ensures that data is persisted regularly when there is either not much data or the project is up to date.

### --store-cache-target

**Positive Integer (default: `10`)** - When the indexer is within this many blocks of the chain head, then data is flushed after indexing every block. This makes data available to be queried quicker.

### --subscription

**Boolean** - This will create a notification trigger on entity, this also is the prerequisite to enable subscription feature in query service. You should also enable [--subscription for the query service](#subscription-1).
Expand Down