VeWorld Indexer

• Coverage Reports:

  • API
  • Common
  • Indexer

• Tests Results:

  • API
  • Common
  • Indexer
  • E2E

Prerequisites

• Docker
• Java (v21)
• Go (>= 1.26) — required to build tools/thor-scheduler, the Go co-process used by the Validator indexer for PoS schedule reconstruction. make build invokes tools/thor-scheduler/Makefile, which looks for go on PATH, then under ~/sdk/go*/bin/go (the golang.org/dl/goX.Y SDK layout), then /opt/homebrew/bin/go, then /usr/local/go/bin/go. Override with GO=/path/to/go if needed. Docker-only workflows (make start, make build-image, make test-e2e) do not require a local Go install — the multi-stage Dockerfile builds the binary inside golang:1.26-alpine.

GitHub Packages Authentication (optional)

Some dependencies (e.g. org.vechain:indexer-core) are hosted on GitHub Packages, which requires authentication even for public repositories. If your build fails to resolve these dependencies, create a GitHub Personal Access Token (classic) with the read:packages scope and add the following to ~/.gradle/gradle.properties:

gpr.user=your-github-username
gpr.key=your-github-pat

Getting Started

• To see a list of all available commands, run make help
• After starting the application, the swagger will be made available at http://localhost:8080

Option 1: Docker only

• Copy env files for the two packages ./package/<package>/.env.example to ./package/<package>/.env and fill in the values for your environment. They should work as-is for docker

• Run:

make start

Option 2: Docker + IntelliJ (Recommended for debugging)

• No need to copy the environment files if you are using IntelliJ. The default variables should connect to the infrastructure using localhost variables
• Run:

make db-all

Connecting to MongoDB

Connect for the various users with the following URIs:

• indexer - mongodb://indexer:password@localhost:27017/vechain?directConnection=true&authMechanism=DEFAULT&authSource=admin
• api - mongodb://api:password@localhost:27017/vechain?directConnection=true&authMechanism=DEFAULT&authSource=admin
• root - mongodb://root:password@localhost:27017/admin?directConnection=true&authMechanism=DEFAULT
• Go to IndexerApplication.kt inside IntelliJ and run/debug:

Restarting

• Clean and restart the DB:

make db-all

Backup MongoDB

You can back up the database by running the following command:

make db-backup

• Will back up the vechain database from localhost:27017
• The backup will be stored in the database/backups/ directory.
• The filename follows this format: database/backups/vechain-YYYYMMDDHHMMSS

You can also specify the host and port of the MongoDB instance you want to backup:

make db-backup MONGO_HOST=my-mongo-host:32423

Restore MongoDB from backup

You can restore the database by running the following command:

make db-restore

• If no backup exists, you will be prompted to specify a backup directory.
• By default, it restores from the latest backup found in the database/backups/ directory.

To restore from a specific backup folder, specify DIR:

make db-restore DIR=backup/mydatabase-20250210

To restore to a different DB:

make db-restore MONGO_HOST=myserver.com

Copy collections between databases

To copy specific collections from one MongoDB cluster to another (e.g. from a local indexer into an Atlas cluster) without doing a full backup/restore round-trip:

make db-copy-collections

This launches the interactive wrapper at database/restore/restore.sh, which prompts for source/destination URIs (with MONGO_PRESET_* env-var presets) and the collection list, then drives the underlying restore_local_dump.sh end-to-end. See database/restore/README.md for full options including non-interactive use.

Features

There are 6 indexers and 6 corresponding APIs. Each indexer can be run in isolation or all together. There is no dependency between indexers for this reason. Each indexer and API pair can be enabled using the corresponding spring profile.

• transactions - enabled with transactions profile
• nfts - enabled with nfts profile
• transfers - enabled with transfers profile
• history - enabled with history profile

As you can see from the list above, the block indexer offers the option to proxy to the Thor node. This is useful if you want the convenience of the Block endpoints without the overhead of indexing the data.

Pruner

Some of the indexers are stateful indexers. This means that records are updated with each block. In order to facilitate rollbacks we must store all previous version of each record. These records are stored in collections with a -archives postfix. As you might imagine these archive collections can get rather large over time. To prevent the collection from blowing up we have implemented an optional pruner service that can be enabled and configured with the following env variablers.

• PRUNER_INTERVAL - How frequently to run the pruner (in blocks)
• PRUNER_REMOVAL_CHUNK_SIZE - Sometimes the number of records to prune can be very large. To prevent mongoDB from blowing up we can set a chunk size for the delete operation
• PRUNER_RECORD_LIMIT - You can set a limit on the number of records to prune in each run.

Testing

• Run all the tests:

make test

Package Testing

• There are 4 packages that can be tested (api, common, e2e, indexer)
• This will run all tests in a package (unit, integration and E2E)
• Run (example for api):

make test-api
make test-common
make test-indexer
make test-e2e

E2E

• Running E2E tests will spin up all the docker infrastructure before the test and tears it down after completion. This is useful for testing the entire system, but it is slow. If you need to debug these tests, it is recommended to spin up the network manually using make clean start and remove the tasks preE2e and postE2e tasks in ./packages/e2e/build.gradle.kts (here)

API Schema Tests

Use the schema-driven harness when you need to validate the public API in a deployed environment. It fetches the OpenAPI document from /api-docs, runs Schemathesis-based checks across every documented operation, and fails if any response exceeds 2 000 ms (override by setting MAX_RESPONSE_MILLISECONDS).

• Local run: packages/api/scripts/run_api_schema_tests.sh [dead|live|<base-url>] [--base-url <base-url>] (defaults to the dead environment). Install schemathesis locally first, for example pip install "schemathesis>=3.19,<4".
• GitHub Action: trigger API Tests from the Actions tab and choose the target environment, or provide a full base URL override. The workflow uses the same script and publishes Schemathesis logs and JUnit XML as artifacts.

Disaster Recovery

MongoDB Atlas snapshots are automatically exported daily to S3 (veworld-indexer-atlas-backups). In the event of data loss or cluster corruption where Atlas native snapshot restores are unavailable, the indexer can be restored from these S3 exports.

The restore process involves downloading the gzipped JSON exports to an EC2 instance, streaming them into the target Atlas cluster via parallel mongoimport processes, and rebuilding indexes from the exported metadata files.

For the full step-by-step runbook, see: Disaster Recovery Runbook

Dead Prod Atlas Restore Workflow

For Atlas-native recovery of the current dead prod color, use the Restore Dead Prod From Atlas Snapshots workflow. It restores both dead-prod Atlas clusters from the latest completed snapshots belonging to the current live prod color.

This workflow does not stop or start ECS services. The dead environment must already be quiesced before restore. Use the Stop or Start Dead Prod Environment Services workflow first, then run the restore, then start services again once the restore summary looks correct.

Recommended sequence:

1. Run Stop or Start Dead Prod Environment Services with stop.
2. Run Restore Dead Prod From Atlas Snapshots.
3. Review the restore summary artifact and Atlas restore job IDs.
4. Run Stop or Start Dead Prod Environment Services with start.
5. Run the schema or regression tests against the dead environment before switching traffic.

Deployment & Testing

The VeWorld Indexer can be deployed via two strategies: Regular or Blue/Green. To trigger a deployment, run the Prod Deployment Workflow. You will be prompted to select the deployment strategy and the version number. Please enter a version in the format major.minor.patch - this will be used to create a new release & tag. If in doubt about which environment is currently live, run the Identify Live/Dead Environments workflow with the default arguments.

Regular Deployment

Selecting the regular deployment strategy will trigger a deployment to the current live production environment. Most deployments will follow this process. Ensure any changes being deployed via this strategy have been sufficiently tested before triggering (testing process described below).

Blue/Green Deployment

For code changes requiring a full reindex from the genesis block, deploy via the blue/green strategy. This will trigger a deployment to the dead color. Following deployment, a sufficient amount of time will need to be left (ie a few days) until the database has caught up with the latest block. After this point, the live environment can be switched from the current live color to the dead color. To do this, run the Switch Live Environment workflow. This will update the appropriate DNS records to redirect traffic to the alternate color environment.

Following the DNS switch, please wait at least 48 hours before tearing down the old live (now dead) environment. This is to allow any remote DNS caches to update to the new live environment.

Testing

Since blue/green deployments are fairly infrequent, the dead color can be used as a transient testing environment when needed. This is preferable to using the dev environment because the dead color is an exact replica of the live environment, whereas dev is a more lightweight, stripped-down version, missing some key components like a mongo atlas cluster. Deployment to the dead color will be performed automatically on merge to main, as long as the dead environment already exists. The environment will therefore need to be deployed manually (either by deploying the terraform locally or by triggering a Prod Deployment of the dead color).

Environment Tear-down

When testing is complete, or when a DNS switch has migrated traffic from one environment to the other, the dead environment can be safely torn down until needed again. To do this, run the Cluster Destroy workflow, and select the appropriate environment when prompted.

Further details on the CICD process can be found here