TopLoc Validator

An inference validation server with a REST API interface.

Installation

Local Development

uv venv .venv
source .venv/bin/activate  # On Unix
uv pip install -e .

Docker

docker build -t toplocvalidator .
docker run -p 8000:8000 toplocvalidator

Quick Start

Pull and run the Docker image:

# Pull the latest image
docker pull primeintellect/toplocvalidator:latest

# Set the GCP credentials
export GCP_CREDENTIALS=ewog...fQo=

# Run the server to validate DeepSeek-R1-Distill-Qwen-1.5B on the toploc-testing/run0 GCP bucket
docker run --gpus all -p 8000:8000 \
	-e GCP_CREDENTIALS=$GCP_CREDENTIALS \
	primeintellect/toplocvalidator:latest \
	--model-path deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
	--gcp-path toploc-testing/run0

Environment Variables

The following environment variables can be used to configure the server:

• SKIP_FILESHA_CHECK: Skip SHA verification of files (default: false)
• TMP_DIR: Directory for temporary files (default: /tmp/toplocvalidator)
• DISABLE_CACHING: Disable caching of validation results (default: false)
• USE_STEP_RELOAD: Enable step-by-step model reloading (default: false)
• TOPLOCVALIDATOR_AUTH_TOKEN: Authentication token for API requests (default: empty)

Usage

Running the Server

Local Development

toplocvalidator [--debug-validation-time SECONDS] [--port PORT]

Options:

• --debug-validation-time: Set validation simulation time (optional, uses real validation if not set)
• --port: Set server port (default: 8000)
• --model-path: Path to the model to use for validation (default: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)
• --gcp-path: GCP bucket path for file storage (default: toploc-testing/run0)
• --config-path: Path to custom validation config file (optional, uses built-in config if not set)

Docker

docker run -p 8000:8000 \
	-e GCP_CREDENTIALS=$GCP_CREDENTIALS \
	toplocvalidator \
	--model-path deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
	--gcp-path toploc-testing/run0

API Endpoints

Start Validation

POST /validate/{filepath}
Content-Type: application/json

{
    "file_sha": "sha256_hash_of_file"
}

Response (Success):

{
    "message": "Validation started for {filepath}"
}

Response (Server Busy - 429):

{
    "detail": "Server is busy processing {current_filepath}. Please retry later."
}

Note: The server processes one validation at a time. If a validation request is made while another validation is in progress, the server will respond with a 429 (Too Many Requests) status code.

Check Validation Status

GET /status/{filepath}

Response:

{
    "status": "pending|accept|reject",
    "is_current": true|false
}

Health Check

GET /

Response:

{
    "message": "File Validation Server"
}