Prime Cluster

A distributed system example for efficient prime factorization leveraging Rust, Actix-Web, and parallel computation techniques. This project serves as a practical illustration of building scalable backend services using Rust's concurrency and parallelism features.

Overview

Prime Cluster consists of multiple components that demonstrate how to orchestrate a distributed computational task:

• Load Balancer: Directs incoming requests to the master node.
• Master Node: Receives factorization requests and distributes tasks among worker nodes.
• Worker Nodes: Perform prime factorization tasks in parallel, optimized for CPU (with optional GPU acceleration).

Project Structure

• load_balancer: Rust-based HTTP server directing traffic.
• master: Coordinates and manages task distribution.
• worker: Processes the actual prime factorization.

Technologies Used

• Rust
• Actix-Web (async HTTP framework)
• Rayon (data parallelism)
• Docker & Docker Compose

Setup and Installation

Prerequisites

• Docker
• Docker Compose

Build and Run

Clone the repository:

git clone https://github.com/copyleftdev/prime_cluster.git
cd prime_cluster

Start the services using Docker Compose:

docker-compose up --build

Usage

Send a POST request to the load balancer to compute prime factors:

curl -X POST -H "Content-Type: application/json" \
-d '{"numbers": [1234567, 9876543, 24680, 13579]}' \
http://localhost:9000/compute

Health Checks

Each worker node provides a health check endpoint:

curl http://localhost:8080/health

Components Explained

Load Balancer

• Listens on port 9000
• Forwards incoming factorization requests to the master node

Master Node

• Coordinates factorization requests
• Distributes tasks to available workers
• Listens on port 8081

Worker Nodes

• Perform the prime factorization using Rayon for CPU parallelism
• Optional GPU acceleration support (CUDA)
• Listens on configurable ports (default: 8080, 8090)

Development and Debugging

A verbose debugging script is provided for detailed request tracing:

./test_tracing.sh

Project Layout

prime_cluster/
├── docker-compose.yml
├── load_balancer/
├── master/
└── worker/
    ├── Dockerfile
    ├── Cargo.toml
    ├── bin/gpu_factor.ptx (Optional CUDA GPU support)
    └── src/
        ├── main.rs (Actix-Web server setup)
        ├── prime_factor.rs (CPU prime factorization logic)
        ├── hybrid_factor.rs (Unified CPU/GPU logic)
        └── cuda.rs (GPU factorization using CUDA)

License

This project is open source, licensed under the MIT License. See LICENSE for details.