Skip to content

Latest commit

 

History

History
109 lines (77 loc) · 2.73 KB

README.md

File metadata and controls

109 lines (77 loc) · 2.73 KB

Prime Cluster

A distributed system example for efficient prime factorization leveraging Rust, Actix-Web, and parallel computation techniques. This project serves as a practical illustration of building scalable backend services using Rust's concurrency and parallelism features.

Overview

Prime Cluster consists of multiple components that demonstrate how to orchestrate a distributed computational task:

  • Load Balancer: Directs incoming requests to the master node.
  • Master Node: Receives factorization requests and distributes tasks among worker nodes.
  • Worker Nodes: Perform prime factorization tasks in parallel, optimized for CPU (with optional GPU acceleration).

Project Structure

  • load_balancer: Rust-based HTTP server directing traffic.
  • master: Coordinates and manages task distribution.
  • worker: Processes the actual prime factorization.

Technologies Used

  • Rust
  • Actix-Web (async HTTP framework)
  • Rayon (data parallelism)
  • Docker & Docker Compose

Setup and Installation

Prerequisites

  • Docker
  • Docker Compose

Build and Run

Clone the repository:

git clone https://github.com/copyleftdev/prime_cluster.git
cd prime_cluster

Start the services using Docker Compose:

docker-compose up --build

Usage

Send a POST request to the load balancer to compute prime factors:

curl -X POST -H "Content-Type: application/json" \
-d '{"numbers": [1234567, 9876543, 24680, 13579]}' \
http://localhost:9000/compute

Health Checks

Each worker node provides a health check endpoint:

curl http://localhost:8080/health

Components Explained

Load Balancer

  • Listens on port 9000
  • Forwards incoming factorization requests to the master node

Master Node

  • Coordinates factorization requests
  • Distributes tasks to available workers
  • Listens on port 8081

Worker Nodes

  • Perform the prime factorization using Rayon for CPU parallelism
  • Optional GPU acceleration support (CUDA)
  • Listens on configurable ports (default: 8080, 8090)

Development and Debugging

A verbose debugging script is provided for detailed request tracing:

./test_tracing.sh

Project Layout

prime_cluster/
├── docker-compose.yml
├── load_balancer/
├── master/
└── worker/
    ├── Dockerfile
    ├── Cargo.toml
    ├── bin/gpu_factor.ptx (Optional CUDA GPU support)
    └── src/
        ├── main.rs (Actix-Web server setup)
        ├── prime_factor.rs (CPU prime factorization logic)
        ├── hybrid_factor.rs (Unified CPU/GPU logic)
        └── cuda.rs (GPU factorization using CUDA)

License

This project is open source, licensed under the MIT License. See LICENSE for details.