15,000+ ops/sec · Strong Consistency · Fault Tolerant · Production Ready
Features · Architecture · Benchmarks · Quick Start · Dashboard · How It Works
distinct-kv is a distributed key-value store engineered from first principles — no off-the-shelf consensus libraries, no shortcuts. It implements the Raft consensus algorithm from scratch in Go to guarantee strong consistency across a multi-node cluster, even in the presence of network partitions and node failures.
Built because I wanted to understand what actually happens inside systems like etcd, Consul, and TiKV — not just use them.
| Feature | Details |
|---|---|
| ⚡ High Throughput | 15,000+ ops/sec sustained across a 5-node cluster |
| 🛡️ Strong Consistency | Raft consensus — every write is committed before acknowledged |
| 🔄 Leader Election | Automatic leader election and failover in < 300ms |
| 📡 gRPC Transport | Efficient binary protocol for inter-node communication |
| 🐳 Docker Ready | Single command spins up a full 5-node cluster |
| 📊 Live Dashboard | Python/Flask dashboard for real-time cluster monitoring |
| 🔁 Log Replication | Append-only log with snapshot compaction |
| 💾 Persistent State | Survives node restarts — state fully recoverable |
| 🧪 Benchmark Suite | Built-in benchmarking tool to stress-test your cluster |
┌─────────────────────────────────────────────────────────┐
│ Client Layer │
│ CLI Tool / Flask Dashboard │
└────────────────────────┬────────────────────────────────┘
│ gRPC
┌────────────────────────▼────────────────────────────────┐
│ Leader Node │
│ ┌─────────────┐ ┌──────────────┐ ┌───────────────┐ │
│ │ Raft Core │ │ Log Manager │ │ State Machine│ │
│ │ (Election) │ │ (Replication│ │ (KV Store) │ │
│ └──────┬──────┘ └──────┬───────┘ └───────────────┘ │
└─────────┼────────────────┼────────────────────────────--┘
│ AppendEntries │ Replicate
┌─────▼──────┐ ┌─────▼──────┐ ┌────────────┐
│ Follower 1 │ │ Follower 2 │ │ Follower 3 │
│ Node │ │ Node │ │ Node │
└────────────┘ └────────────┘ └────────────┘
Quorum = ⌊N/2⌋ + 1 nodes must agree for commit
distinct-kv/
├── raft/
│ ├── node.go # Raft node state machine
│ ├── election.go # Leader election logic
│ ├── log.go # Append-only log + replication
│ ├── snapshot.go # Log compaction
│ └── rpc.go # gRPC AppendEntries / RequestVote
├── store/
│ ├── kv.go # In-memory key-value state machine
│ └── persistence.go # WAL + snapshot recovery
├── server/
│ ├── grpc_server.go # gRPC server (inter-node + client)
│ └── http_server.go # REST API for dashboard
├── dashboard/
│ ├── app.py # Flask dashboard server
│ └── templates/ # Real-time cluster UI
├── benchmark/
│ └── bench.go # Throughput + latency benchmark tool
└── docker/
└── docker-compose.yml # 5-node cluster setup
Tested on: 5-node cluster · Docker · M1 MacBook Pro · 16GB RAM
| Operation | ops/sec | p50 latency | p99 latency |
|---|---|---|---|
| PUT | 15,200 | 0.8ms | 3.2ms |
| GET (Leader) | 28,400 | 0.3ms | 1.1ms |
| GET (Any node) | 41,000 | 0.2ms | 0.9ms |
| Mixed (70% R / 30% W) | 22,800 | 0.5ms | 2.4ms |
| Scenario | Recovery Time | Data Loss |
|---|---|---|
| Leader crash | < 300ms | Zero |
| 1 follower crash (5-node) | Instant | Zero |
| Network partition (minority) | Instant | Zero |
| Full restart (all nodes) | < 2s | Zero |
Quorum math: In a 5-node cluster, the system tolerates 2 simultaneous node failures and still makes progress.
go 1.21+
docker & docker-compose
python 3.9+ (for dashboard)Spin up a full 5-node cluster in one command:
git clone https://github.com/PrajwaL-N-TECHIE/distinct-kv.git
cd distinct-kv
docker-compose up --buildCluster is live at:
- Node 1:
localhost:8001 - Node 2:
localhost:8002 - Node 3:
localhost:8003 - Dashboard:
localhost:5000
# Terminal 1 — Start node 1 (bootstrap leader)
go run main.go --id=1 --port=8001 --peers=localhost:8002,localhost:8003
# Terminal 2 — Start node 2
go run main.go --id=2 --port=8002 --peers=localhost:8001,localhost:8003
# Terminal 3 — Start node 3
go run main.go --id=3 --port=8003 --peers=localhost:8001,localhost:8002# PUT a key
./kv-cli put --key="user:1" --value="prajwal" --addr=localhost:8001
# GET a key
./kv-cli get --key="user:1" --addr=localhost:8001
# DELETE a key
./kv-cli delete --key="user:1" --addr=localhost:8001
# Cluster status
./kv-cli status --addr=localhost:8001# Throughput benchmark — 100K operations
go run benchmark/bench.go --ops=100000 --concurrency=50 --addr=localhost:8001
# Latency test
go run benchmark/bench.go --mode=latency --addr=localhost:8001The Python/Flask dashboard gives you real-time visibility into your cluster:
cd dashboard
pip install -r requirements.txt
python app.py
# Open http://localhost:5000Dashboard shows:
- 🟢 Live node status (Leader / Follower / Candidate)
- 📈 Real-time ops/sec throughput graph
- 📋 Raft log viewer — see every entry being replicated
- ⚡ Current term and commit index
- 🔄 Leader election events
Raft solves the distributed consensus problem in three parts:
All nodes start as FOLLOWERS
↓
Election timeout fires (150-300ms randomized)
↓
Node becomes CANDIDATE → sends RequestVote RPCs
↓
Majority votes received → becomes LEADER
↓
Leader sends heartbeats every 50ms to prevent new elections
Client sends PUT request to Leader
↓
Leader appends entry to its log (uncommitted)
↓
Leader sends AppendEntries RPC to all followers in parallel
↓
Majority (3/5 nodes) acknowledge → entry COMMITTED
↓
Leader applies to state machine → responds to client
↓
Followers apply on next heartbeat
A log entry is only ever committed if it is stored on a majority of nodes. This means:
- Even if the leader crashes immediately after commit, the entry survives
- No two leaders can exist in the same term
- Committed entries are never lost
Why Go? Goroutines make concurrent RPC handling trivial. Each node's election timer, heartbeat sender, and RPC handlers run as independent goroutines with channel-based coordination — no callback hell.
Why gRPC over REST? Inter-node communication needs to be fast and typed. Protobuf serialization is ~5x faster than JSON for the high-frequency AppendEntries RPCs that happen every 50ms per follower.
Why randomized election timeouts? Without randomization, all nodes would call an election simultaneously and split votes forever. Randomizing between 150-300ms ensures one node almost always wins the first election.
Log compaction via snapshots: Without compaction, the Raft log grows forever. Once the log exceeds a threshold, distinct-kv takes a snapshot of the current state machine, writes it to disk, and truncates all log entries before the snapshot index.
| Layer | Technology |
|---|---|
| Language | Go 1.21 |
| Consensus | Raft (implemented from scratch) |
| RPC | gRPC + Protocol Buffers |
| Storage | In-memory + WAL for persistence |
| Dashboard | Python 3.11 + Flask |
| Containerization | Docker + Docker Compose |
| Testing | Go testing + testify |
- Raft Paper — In Search of an Understandable Consensus Algorithm — Ongaro & Ousterhout, 2014
- Raft Visualization — Interactive demo that helped me debug election logic
- Designing Data-Intensive Applications — Martin Kleppmann — Chapter 9 was invaluable
- etcd — Production Raft implementation I studied for reference
- Raft leader election
- Log replication
- Log compaction (snapshots)
- gRPC transport
- Flask dashboard
- Docker compose cluster
- TLS encryption for inter-node communication
- Client-side load balancing
- Membership changes (add/remove nodes dynamically)
- Linearizable reads from followers
PRs welcome! Especially for items on the roadmap above.
git clone https://github.com/PrajwaL-N-TECHIE/distinct-kv.git
cd distinct-kv
go test ./... # run all testsPrajwal N — AI Engineer Intern @ InFynd | Chief Architect @ EduSpine
12x National Tech Winner building at the intersection of distributed systems and AI infrastructure.
If this project helped you understand distributed systems, drop a ⭐ — it helps others find it too!
Built from scratch. No shortcuts. Just Go, Raft, and stubbornness. 🔥