Distributed Marketplace System

Python | TCP Sockets | Multithreading | Distributed State Replication | Concurrency Control

Author: Noel McCarthy

---

Overview

This project implements a fully distributed food marketplace system designed to demonstrate core distributed systems concepts including:

• Multi-Server Communication | Client-Server Architecture | Concurrency & Race Condition Management | Distributed State Replication | Fault Tolerance | Decentralised Coordination | Asynchronous Event-Driven Systems

The system models a realistic decentralised multi-vendor marketplace, where:

• Multiple independent sellers operate as standalone TCP servers

• Multiple buyers connect to all sellers concurrently

• There is no central server

• No seller-to-seller communication exists

• All coordination emerges through buyers aggregating distributed state

Each seller maintains its own inventory, rotates items over time, and broadcasts updates. Buyers act as intelligent clients that consume distributed data streams and maintain a replicated local marketplace view.

This architecture enables:

• True distributed concurrency | Horizontal scalability | Independent node failure | Real-time updates | No single point of failure

---

System Architecture

Architecture Model

• Client-Server Communication (buyer requests, sellers serve)

• Multi-Server System (multiple independent seller servers)

• Decentralised Architecture

• No Central Coordinator

• Distributed State Aggregation at the Client Level

Sellers do not communicate with each other, buyers consume state from all sellers and construct a global view.

• Coordination is emergent by design, not centrally orchestrated

This design models real distributed enviornments such as

• Federated Systems | Multi-Vendor Marketplaces | Decentralised Service Networks | Microservice-like Topologies

---

Core Components

Seller Nodes (TCP Servers)

Each seller runs as a standalone multi-threaded TCP server with responsiblity for:

• Hosting its own item catalogue

• Maintaining inventory state

• Rotating active items at timed intervals

• Accepting multiple concurrent buyer connections

• Processing purchase requests concurrently

• Broadcasting state updates to all buyers

Each seller is an independent system node with its own lifecycle, state, and failure domain.

---

Buyer Nodes (Distributed Clients)

Buyers function as intelligent distributed clients:

• Connect to all configured sellers at startup

• Register with sellers using protocol messages

• Maintain a replicated local marketplace state

• Spawn a listener thread per seller

• Consume push-based updates asynchronously

• Route purchase requests dynamically

• Select optimal sellers based on real-time stock data

Buyers act as distributed state aggregators, integrating multiple independent data streams into a coherent local system view.

Concurrency Model

Concurrency exists at three distinct layers:

1. Seller-Side Concurrency

Each seller is a multi-threaded TCP server. Layers of concurrency:

1a. Per-connection threads

• One thread per buyer connection | No blocking between buyers | Linear scalability with number of buyers

1b. Background item-rotation thread

• Timed item rotation | Automatic stock transitions | Broadcasts state updates | Runs concurrently with buyer operations

1c. Shared resource protection

• threading.Lock protects stock state | Prevents race conditions | Ensures atomic check-then-update | Eliminates TOCTOU errors

This prevents:

• Double-spend

• Negative inventory

• Inconsistent state

• Partial updates

2. Buyer-Side Concurrency

Each buyer is a multi-threaded distributed client:

• One listener thread per seller | One main command thread

This enables:

• Concurrent message processing | Real-time updates | Non-blocking user input | Asynchronous event handling

Buyers continuously receive updates while processing user commands.

3. Distributed Concurrency

Because sellers are independent:

• Multiple sellers update state simultaneously

• Buyers integrate concurrent streams

• No global ordering guarantees

• Fully asynchronous system behavior

This creates true distributed concurrency, not simulated concurrency.

---

Communication Model

All communication uses persistent TCP sockets. Model characteristics:

• Asynchronous | Event-driven | Push-based updates | No polling | No central coordination

Protocol Design

Communication uses a newline-terminated textual protocol, implementing:

• Predictable Structure | Simple Parsing | Extensible | No Partial Reads | Consistent Message Framing

Direction	Message Type	Purpose
SELLER -> BUYER	WELCOME	Inital handshake
SELLER -> BUYER	ITEM, ITEM_CHANGE	Active item updates
SELLER -> BUYER	STOCK	Update stock count
SELLER -> BUYER	SOLD_OUT	Seller has no items left
SELLER -> BUYER	SELLER_LEFT	Shutdown notification
BUYER -> SELLER	JOIN	Register the buyer
BUYER -> SELLER	BUY	Purchase request
BUYER -> SELLER	LEAVE	Disconnect Cleanly

This creates a streaming distributed system, where buyers consume continious state updates from multiple independent sources.

---

Distributed State Model

Buyers maintain a replicated local marketplace state:

{
  seller_id: {
    item,
    stock,
    time_remaining
  }
}

State is updated by

• ITEM broadcasts

• STOCK updates

• SOLD_OUT messages

• SELLER_LEFT events

This allows buyers to computer a global system view without central coordination

Purchase Routing Logic

Buy requests follow a distributed decision model:

1. Buyer requests an item

2. Buyer selects seller with highest stock

3. BUY request sent to selected seller

4. Seller validates stock atomically

5. Transaction completes with BUY_OK or BUY_FAIL

This implements distributed load balancing and conflict minimisation.

Fault Tolerance

The system is naturally fault tolerant by design. Failures are isolated, not systemic.

• Sellers can leave at any time

• Buyers adapt dynamically

• No central failure point

• No global coordinator

• Independent node lifecycles

• Graceful disconnect handling

• Automatic state reconciliation

Directory Structure

.
├── buyer/
│   └── buyer_multi.py
├── config/
│   ├── config_multi.json
│   ├── config_s1.json
│   ├── config_s2.json
│   └── config_s3.json
├── seller/
│   └── seller.py
├── util/
│   └── protocol.py
├── configure_marketplace.ps1
└── README.md

Running The System

Automatic Setup

Open a PowerShell window in the root of the project folder and run the script configure_marketplace.ps1

If you are blocked, bypass execution restrictions using the command below:

powershell -ExecutionPolicy Bypass -File .\configure_marketplace.ps1

3 seller terminal windows and 4 buyer terminal windows will launch, all connections automatically established.

Manual Setup: Step 1 — Start the Seller Servers

Open three PowerShell windows.

In each window, navigate to the project folder then run each seller using its configuration file

Terminal 1:

python -m seller.seller config\config_s1.json

Terminal 2:

python -m seller.seller config\config_s2.json

Terminal 3:

python -m seller.seller config\config_s3.json

Manual Setup: Step 2 — Start the Buyer Clients

Open four PowerShell windows

In each window, navigate to the project folder then start each buyer passing its ID

Terminal 1:

python -m buyer.buyer_multi B1

Terminal 2:

python -m buyer.buyer_multi B2

Terminal 3:

python -m buyer.buyer_multi B3

Terminal 4:

python -m buyer.buyer_multi B4

Terminal Commands

Buyer

Show marketplace state:

• list

Purchase an item:

• buy e.g., buy oil 1

Disconnect gracefully:

• leave

---

Seller

Inside a seller window the following can be ran

Disconnect gracefully:

• leave