Traffic Light Optimization using Deep Q-Learning

An intelligent traffic signal control system that uses Deep Reinforcement Learning to optimize traffic flow at intersections. The AI agent learns to minimize vehicle waiting times through trial and error, adapting to different traffic patterns.

🎯 Overview

Traditional traffic lights operate on fixed timers, causing unnecessary delays and congestion. This project implements an adaptive traffic control system using:

• Deep Q-Learning: Reinforcement learning algorithm for decision making
• Neural Networks: 5-layer deep network with 400 neurons per layer
• SUMO Simulator: Realistic traffic simulation environment
• Experience Replay: Stable learning from past experiences

Key Features

• ✅ Real-time Decision Making: AI observes traffic and adapts signal timings
• ✅ Learning from Experience: Improves over 100+ training episodes
• ✅ Smart Traffic Management: Minimizes cumulative waiting time
• ✅ Visualization: Training metrics and testing results with graphs

---

🚀 Quick Start

Prerequisites

1. Python 3.12+ - Download
2. SUMO 1.24+ - Download
3. Git - Download

Installation

# Clone the repository
git clone https://github.com/sumitsingh3072/Traffic-Light-Optimization.git
cd Traffic-Light-Optimization

# Create virtual environment
python -m venv .venv

# Activate virtual environment
# Windows:
.venv\Scripts\activate
# Linux/Mac:
source .venv/bin/activate

# Install dependencies
pip install -r Requirements.txt

# Setup SUMO environment
# Windows: Set SUMO_HOME and add to PATH
setx SUMO_HOME "C:\Program Files (x86)\Eclipse\Sumo"
# Linux/Mac: Add to ~/.bashrc or ~/.zshrc
export SUMO_HOME="/usr/share/sumo"
export PATH="$SUMO_HOME/bin:$PATH"

Training the Model

cd TCLS
python training_main.py

Training Configuration (training_settings.ini):

• Episodes: 100 (customizable)
• Duration: ~6 hours for 100 episodes
• GUI: Disabled by default (set gui = True to visualize)

Testing the Model

cd TCLS
python testing_main.py

Testing Configuration (testing_settings.ini):

• Set model_to_test = X (model number to test)
• GUI enabled by default
• Single episode evaluation

---

📊 How It Works

The AI Agent

State Space (80 dimensions):

• 4 incoming directions (North, South, East, West)
• Each direction divided into 20 position cells
• Binary representation: 1 = vehicle present, 0 = empty
• Captures vehicle positions up to 750m from intersection

Action Space (4 discrete actions):

• Action 0: North-South Green (straight/right turns)
• Action 1: North-South Left Green (left turns only)
• Action 2: East-West Green (straight/right turns)
• Action 3: East-West Left Green (left turns only)

Each green phase lasts 10 seconds, with 4-second yellow transitions.

Reward Function:

reward = previous_cumulative_waiting_time - current_cumulative_waiting_time

• Positive reward → Reduced waiting time (good decision)
• Negative reward → Increased waiting time (poor decision)

Neural Network Architecture

Input Layer:    80 neurons (state representation)
Hidden Layer 1: 400 neurons (ReLU activation)
Hidden Layer 2: 400 neurons (ReLU activation)
Hidden Layer 3: 400 neurons (ReLU activation)
Hidden Layer 4: 400 neurons (ReLU activation)
Hidden Layer 5: 400 neurons (ReLU activation)
Output Layer:   4 neurons (Q-values for each action)

Learning Algorithm: Deep Q-Learning with Experience Replay

Q(s,a) = reward + γ × max Q(s',a')

Where:

• γ = 0.75 (discount factor)
• Experience replay buffer: 50,000 samples
• Batch size: 100
• Training epochs per episode: 800

---

📁 Project Structure

Traffic-Light-Optimization/
├── TCLS/                          # Main source code
│   ├── training_main.py          # Training entry point
│   ├── testing_main.py           # Testing entry point
│   ├── model.py                  # Neural network definition
│   ├── training_simulation.py    # Training simulation logic
│   ├── testing_simulation.py     # Testing simulation logic
│   ├── generator.py              # Traffic generation
│   ├── memory.py                 # Experience replay buffer
│   ├── utils.py                  # Helper functions
│   ├── visualization.py          # Plotting utilities
│   ├── training_settings.ini     # Training configuration
│   ├── testing_settings.ini      # Testing configuration
│   ├── intersection/             # SUMO simulation files
│   │   ├── environment.net.xml   # Road network definition
│   │   ├── episode_routes.rou.xml # Generated vehicle routes
│   │   └── sumo_config.sumocfg   # SUMO configuration
│   └── models/                   # Saved trained models
│       └── model_X/
│           ├── trained_model.h5  # Trained neural network
│           └── test/             # Test results
├── .venv/                        # Virtual environment
├── Requirements.txt              # Python dependencies
├── LICENSE                       # MIT License
├── README.md                     # This file
└── DOCUMENTATION.md              # Detailed documentation

---

🎓 Technical Details

Hyperparameters

Parameter	Value	Description
Learning Rate	0.001	Adam optimizer learning rate
Discount Factor (γ)	0.75	Future reward discount
Epsilon Decay	Linear (1.0 → 0.0)	Exploration rate
Memory Size	50,000	Experience replay buffer
Batch Size	100	Training samples per batch
Training Epochs	800	Per episode training iterations

Performance Metrics

• Cumulative Negative Reward: Total waiting time penalty
• Average Queue Length: Mean number of waiting vehicles
• Cumulative Delay: Total seconds all vehicles waited

---

📈 Results

After 100 training episodes:

• ✅ Learned optimal traffic light control strategies
• ✅ Reduced average waiting time by 20-30% vs fixed timing
• ✅ Adapted to varying traffic patterns
• ✅ Made decisions in 30-50ms (real-time capable)

View results in TCLS/models/model_X/test/:

• plot_reward.png - Reward progression
• plot_queue.png - Queue length over time

---

🛠️ Configuration

Training Settings (training_settings.ini)

[simulation]
gui = False              # Enable/disable visualization
total_episodes = 100     # Number of training episodes
max_steps = 5400        # Steps per episode
n_cars_generated = 1000 # Traffic density

[model]
num_layers = 4          # Hidden layers (+ 1 = 5 total)
width_layers = 400      # Neurons per layer
batch_size = 100        # Training batch size
learning_rate = 0.001   # Adam optimizer rate
training_epochs = 800   # Training iterations/episode

[agent]
gamma = 0.75           # Discount factor

Testing Settings (testing_settings.ini)

[simulation]
gui = True             # Enable visualization
model_to_test = 5     # Which model to test

[agent]
episode_seed = 10000  # Random seed for reproducibility

---

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

🙏 Acknowledgments

• SUMO - Simulation of Urban MObility
• TensorFlow - Machine Learning framework
• Deep Q-Learning - Based on DeepMind's DQN research

---

📚 Further Reading

For detailed documentation, see DOCUMENTATION.md

For questions or issues, please open an issue on GitHub.