Trip Matrix Summation Tool

A Python tool for aggregating TfWM (Transport for West Midlands) trip matrices by mode type across multiple trip purposes.

Overview

This tool processes trip matrix data stored as CSV files, where each file represents a 24-hour origin-destination (OD) person trip matrix for a specific trip purpose and mode. The matrices contain trip numbers across 900+ zones in the TfWM area.

The tool recursively searches for matrix files and sums them by mode type (e.g., Car Passenger, Bus, Train, etc.) across all trip purposes (e.g., Commute, Home Business, Shopping, etc.), producing consolidated TOTAL_24Hrtours_{ModeType}.csv files.

Data Structure

Input Files

Matrix CSV files follow the naming pattern: {TripPurpose}_24Hrtours_{ModeType}.csv

Example files:

• Com_24Hrtours_CP.csv - Commute trips by Car Passenger
• HBiz_24Hrtours_Bus.csv - Home Business trips by Bus
• HomeWork_24Hrtours_Trn.csv - Home Work trips by Train

Each CSV file contains:

• Numeric matrix data (no headers)
• Rows and columns represent zones (994 x 994)
• Cell values represent trip counts between zones

Trip Purposes and Modes

Full definitions of trip purposes and mode types are available in PRISM BY Travel Demand File List.xlsx.

Common Trip Purposes:

• Com - Commute
• HBiz - Home Business
• Hesc - Home Escort
• HomeWork - Home Work Commute
• And more...

Common Mode Types:

• CP - Car Passenger
• CD - Car Driver
• Bus - Bus
• Trn - Train
• Met - Metro
• Bke - Cycle
• Wlk - Walk

Requirements

• Python 3.7 or higher
• Dependencies listed in requirements.txt

Installation

1. Clone or download this repository

2. Create a virtual environment (recommended):

   python3 -m venv venv

3. Activate the virtual environment:

   # On Linux/Mac
   source venv/bin/activate
   
   # On Windows
   venv\Scripts\activate

4. Install dependencies:

   pip install -r requirements.txt

Usage

Basic Usage

Run the script from the current directory (searches recursively):

python sum_matrices.py

Specify Source Folder

Point to a specific folder containing your CSV files:

python sum_matrices.py /path/to/your/data

Or using relative paths:

python sum_matrices.py ./my-source-data

Output

The tool creates an outputs/ folder in the current directory containing:

• TOTAL_24Hrtours_CP.csv - Sum of all Car Passenger trips
• TOTAL_24Hrtours_Bus.csv - Sum of all Bus trips
• TOTAL_24Hrtours_Trn.csv - Sum of all Train trips
• And one file for each mode type found

Example

$ python sum_matrices.py ./data

======================================================================
Trip Matrix Summation Script
======================================================================
Source folder: /home/user/project/data

Searching for matrix files...
Found: Com_24Hrtours_CP.csv -> Trip Purpose: Com, Mode: CP
Found: HBiz_24Hrtours_CP.csv -> Trip Purpose: HBiz, Mode: CP
Found: Com_24Hrtours_Bus.csv -> Trip Purpose: Com, Mode: Bus

Found 2 mode types:
  Bus: 1 files
  CP: 2 files

======================================================================
Processing matrices...
======================================================================

Processing 2 files for mode: CP
  [1/2] Reading: Com_24Hrtours_CP.csv
  [2/2] Reading: HBiz_24Hrtours_CP.csv

Saved: outputs/TOTAL_24Hrtours_CP.csv
  Matrix shape: (994, 994)
  Total trips: 254,337.25
...
======================================================================
Processing complete!
======================================================================

Features

• Recursive search: Automatically finds CSV files in all subdirectories
• Pattern matching: Only processes files matching the expected naming pattern
• Mode grouping: Automatically groups files by mode type
• Error handling: Validates matrix dimensions and handles mismatches
• Progress tracking: Shows detailed progress during processing
• Data validation: Converts text to numeric values and handles formatting issues

Troubleshooting

No files found

• Verify your CSV files follow the naming pattern: {TripPurpose}_24Hrtours_{ModeType}.csv
• Check that the source folder path is correct
• Ensure CSV files are not in ignored directories

Shape mismatch warnings

• Files with mismatched dimensions (not matching the first file's shape) will be skipped
• Check that all matrices represent the same zone structure

Dependencies issues

• Make sure you've activated your virtual environment
• Try upgrading pip: pip install --upgrade pip
• Reinstall dependencies: pip install -r requirements.txt --force-reinstall

Project Structure

.
├── sum_matrices.py                      # Main script
├── requirements.txt                      # Python dependencies
├── README.md                             # This file
├── PRISM BY Travel Demand File List.xlsx # Reference file with mode/purpose definitions
├── outputs/                              # Generated output files (created automatically)
│   ├── TOTAL_24Hrtours_CP.csv
│   ├── TOTAL_24Hrtours_Bus.csv
│   └── ...
└── [source data folders]/                # Your CSV files
    ├── Com_24Hrtours_CP.csv
    ├── HBiz_24Hrtours_CP.csv
    └── ...

License

[Add your license information here]

Contact

[Add contact information here]