This project explores the acceleration of a simple neural network model for MNIST digit classification using GPU programming via CUDA. Starting from a baseline CPU implementation (V1), we develop multiple versions with increasing levels of GPU optimization:
- V1: Sequential (CPU-based) implementation
- V2: Naive CUDA implementation
- V3: Optimized CUDA implementation using memory, communication, and kernel-level optimizations
- V4: Final version using NVIDIA Tensor Cores for peak performance
- Learn GPU programming with CUDA
- Apply memory hierarchy and thread-level parallelism
- Optimize performance using CUDA best practices
- Leverage Tensor Cores for dense matrix operations
- Input: 28x28 pixel grayscale images (784 features)
- Output: Classification into digits 0 through 9
- Architecture:
- Input Layer (784 neurons)
- One or more Hidden Layers
- Output Layer (10 neurons)
- Training: Supervised learning using stochastic gradient descent
- Loss Function: Cross Entropy or MSE
- Activation Functions: ReLU / Sigmoid
The MNIST dataset contains 70,000 labeled images:
- 60,000 for training
- 10,000 for testing
Each image is a 28x28 grayscale pixel matrix. Labels range from 0 to 9.
Ensure the dataset files are placed in the data/ folder. Use the .csv format or convert from .idx format using available scripts if needed.
.
├── src/ # Source code for all implementations
│ ├── V1/ # Sequential version (CPU)
│ ├── V2/ # Naive GPU version
│ ├── V3/ # Optimized GPU version
│ └── V4/ # Tensor Core accelerated version
│
├── data/ # Contains the MNIST dataset
│
├── report/ # Project report (PDF or LaTeX)
│
├── slides/ # Presentation slides (PDF or PPT)
│
├── Makefile # Compilation script
│
└── README.md # Project documentation and usage guide
| Name |
|---|
| Muhammad Hamza Nadeem |
| Faiz-ul-hassan |
| Faris Ali |