NN in C++

A lightweight, educational deep learning library built from scratch to understand neural network internals.

Goal

Create an API capable of building, training, and running deep learning models. As a milestone, it is intended to train a CNN on MNIST dataset.

Features

• Tensor Operations: Multi-dimensional arrays with mathematical operations
• Layers: Dense and Activation (ReLU, Sigmoid, Tanh, Softmax)
• Models: Sequential architecture for stacking layers
• Loss Functions: Mean Squared Error (MSE)
• Optimizers: Stochastic Gradient Descent (SGD)
• Training: Complete forward/backward propagation with automatic gradient computation

Project Structure

nn-in-cpp
├── tensor/     # Core tensor implementation
├── layers/     # Dense, Activation layers
├── model/      # Sequential model
├── loss/       # MSE loss
├── optimizer/  # SGD optimizer
├── examples/   # Examples (XOR proble, MNIST dataset in future)
└── tests/      # Unit tests

Current Status

• Tensor operations
• Dense & Activation layers
• Sequential model
• MSE loss & SGD optimizer
• Training pipeline
• Unit tests
• Convolutional layers
• MNIST support in development
• Model serialization in development

Key Implementation

Tensor Layout

Uses row-major order (C-style) for cache efficiency:

std::vector<size_t> shape;    // Dimensions
std::vector<double> data;      // Flattened data

Access element [i,j,k] via: flat_index = i*stride0 + j*stride1 + k*stride2

Backpropagation

Reverse-mode automatic differentiation using the chain rule:

gradInput = backward(gradOutput)  // ∂L/∂input = (∂output/∂input)ᵀ × ∂L/∂output

Each layer caches forward-pass values for efficient gradient computation.

Building

cd tests
make
make all

Requirements: C++17, Make

License

MIT - For educational use