Machine Learning From Scratch – Theory to PyTorch

This repository contains a set of machine learning implementations, ranging from optimization algorithms and linear models to neural networks, with an emphasis on understanding the underlying mathematics and implementing models from first principles.

The goal is conceptual clarity, numerical stability, and explicit control over the learning dynamics.

---

Contents

Optimization Fundamentals

• Gradient Descent
  • Derivation of the update rule
  • Effect of learning rate on convergence and stability
  • Visualization of loss landscapes

Linear Models

• Logistic Regression (Binary and Multiclass)
  • Maximum likelihood formulation
  • Cross-entropy loss
  • Softmax and decision boundaries
  • Explicit gradient computation

Regression

• California Housing Regression
  • Feature normalization
  • Bias–variance considerations
  • Comparison of linear vs non-linear fits

Neural Networks (PyTorch)

• Multilayer Perceptron

  • Manual architecture definition
  • Backpropagation via autograd
  • Effect of depth and non-linearities

• Autoencoder for Fraud Detection

  • Unsupervised anomaly detection
  • Reconstruction error as anomaly score
  • Latent space interpretation

---

Prerequisites

• Linear algebra
• Probability theory
• Optimization
• Basic familiarity with PyTorch