A curated collection of resources to help you understand numerical optimization methods and their applications in machine learning and AI.
- Linear Algebra
- Calculus and Multivariate Calculus
- Probability and Statistics
- Programming (Python/R)
- Convex Sets and Functions
- Gradient-Based Methods
- Constrained vs Unconstrained Optimization
- Convergence Theory
- Linear Programming
- Nonlinear Programming
- Global Optimization
- Stochastic Optimization
-
Mathematics for Optimization
-
Programming Foundations
-
Core Optimization
- Convex Optimization (Stanford)
- Book: "Numerical Optimization" by Nocedal & Wright
-
Practical Implementations
- Specialized Topics
- Book: "Optimization Methods for Large-Scale Machine Learning"
- Advanced Optimization Methods
-
Introductory
- "Introduction to Linear Optimization" by Bertsimas & Tsitsiklis
- "Convex Optimization" by Boyd & Vandenberghe
-
Advanced
- "Numerical Optimization" by Nocedal & Wright
- "Optimization Methods in Finance" by Cornuejols & Tütüncü
-
Code Implementations
- This repository's examples
- SciPy Optimization Examples
-
Problem Sets
Remember: The best way to learn optimization is through practice and implementation. Start with simple problems and gradually work your way up to more complex ones.