Convex Optimization Labs

Homework implementations and proofs from the graduate course Optimization Algorithms (NYCU, Fall 2024). Covers convex analysis fundamentals, first-order methods (gradient descent / heavy-ball / Nesterov's accelerated gradient), variance-reduced stochastic optimization (SGD / SVRG), and quadratic / conic programming with Gurobi (Markowitz portfolio optimization, Frank-Wolfe method).

Structure

hw0/ — Convex Analysis Fundamentals

• report/HW0.pdf — written proofs: first-order optimality conditions, convexity of sets/functions, Lagrangian duality and Slater's condition.
• No code: this assignment was proof-only.

hw1/ — First-Order Methods & Variance-Reduced SGD

• report/HW1.pdf — written proofs (e.g. strong convexity characterizations).
• code/P2_code/ — gradient descent, heavy-ball, and Nesterov's accelerated gradient (gd_hb_nag.py) with a convergence upper-bound check (upper_bound.py).
• code/P4_code/ — Markowitz portfolio quadratic program solved with Gurobi (QP.py).
• code/p5_code/ — SGD and SVRG optimizers implemented as custom PyTorch Optimizer subclasses (sgd.py, svrg.py), with the MNIST training/eval loop (train.py, utils.py).

hw2/ — Projected Gradient Descent & Frank-Wolfe

• report/HW2.pdf — written proofs (convergence rate of projected gradient descent) and experiment writeup.
• code/main.py — Frank-Wolfe method for a log-determinant portfolio design problem, solving the linear minimization oracle at each step with Gurobi. Requires an asset-return-matrix input (473500_wk.mat, provided by the course on E3, not included here — supply your own copy to run this).

Notes

• Assignment PDFs from the instructor are intentionally excluded — only original work (reports and code) is included here.
• A TA-provided skeleton (sample_code.zip for HW1) was excluded as it is not original work; the completed implementation in hw1/code/p5_code/ fills in the marked sections of that skeleton.
• Running hw1/code/P4_code/QP.py and hw2/code/main.py requires a Gurobi license and the gurobipy package.