update week 1

Author: mhjensen

doc/pub/week1/html/week1-bs.html

@@ -0,0 +1,161 @@
+\documentclass[11pt,a4paper]{article}
+
+\usepackage{amsmath,amssymb,amsfonts}
+\usepackage{geometry}
+\usepackage{hyperref}
+\usepackage{physics}
+\usepackage{graphicx}
+
+\geometry{margin=1in}
+
+\title{\textbf{Discriminative and Generative Deep Learning Models:\\
+A Mathematical and Computational Study}}
+\author{}
+\date{}
+
+\begin{document}
+\maketitle
+
+\section*{Project Overview}
+
+Deep learning methods can broadly be divided into \emph{discriminative
+models}, which learn decision boundaries for labeled data, and
+\emph{generative models}, which learn probability distributions over
+data. Convolutional neural networks (CNNs) dominate modern
+classification tasks, while generative models such as variational
+autoencoders (VAEs), Boltzmann machines, and diffusion models provide
+probabilistic descriptions of data and enable synthesis, uncertainty
+quantification, and representation learning.
+
+The goal of this project is to develop a unified mathematical and
+computational understanding of these model classes. Students will
+analyze classification and generative learning as optimization
+problems over high-dimensional function spaces, emphasizing
+probabilistic modeling, variational principles, and numerical
+optimization.
+
+
+
+\section{Classification with Convolutional Neural Networks}
+
+A convolutional neural network defines a parametric mapping
+\begin{equation}
+f_\theta : \mathbb{R}^{H \times W \times C} \rightarrow \{1,\dots,K\},
+\end{equation}
+where inputs are structured data (e.g.\ images) and outputs are class labels.
+
+Mathematically, CNNs combine:
+\begin{itemize}
+  \item Convolutional linear operators with local receptive fields,
+  \item Nonlinear activation functions,
+  \item Pooling and subsampling operations.
+\end{itemize}
+
+Students will analyze:
+\begin{itemize}
+  \item Convolutions as structured sparse linear maps,
+  \item Translation equivariance and symmetry reduction,
+  \item Parameter sharing and its effect on sample complexity.
+\end{itemize}
+
+The classification problem is formulated as empirical risk minimization with cross-entropy loss,
+\begin{equation}
+\mathcal{L}_{\text{clf}}(\theta) = -\frac{1}{N}\sum_{i=1}^N \log p_\theta(y_i \mid x_i),
+\end{equation}
+and optimized using stochastic gradient descent.
+
+\section{Probabilistic Generative Modeling}
+
+Generative models aim to learn an approximation $p_\theta(x)$ to an unknown data distribution. This project considers three complementary paradigms.
+
+\subsection{Variational Autoencoders}
+
+VAEs introduce latent variables $z$ and define
+\begin{equation}
+p_\theta(x,z) = p_\theta(x \mid z)p(z),
+\end{equation}
+with training based on variational inference.
+
+Students will derive the evidence lower bound (ELBO),
+\begin{equation}
+\mathcal{L}_{\text{VAE}} = \mathbb{E}_{q_\phi(z\mid x)}[\log p_\theta(x\mid z)] - \mathrm{KL}(q_\phi(z\mid x)\|p(z)),
+\end{equation}
+and analyze:
+\begin{itemize}
+  \item Encoder--decoder architectures,
+  \item Reparameterization trick and differentiability,
+  \item Trade-offs between reconstruction accuracy and latent regularization.
+\end{itemize}
+
+\subsection{Boltzmann Machines}
+
+Boltzmann machines define energy-based models
+\begin{equation}
+p_\theta(x) = \frac{1}{Z_\theta} e^{-E_\theta(x)},
+\end{equation}
+where $Z_\theta$ is the partition function.
+
+The project examines:
+\begin{itemize}
+  \item Energy landscapes and statistical mechanics analogies,
+  \item Maximum likelihood learning and gradient structure,
+  \item Approximate inference methods such as contrastive divergence.
+\end{itemize}
+
+Connections between Boltzmann machines and variational principles are emphasized.
+
+\subsection{Diffusion Models}
+
+Diffusion models define a forward noising process and a learned reverse-time denoising process. Training minimizes a denoising objective equivalent to variational inference.
+
+Students will analyze:
+\begin{itemize}
+  \item Stochastic differential equations and discretization,
+  \item Score matching and reverse-time dynamics,
+  \item Relations to Langevin sampling and thermodynamics.
+\end{itemize}
+
+Diffusion models are interpreted as iterative numerical solvers for sampling from complex distributions.
+
+\section{Discriminative vs Generative Learning}
+
+A central theme of the project is the comparison between discriminative and generative objectives.
+
+Topics include:
+\begin{itemize}
+  \item Decision boundaries vs density estimation,
+  \item Sample efficiency and representation learning,
+  \item Uncertainty quantification and out-of-distribution detection.
+\end{itemize}
+
+Students will explore how latent representations learned by generative models can support downstream classification tasks.
+
+\section{Implementation and Experiments}
+
+The practical component consists of implementing:
+\begin{itemize}
+  \item A CNN for image classification,
+  \item At least one generative model (VAE, Boltzmann machine, or diffusion model).
+\end{itemize}
+
+Experiments will use standard labeled datasets and focus on:
+\begin{itemize}
+  \item Classification accuracy and confusion structure,
+  \item Quality of generated samples,
+  \item Latent-space geometry and interpolation.
+\end{itemize}
+
+Computational results are interpreted through the lens of the mathematical models.
+
+\section*{Expected Outcomes}
+
+By completing this project, students will:
+\begin{itemize}
+  \item Understand deep learning through probabilistic and variational principles,
+  \item Connect classification and generation within a unified framework,
+  \item Analyze neural networks as numerical optimization problems,
+  \item Gain insight applicable to physics-inspired machine learning, statistical inference, and scientific data analysis.
+\end{itemize}
+
+\end{document}
+

doc/pub/week1/html/week1-reveal.html

@@ -0,0 +1,163 @@
+\documentclass[11pt,a4paper]{article}
+
+\usepackage{amsmath,amssymb,amsfonts}
+\usepackage{geometry}
+\usepackage{hyperref}
+\usepackage{physics}
+\usepackage{graphicx}
+
+\geometry{margin=1in}
+
+\title{\textbf{Reinforcement Learning, Generative Models, and PDEs:\\
+A Mathematical Project in Control and Inference}}
+\author{}
+\date{}
+
+\begin{document}
+\maketitle
+
+\section*{Project Overview}
+
+Reinforcement learning (RL) and modern generative models are
+increasingly understood through the lens of partial differential
+equations (PDEs), stochastic processes, and variational
+principles. Reinforcement learning is closely related to optimal
+control and Hamilton--Jacobi--Bellman (HJB) equations, while
+generative models such as diffusion models and score-based methods are
+connected to Fokker--Planck equations, stochastic differential
+equations (SDEs), and gradient flows in probability space.
+
+The goal of this project is to develop a unified mathematical
+understanding of reinforcement learning and generative learning as
+PDE-driven optimization problems. Students will analyze value
+functions, policies, and probability densities as solutions to PDEs,
+and compare how control and inference emerge from related mathematical
+structures.
+
+
+
+\section{Reinforcement Learning and Optimal Control}
+
+Reinforcement learning problems are commonly formulated as Markov decision processes, but in the continuous-state and continuous-time limit they are naturally described by stochastic control theory.
+
+Consider a controlled stochastic differential equation
+\begin{equation}
+dX_t = f(X_t,u_t)\,dt + \sigma(X_t)\,dW_t,
+\end{equation}
+where $u_t$ is a control policy. The objective is to minimize the expected cost functional
+\begin{equation}
+J(u) = \mathbb{E}\left[ \int_0^T \ell(X_t,u_t)\,dt + g(X_T) \right].
+\end{equation}
+
+The associated value function
+\begin{equation}
+V(x,t) = \inf_u \mathbb{E}_{x,t} \left[ \int_t^T \ell(X_s,u_s)\,ds + g(X_T) \right]
+\end{equation}
+satisfies the Hamilton--Jacobi--Bellman (HJB) equation
+\begin{equation}
+\partial_t V + \min_u \left\{ \ell(x,u) + \nabla V \cdot f(x,u) \right\}
++ \frac{1}{2}\mathrm{Tr}\!\left(\sigma\sigma^T \nabla^2 V\right) = 0.
+\end{equation}
+
+\subsection*{Derivation Task 1}
+Derive the HJB equation from the dynamic programming principle for the continuous-time control problem.
+
+\section{Deep Reinforcement Learning as PDE Approximation}
+
+In practical reinforcement learning, the value function $V(x)$ or action-value function $Q(x,u)$ is approximated by a neural network $V_\theta(x)$. Learning corresponds to minimizing a residual of the Bellman equation,
+\begin{equation}
+\mathcal{L}(\theta) = \mathbb{E}\left[ \left( \mathcal{T}V_\theta - V_\theta \right)^2 \right],
+\end{equation}
+where $\mathcal{T}$ denotes the Bellman operator.
+
+From a PDE perspective:
+\begin{itemize}
+  \item Neural networks act as nonlinear trial spaces,
+  \item Training corresponds to a Galerkin or collocation method,
+  \item Instabilities arise from nonlinearity and bootstrapping.
+\end{itemize}
+
+\subsection*{Derivation Task 2}
+Show that the Bellman operator is a contraction in the discounted case and explain why this property is generally lost under nonlinear function approximation.
+
+\section{Generative Models and Forward--Backward PDEs}
+
+Generative models aim to learn a probability density $\rho(x)$ rather than an optimal control. Many modern generative models are governed by diffusion processes
+\begin{equation}
+dX_t = b(X_t,t)\,dt + \sqrt{2\beta^{-1}}\,dW_t,
+\end{equation}
+whose probability density evolves according to the Fokker--Planck equation
+\begin{equation}
+\partial_t \rho = -\nabla \cdot (b\rho) + \beta^{-1}\Delta \rho.
+\end{equation}
+
+Diffusion models learn the \emph{reverse-time dynamics}, which can be written as
+\begin{equation}
+dX_t = \left[ b(X_t,t) - 2\beta^{-1}\nabla \log \rho_t(X_t) \right]dt + \sqrt{2\beta^{-1}}\,dW_t.
+\end{equation}
+
+\subsection*{Derivation Task 3}
+Derive the reverse-time SDE associated with the Fokker--Planck equation and explain its connection to score matching.
+
+\section{Variational and Entropic Perspectives}
+
+Both reinforcement learning and generative modeling admit variational formulations.
+
+In entropy-regularized RL, the objective becomes
+\begin{equation}
+J(\pi) = \mathbb{E}_\pi \left[ \sum_t r_t - \alpha \sum_t \log \pi(a_t|s_t) \right],
+\end{equation}
+leading to a modified HJB equation with a log-sum-exp structure.
+
+Similarly, diffusion and score-based models can be interpreted as minimizing free-energy or Kullback--Leibler functionals over probability paths.
+
+\subsection*{Derivation Task 4}
+Show that entropy-regularized reinforcement learning leads to a soft HJB equation and compare it to the variational objective of diffusion models.
+
+\section{Control vs Inference: A PDE Comparison}
+
+A central comparison explored in this project is:
+\begin{center}
+\begin{tabular}{l l}
+\textbf{Reinforcement Learning} & \textbf{Generative Learning} \\
+\hline
+Optimal control & Probabilistic inference \\
+HJB equation & Fokker--Planck equation \\
+Backward PDE & Forward--backward PDE \\
+Policy optimization & Density evolution \\
+\end{tabular}
+\end{center}
+
+Students will analyze how:
+\begin{itemize}
+  \item Policies correspond to optimal drift fields,
+  \item Value functions resemble logarithmic transforms of densities,
+  \item Control and sampling differ mathematically but share PDE structure.
+\end{itemize}
+
+\subsection*{Derivation Task 5}
+Demonstrate the formal correspondence between a logarithmic transformation of the value function and a density-based formulation.
+
+\section{Computational Experiments}
+
+The computational component consists of:
+\begin{itemize}
+  \item Solving a low-dimensional HJB equation numerically,
+  \item Implementing a reinforcement learning agent approximating the same solution,
+  \item Training a diffusion or score-based model on a related stochastic system.
+\end{itemize}
+
+Results are compared in terms of convergence, stability, and approximation quality.
+
+\section*{Expected Outcomes}
+
+By completing this project, students will:
+\begin{itemize}
+  \item Understand reinforcement learning and generative models as PDE problems,
+  \item Connect stochastic control, inference, and variational principles,
+  \item Analyze neural networks as numerical solvers,
+  \item Gain tools relevant to scientific machine learning, control, and physics-informed AI.
+\end{itemize}
+
+\end{document}
+