Creative Commons License
Introduction to Stochastic Optimization by Raghu Pasupathy is licensed under a Creative Commons Attribution 4.0 International License.
raghu pasupathy

Stochastic Optimization (STAT 598)
lecture number 1 lecture notes stochastic optimization introduction
lecture number 2 lecture video, lecture notes preliminaries
lecture number 3 lecture video, lecture notes fundamental inequality for smooth functions, gradient method convergence
lecture number 4 lecture video, lecture notes gradient method complexity
lecture number 5 lecture video, lecture notes local convergence rate of the gradient method
lecture number 6 lecture video, lecture notes quasi-Newton method
lecture number 7 lecture video, lecture notes convex function preliminaries
lecture number 8 lecture video, lecture notes convex function preliminaries, complexity lower bound for sm. conv. with first-order method
lecture number 9 lecture video, lecture notes gradient method complexity for sm. convex functions
lecture number 10 lecture video, lecture notes Nesterov's acceleration
lecture number 11 lecture video, lecture notes Nesterov's acceleration
lecture number 12 lecture video, lecture notes nonsmooth convex opt., preliminaries
lecture number 13 lecture video, lecture notes subgradient, sub-differential preliminaries
lecture number 14 lecture video, lecture notes subgradient method
lecture number 15 lecture video, lecture notes subgradient method, level method
lecture number 16 lecture video, lecture notes level method convergence
lecture number 17 lecture video, lecture notes stochastic optimization introduction
lecture number 18 lecture video, lecture notes fixed-step stochastic gradient descent
lecture number 19 lecture video, lecture notes Robbins-Siegmund Theorem, diminishing step stochastic gradient descent
lecture number 20 lecture video, lecture notes diminishing step stochastic gradient descent
lecture number 21 lecture video, lecture notes Offline Iterate Averaging (Polyak and Juditsky's Method)
lecture number 22 lecture video, lecture notes continuous time approximation
lecture number 23 lecture video, lecture notes sample average approximation