Convex Optimization: Fall 2019

Machine Learning 10-725

Instructor: Ryan Tibshirani (ryantibs at cmu dot edu)

Important note: please direct emails on all course related matters to the Education Associate, not the Instructor. The subject line of all emails should begin with "[10-725]".

Education Associate: Daniel Bird (dpbird at andrew dot cmu dot edu)

Chen Dan (cdan at andrew dot cmu dot edu)
Po-Wei Wang (powei at andrew dot cmu dot edu)
Lingxiao Zhao (lingxia1 at andrew dot cmu dot edu)

Lecture times: Mondays and Wednesdays 1:30-2:50pm, Baker Hall A51

Office hours:
RT: Wednesdays 3-4pm, Baker 229B

Syllabus: here. Scribing: sign up, latex template. Discussions: Piazza group.

Go to:   Schedule | Homework | Review aids


Theory I: Fundamentals
Mon Aug 26 Introduction Slides,
Wed Aug 28 Convexity I: Sets and functions Slides,
Mon Sept 2 (Labor day, no class)
Wed Sept 4 Convexity II: Optimization basics Slides,
Mon Sept 9 Canonical problem forms Slides, Hw 1 due Fri Sept 13
Algorithms I: First-order methods
Wed Sept 11 Gradient descent
Mon Sept 16 Subgradients
Wed Sept 18 Subgradient method
Mon Sept 23 Proximal gradient descent
Wed Sept 25 Stochastic gradient descent Hw 2 due Fri Sept 27
Theory II: Optimality and duality
Mon Sept 30 Duality in linear programs
Wed Oct 2 Duality in general programs
Mon Oct 7 KKT conditions
Wed Oct 9 Duality uses and correspondences Hw 3 due Fri Oct 11
Algorithms II: Second-order methods
Mon Oct 14 Newton's method
Wed Oct 16 Barrier method
Mon Oct 21 Primal-dual interior-point methods
Wed Oct 23 Quasi-Newton methods Hw 4 due Fri Oct 25
Advanced topics
Mon Oct 28 Numerical linear algebra
Wed Oct 30 Coordinate descent
Mon Nov 4 Dykstra's algorithm
Wed Nov 6 Dual decomposition, ADMM
Mon Nov 11 Distributed ADMM Hw 5 due Fri Nov 15
Wed Nov 13 Frank-Wolfe method
Mon Nov 18 Implicit regularization
Wed Nov 20 Guest lecture I
Mon Nov 25 Guest lecture II
Wed Nov 27 (Thanksgiving break, no class)
Mon Dec 2 Guest lecture III or recap Hw 6 due Mon Dec 2
Wed Dec 4 Little test Little test




Review aids