Convex Optimization: Fall 2019

Machine Learning 10-725

Instructor: Ryan Tibshirani (ryantibs at cmu dot edu)

Important note: please direct emails on all course related matters to the Education Associate, not the Instructor. The subject line of all emails should begin with "[10-725]".

Education Associate: Daniel Bird (dpbird at andrew dot cmu dot edu)

Chen Dan (cdan at andrew dot cmu dot edu)
William Guss (whg at andrew dot cmu dot edu)
Aaron Rumack (arumack at andrew dot cmu dot edu)
Po-Wei Wang (poweiw at andrew dot cmu dot edu)
Lingxiao Zhao (lingxia1 at andrew dot cmu dot edu)

Lecture times: Mondays and Wednesdays 1:30-2:50pm, Baker Hall A51

Office hours:
RT: Wednesdays 3:00pm-4:00pm, Baker 229B
CD: Tuesdays 2:00pm-3:00pm
WG: Wednesdays 12:15pm-1:15pm
AR: Thursdays 10:00am-11:00am
PW: Mondays 3:00pm-4:00pm

Syllabus: here. Scribing: sign up, latex template. Discussions: Piazza group.

Go to:   Schedule | Homework | Review aids


Theory I: Fundamentals
Mon Aug 26 Introduction Slides
Wed Aug 28 Convexity I: Sets and functions Slides (Scribed notes)
Mon Sept 2 (Labor day, no class)
Wed Sept 4 Convexity II: Optimization basics Slides (Scribed notes)
Mon Sept 9 Canonical problem forms Slides (Scribed notes) Hw 1 due Fri Sept 13
Algorithms I: First-order methods
Wed Sept 11 Gradient descent Slides (Scribed notes)
Mon Sept 16 Subgradients Slides (Scribed notes)
Wed Sept 18 Subgradient method Slides (Scribed notes)
Mon Sept 23 Proximal gradient descent Slides (Scribed notes)
Wed Sept 25 Stochastic gradient descent Slides (Scribed notes) Hw 2 due Fri Sept 27
Theory II: Duality and optimality
Mon Sept 30 Duality in linear programs Slides (Scribed notes)
Wed Oct 2 Duality in general programs Slides (Scribed notes)
Mon Oct 7 KKT conditions Slides (Scribed notes)
Wed Oct 9 Duality uses and correspondences Slides (Scribed notes) Hw 3 due Fri Oct 11
Algorithms II: Second-order methods
Mon Oct 14 Newton's method Slides
Wed Oct 16 Barrier method Slides
Mon Oct 21 Primal-dual interior-point methods Slides
Wed Oct 23 Quasi-Newton methods Slides Hw 4 due Fri Oct 25
Advanced topics
Mon Oct 28 Numerical linear algebra
Wed Oct 30 Coordinate descent
Mon Nov 4 Dykstra's algorithm
Wed Nov 6 Dual decomposition, ADMM
Mon Nov 11 Distributed ADMM Hw 5 due Fri Nov 15
Wed Nov 13 Frank-Wolfe method or implicit regularization?
Mon Nov 18 Guest lecture I
Wed Nov 20 Guest lecture II
Mon Nov 25 (Thanksgiving break, no class)
Wed Nov 27 (Thanksgiving break, no class)
Mon Dec 2 Recap Hw 6 due Mon Dec 2
Wed Dec 4 Little test Little test




Review aids