# Convex Optimization: Fall 2015

## Machine Learning 10-725 (cross-listed as Statistics 36-725)

Instructor: Ryan Tibshirani (ryantibs at cmu dot edu)

TAs:
Dallas Card (dcard at cmu dot edu)
Hanzhang Hu (hanzhang at cmu dot edu)
Shashank Srivastava (ssrivastava at cmu dot edu)
Matt Wytock (mwytock at cs dot cmu dot edu)

Course assistant: Mallory Deptola (mdeptola at cs dot cmu dot edu)

Lecture times: Tuesdays and Thursdays 12-1:20pm, Doherty Hall A302

Office hours:
RT: Tuesdays 1:30-2:30pm, Baker Hall 229B
DC: Fridays 3-4pm, Gates Bldg 8009
HH: Mondays 5-6pm, Gates Bldg 8005
SS: Wednesdays 12-1pm, Gates Bldg 8005
MW: Tuesdays 3-4pm, Gates Bldg 8015

Syllabus: here

Scribing: Sign up for scribing here, and find the Latex scribing template here

Discussions: Class discussions will be run through Piazza, linked here

Go to:   Schedule | Assignments | Miscellaneous

## Schedule

Here is the estimated class schedule. It is subject to change, depending on time and class interests.

 Theory I: Fundamentals Tues Sept 1 Introduction Slides, notes, video Quiz Thur Sept 3 Convexity I: Sets and functions Slides, notes, video (1/2), (2/2) Quiz Scribe Tues Sept 8 Convexity II: Optimization basics Slides, notes, video (1/2), (2/2) Quiz Scribe Thur Sept 10 Canonical problem forms Slides, notes, video (1/2), (2/2) Quiz Scribe Algorithms I: First-order methods Tues Sept 15 Gradient descent Slides, notes, video (1/2), (2/2) Quiz Scribe Hw 1 due (Weds) Thur Sept 17 Subgradients Slides, notes, video (1/2), (2/2) Quiz Scribe Tues Sept 22 Subgradient method Slides, notes, video (1/2), (2/2) Quiz Thur Sept 24 Proximal gradient descent, acceleration Slides, notes, video (1/2), (2/2) Quiz Scribe Tues Sept 29 Numerical linear algebra primer Slides, notes, video (1/2), (2/2) Quiz Project milestone 1 due Theory II: Optimality and duality Thur Oct 1 Duality in linear programs Slides, notes, video (1/2), (2/2) Quiz Hw 2 due (Fri) Tues Oct 6 Duality in general programs Slides, notes, video (1/2), (2/2) Quiz Scribe Thur Oct 8 KKT conditions Slides, notes, video (1/2), (2/2) Quiz Scribe Tues Oct 13 Duality uses and correspondences Slides, notes, video (1/2), (2/2) Quiz Algorithms II: Second-order methods Thur Oct 15 Newton's method Slides, notes, video (part 2 only) Quiz Scribe Tues Oct 20 Barrier method Slides, notes, video (1/2), (2/2) Quiz Scribe Hw 3 due (Weds) Thur Oct 22 Primal-dual interior point methods Slides, notes, video (1/2), (2/2) Quiz Tues Oct 27 Proximal Newton method Slides, notes, video (1/2), (2/2) Quiz Scribe Midterm Thur Oct 29 Review Algorithms table, notes, video Tues Nov 3 Midterm Midterm Applications Thur Nov 5 Case studies: sum of norms regularization Slides, notes, video (1/2), (2/2) Quiz Scribe Tues Nov 10 Case studies: sum of norms regularization (continued) Slides, notes, video (1/2), (2/2) Quiz Project milestone 2 due Special topics Thur Nov 12 Dual methods and ADMM Slides, notes, video (1/2), (2/2) Quiz Hw 4 due (Fri) Tues Nov 17 Dual methods and ADMM (continued) Slides, notes, video (1/2), (2/2) Quiz Thur Nov 19 Coordinate descent Slides, notes, video (1/2), (2/2) Quiz Tues Nov 24 Conditional gradient method Slides, notes, video (1/2), (2/2) Quiz Thur Nov 26 (Thanksgiving break, no class) Tues Dec 1 Fast stochastic methods Slides, notes, video (1/2), (2/2) Quiz Thur Dec 3 Nonconvex? No problem! Slides, notes, video Hw 5 due (Fri) Tues Dec 8 Little test Little test Thur Dec 10 (Work on projects, no class) Wed Dec 16 COPTS conference, 3-6pm COPTS conference

Top

## Assignments

All assignments are to be turned into Mallory Deptola, in 8001 GHC, unless specified otherwise.
Top

## Miscellaneous

Some review aids:
Top