Convex Optimization: Fall 2015

Machine Learning 10-725
(cross-listed as Statistics 36-725)

Instructor: Ryan Tibshirani (ryantibs at cmu dot edu)

TAs:
Dallas Card (dcard at cmu dot edu)
Hanzhang Hu (hanzhang at cmu dot edu)
Shashank Srivastava (ssrivastava at cmu dot edu)
Matt Wytock (mwytock at cs dot cmu dot edu)

Course assistant: Mallory Deptola (mdeptola at cs dot cmu dot edu)

Lecture times: Tuesdays and Thursdays 12-1:20pm, Doherty Hall A302

Office hours:
RT: Tuesdays 1:30-2:30pm, Baker Hall 229B
DC: Fridays 3-4pm, Gates Bldg 8009
HH: Mondays 5-6pm, Gates Bldg 8005
SS: Wednesdays 12-1pm, Gates Bldg 8005
MW: Tuesdays 3-4pm, Gates Bldg 8015

Syllabus: here

Scribing: Sign up for scribing here, and find the Latex scribing template here

Discussions: Class discussions will be run through Piazza, linked here

Go to:   Schedule | Assignments | Miscellaneous

Schedule

Here is the estimated class schedule. It is subject to change, depending on time and class interests.

Theory I: Fundamentals
Tues Sept 1 Introduction Slides, notes, video Quiz
Thur Sept 3 Convexity I: Sets and functions Slides, notes, video (1/2), (2/2) Quiz Scribe
Tues Sept 8 Convexity II: Optimization basics Slides, notes, video (1/2), (2/2) Quiz Scribe
Thur Sept 10 Canonical problem forms Slides, notes, video (1/2), (2/2) Quiz Scribe
Algorithms I: First-order methods
Tues Sept 15 Gradient descent Slides, notes, video (1/2), (2/2) Quiz Scribe Hw 1 due (Weds)
Thur Sept 17 Subgradients Slides, notes, video (1/2), (2/2) Quiz Scribe
Tues Sept 22 Subgradient method Slides, notes, video (1/2), (2/2) Quiz
Thur Sept 24 Proximal gradient descent, acceleration Slides, notes, video (1/2), (2/2) Quiz Scribe
Tues Sept 29 Numerical linear algebra primer Slides, notes, video (1/2), (2/2) Quiz Project milestone 1 due
Theory II: Optimality and duality
Thur Oct 1 Duality in linear programs Slides, notes, video (1/2), (2/2) Quiz Hw 2 due (Fri)
Tues Oct 6 Duality in general programs Slides, notes, video (1/2), (2/2) Quiz Scribe
Thur Oct 8 KKT conditions Slides, notes, video (1/2), (2/2) Quiz Scribe
Tues Oct 13 Duality uses and correspondences Slides, notes, video (1/2), (2/2) Quiz
Algorithms II: Second-order methods
Thur Oct 15 Newton's method Slides, notes, video (part 2 only) Quiz Scribe
Tues Oct 20 Barrier method Slides, notes, video (1/2), (2/2) Quiz Scribe Hw 3 due (Weds)
Thur Oct 22 Primal-dual interior point methods Slides, notes, video (1/2), (2/2) Quiz
Tues Oct 27 Proximal Newton method Slides, notes, video (1/2), (2/2) Quiz Scribe
Midterm
Thur Oct 29 Review Algorithms table, notes, video
Tues Nov 3 Midterm Midterm
Applications
Thur Nov 5 Case studies: sum of norms regularization Slides, notes, video (1/2), (2/2) Quiz Scribe
Tues Nov 10 Case studies: sum of norms regularization (continued) Slides, notes, video (1/2), (2/2) Quiz Project milestone 2 due
Special topics
Thur Nov 12 Dual methods and ADMM Slides, notes, video (1/2), (2/2) Quiz Hw 4 due (Fri)
Tues Nov 17 Dual methods and ADMM (continued) Slides, notes, video (1/2), (2/2) Quiz
Thur Nov 19 Coordinate descent Slides, notes, video (1/2), (2/2) Quiz
Tues Nov 24 Conditional gradient method Slides, notes, video (1/2), (2/2) Quiz
Thur Nov 26 (Thanksgiving break, no class)
Tues Dec 1 Fast stochastic methods Slides, notes, video (1/2), (2/2) Quiz
Thur Dec 3 Nonconvex? No problem! Slides, notes, video Hw 5 due (Fri)
Tues Dec 8 Little test Little test
Thur Dec 10 (Work on projects, no class)
Wed Dec 16 COPTS conference, 3-6pm COPTS conference

Top

Assignments

All assignments are to be turned into Mallory Deptola, in 8001 GHC, unless specified otherwise. Top

Miscellaneous

Some review aids: Top