Convex Optimization: Fall 2015

Machine Learning 10-725
(cross-listed as Statistics 36-725)

Instructor: Ryan Tibshirani (ryantibs at cmu dot edu)

Dallas Card (dcard at cmu dot edu)
Hanzhang Hu (hanzhang at cmu dot edu)
Shashank Srivastava (ssrivastava at cmu dot edu)
Matt Wytock (mwytock at cmu dot edu)

Course assistant: Mallory Deptola (mdeptola at cs dot cmu dot edu)

Lecture times: Tuesdays and Thursdays 12-1:20pm, Doherty Hall A302

Office hours:
RT: Tuesdays 1:30-2:30pm, Baker Hall 229B
DC: Fridays 3-4pm, Gates Bldg 8009
HH: Mondays 5-6pm, Gates Bldg 8005
SS: Wednesdays 12-1pm, Gates Bldg 8005
MW: Tuesdays 3-4pm, Gates Bldg 8015

Syllabus: here

Scribing: Sign up for scribing here, and find the Latex scribing template here

Discussions: Class discussions will be run through Piazza, linked here

Go to:   Schedule | Assignments | Miscellaneous


Here is the estimated class schedule. It is subject to change, depending on time and class interests.

Theory I: Fundamentals
Tues Sept 1 Introduction Slides Quiz
Thur Sept 3 Convexity I: Sets and functions
Tues Sept 8 Convexity II: Optimization basics
Thur Sept 10 Canonical problem forms
Algorithms I: First-order methods
Tues Sept 15 Gradient descent Hw 1 due (Weds)
Thur Sept 17 Subgradients
Tues Sept 22 Subgradient method
Thur Sept 24 Proximal gradient descent, acceleration
Tues Sept 29 Numerical linear algebra primer Project milestone 1 due
Theory II: Optimality and duality
Thur Oct 1 Duality in linear programs Hw 2 due (Fri)
Tues Oct 6 Duality in general programs
Thur Oct 8 KKT conditions
Tues Oct 13 Duality uses and correspondences
Algorithms II: Second-order methods
Thur Oct 15 Newton's method
Tues Oct 20 Barrier method Hw 3 due (Weds)
Thur Oct 22 Primal-dual interior point methods
Tues Oct 27 Proximal Newton method
Thur Oct 29 Review
Tues Nov 3 Midterm Midterm
Thur Nov 5 Case study: ???
Tues Nov 10 Case study: ??? (continued) Project milestone 2 due, and Hw 4 due (Weds)
Special topics
Thur Nov 12 Dual methods and ADMM
Tues Nov 17 Coordinate descent
Thur Nov 19 Conditional gradient method
Tues Nov 24 Projected Newton method
Thur Nov 26 (Thanksgiving break, no class)
Tues Dec 1 Fast stochastic methods Hw 5 due (Weds)
Thur Dec 3 Nonconvex? Exact path algorithms? Alternating min?
Tues Dec 8 Little test Little test
Thur Dec 10 (Work on projects, no class)



All assignments are to be turned into Mallory Deptola, in 8001 GHC, unless specified otherwise. Top


Some review aids: Top