(cross-listed as Statistics 36-725)

Barnabas Poczos (bapoczos at cs dot cmu dot edu) Ryan Tibshirani (ryantibs at stat dot cmu dot edu) Adona Iosif (aiosif at cs dot cmu dot edu) Yifei Ma (yifeim at cs dot cmu dot edu) Aaditya Ramdas (aramdas at cs dot cmu dot edu) Sashank Reddi (sjakkamr at cs dot cmu dot edu) BP: Thursdays 4:30-5:30pm, Gates 8231 RT: Wednesdays 10-11am, Baker 229B AI: Wedsnesdays 2-3pm, Gates 8221 YM: Mondays 4:30-5:30pm, Gates 8223 AR: Tuesdays 4:30-5:30pm, Gates 8223 SR: Mondays 2-3pm, Gates 8223 |

Go to: Lectures | Recitations | Videos | Assignments | Schedule

- Introduction (Ryan)

- Linear programming I (Barnabas)

- Linear programming II (Barnabas)

- Convexity (Barnabas)

- Gradient descent (Ryan)

- Subgradients (Ryan)

- Subgradient method (Ryan)

- Generalized gradient descent and acceleration (Ryan)
- Slides, more slides (didn't cover these)
- Handwritten notes, cont'd
- Scribed notes

- Newton method (Barnabas)

- Conjugate gradient (Barnabas)

- Quasi-Newton (Barnabas)

- Duality (Ryan)

- KKT conditions (Ryan)

- Duality uses and correspondences (Ryan)

- Ellipsoid methods (Barnabas)

- Penalty methods (Barnabas)

- Barrier methods (Barnabas)

- Semidefinite programming I (Barnabas)

- Semidefinite programming II (Barnabas)
- Slides (same as lecture 18)

- Independent components analysis (Barnabas)

- Putting it all together (Ryan)

- Simulated Annealing (Barnabas)

- Dual methods & ADMM (Ryan)

- Coordinate descent (Ryan)

- Nonconvex? No problem! (Ryan)

- Math review, alternate view of simplex (Aaditya)

- Convexity, strong convexity, Lipschitz gradients, etc. (Aaditya)

- Duality and the KKT conditions (Adona)

Top

- Homework 1, due Sept 19

Zipped tex files: hw1.zip

Demo Matlab code: demo.m

Solutions

- Project milestone 1: Proposal, due Sept 26

2 page write up in NIPS format

More details

- Homework 2, due Oct 3

Zipped tex files: hw2.zip

Data files: movie_data.zip, mona_bw.jpg

Solutions

- Homework 3, due Oct 17

Zipped tex files: hw3.zip

Data and code files: face.txt, face_test.txt, compact_canvas.m, loadfaceimages.m

Solutions

- Homework 4, due Oct 31

Zipped tex files: hw4.zip

Data and code files: binseq.txt, prox.R, prox_R.cpp, prox.m, prox_matlab.cpp

Solutions, and solution code: bmfuns.R, hw4Q3sol.R, ggfuns.R, hw4Q5sol.R

- Project milestone 2: Progress report, due Nov 12

5-6 page write up in NIPS format

More details

- Homework 5, due Nov 27

Zipped tex files: hw5.zip

Zipped data files: Data

Solutions, and solution code: FastICAdemo.zip, David Fouhey's Q5 solution

- Project poster, due Dec 13

A poster no larger than 32"x40", using this template if you desire

For students in SCS, you may make use of the poster printing facilities

More details

- Project final report, due Dec 13

7-8 page write up in NIPS format

More details

Tues Aug 27 | 1. Introduction (R) | |

Thurs Aug 29 | 2. Linear programs I (B) | |

Tues Sept 3 | 3. Linear programs II (B) | |

Thurs Sept 5 | 4. Convexity (B) | Hw 1 out |

Tues Sept 10 | 5. Gradient descent (R) | |

Thurs Sept 12 | 6. Subgradients (R) | |

Tues Sept 17 | 7. Subgradient method (R) | |

Thurs Sept 19 | 8. Generalized gradient and acceleration (R) | Hw 1 in; Hw 2 out |

Tues Sept 24 | 9. Newton's method (B) | |

Thurs Sept 26 | 10. Conjugate gradient (B) | Project milestone 1 in |

Tues Oct 1 | 11. Quasi-Newton (B) | |

Thurs Oct 3 | 12. Duality (R) | Hw 2 in; Hw 3 out |

Tues Oct 8 | 13. KKT conditions (R) | |

Thurs Oct 10 | 14. Duality uses & correspondences (R) | |

Tues Oct 15 | 15. Ellipsoid methods (B) | |

Thurs Oct 17 | 16. Penalty methods (B) | Hw 3 in; Hw 4 out |

Tues Oct 22 | 17. Barrier methods (B) | |

Thurs Oct 24 | 18. Semidefinite programs I (B) | |

Tues Oct 29 | 19. Semidefinite programs II (B) | |

Thurs Oct 31 | 20. Semidefinite programs III (B) | Hw 4 in |

Tues Nov 5 | Midterm (covers up to lec 18) | Midterm |

Thurs Nov 7 | 21. Putting it all together (R) | |

Tues Nov 12 | 22. Matrix differentials, nonconvex tools (B) | Project milestone 2 in; Hw 5 out |

Thurs Nov 14 | 23. Dual methods & ADMM (R) | |

Tues Nov 19 | 24. Coordinate-wise minimization (R) | |

Thurs Nov 21 | 25. Nonconvex? No problem! (R) | |

Tues Nov 26 | 26. Large-scale optimization (Guest) | Hw 5 in |

Thurs Nov 28 | No class (Thanksgiving) | |

Tues Dec 3 | Little test (covers up to lec 25) | Little test |

Thurs Dec 5 | No class (work on projects) | |

Fri Dec 13, 1-4pm | Final project presentations | Final project in |

Top