36755, Fall 2016 Class Schedule  
Date  Lecture Topic  Readings  Scribe Notes  Notes 

Aug 29, M  Introduction: highdimensional statistical models 


Aug 31, M  SubGaussian variables and Hoeffding Inequality 


Sep 7, W  SubExponential variables and Bernstein Inequality 


Sep 12, M  Maxima, Quadratic Forms, Martingales 


Sep 14, W  The bounded difference inequality with applications. Concentration of Lipschitz functions of Gaussians. 

HW1 is due. Here are the Solutions  
Sep 19, M  Covering and packing numbers, metric entropy volumetric ratios, concentration of maxima over infinite sets. 


Sep 21, W  Matrix algebra review. covariance estimation in the operator norm. 


Sep 26, M  Upper bound on the prediction consistency of least squares and penalized least squares. 

HW2 is due. Here are the Solutions  
Sep 28, W  Slow rates for the lasso. Restricted eigenvalue condition for fast rates. 


Oct 3, M  Class canceled.  
Oct 5, W  Fast rates for the lasso. Oracle inequality for least squares. 

HW3 is due. Here are the Solutions  
Oct 10, M  Persistence and Introduction to PCA 


Oct 12, W  DavisKahan theorem and application to eigenspace estimation.  See references page.  
Oct 17, M  Eigenvector recovery for sparse PCA under the spiked covariance model. Introduction to the uniform law of large numbers. 


Oct 19, W  Uniform law of large numbers via Rademchaer complexities. 

HW4 is due. Here are the Solutions  
Oct 24, M  Polynomial discrimination, shattering coefficients, VC dimension, Sauer Lemma and VC inequality. Examples of VC classes of sets. 


Oct 26, W  Classic derivation of the VC inequality, relative deviations, extension to functions, Talagrand inequality for empirical processes and separability. 


Oct 31, M  SubGaussian processes and onestep discretization technique with applications. 

HW5 is due. Here are the Solutions  
Nov 2, W  Chaining and Dudley entropy integral bound. 


Nov 7, M  Orlicz processes and their concentration properties. 


Nov 9, W  Nonparametric least squares. Critical radius from local Gaussian complexity. 

HW6 is due. Here are the Solutions  
Nov 14, M  Nonparametric least squares. Critical radius from local Gaussian complexity. 


Nov 16, W  Oracle Inequality for Nonparametric least squares. 


Nov 21, M  Uses of Oracle Inequality for Nonparametric least squares. Introduction to Ustatistics. 


Nov 28, M  Examples of Ustatistics. Variance of a Ustatistics. 


Nov 30, W  Hajek projection and asymptotic normality of Ustatistics. 

HW7 is due. Here are the Solutions  
Dec 5, M  Concentration of Ustatistics and review.  
Dec 7, W  Final exam.  Here are the Solutions 