Recommended Books

Mon Aug 28
To read more about what I referred to as the "master theorem on the asymptotics of parametric models" see these notes by Jon Wellner. In particular, I highly recommend looking at the excellent notes he made for the sequence of three classes on theoretical statistics he has been teaching at the University of Washington.

Parameter consistency and central limit theorems for models with increasing dimension d (but still d < n): Some central limit theorem results in increasing dimension:

Wed Aug 30
Some references to concentration inequalities: For a comprehensive treatment of sub-gaussian variables and processes (and more) see: Finally, here is the traditional bound on the mgf of a centered bounded random variable (due to Hoeffding), implying that bounded centered variables are sub-Guassian. It should be compared to the proof given in class.

Wed Sep 6
References for Chernoff bounds for Bernoulli (and their multiplicative forms):

Mon Sep 11
For an example of the improvement afforded by Bernstein versus Hoeffding, see Theorem 7.1 of available here. By the way, this is an excellent book. For sharp tail bounds for chi-squared see: For a more detailed treatment of sub-exponential variables and sharp calculations for the corresponding tail bounds see: For a detailed treatment of Chernoff bounds, see:

Wed Sep 13
For some refinement of the bounded difference inequality and applications, see: For a comprehensive treatment of density estimation under the L1 norm see the book see:

Mon Sep 25
For matrix estimation in the operator norm depending on the effective dimension, see For a treatment of the matrix calculus concepts needed for proving matrix concentration inequalities (namely operator monotone and convex matrix functions), see: To read up about matrix concentration inequalities, I recommend:

Mon Oct 2
To see how Matrix Bernstein inequality can be used in the study of random graphs, see Tropp's monograph and this readable reference: To see how Matrix Bernstein inequality can be used to analyze the performance of spectral clustering for the purpose of community recovery under a stochastic block model, see this old failed NIPS submission (in particular, the appendix). Finally, this is a paper on linear regression every Phd students in statistics (and everyone taking this class) should read:

Mon Oct 9
A nice reference on ridge and least squares regression with random covariate is For further references on rates for the lasso, restricted eigenvalue conditions, oracle inequalities, etc, see

Wed Oct 11
A highly recommended book dealing extensively with the normal means problem is

Wed Oct 18
To read about persistence and related concepts see: A highly recommended book dealing extensively with the normal means problem is

Mon Oct 23
Good references on perturbation theory are The following version of Davis-Kahan is epically useful

Wed Oct 25
Good modern references on PCA:

Wed Nov 8
Good references on ULLN:

Mon Nov 13
For relative VC deviations see: For Talagrand's inequality, see, e.g.,