Recommended Books

Lecture 1, Mon Aug 29
To read more about what I referred to as the "master theorem on the asymptotics of parametric models" see these notes by Jon Wellner. In particular, I highly recommend looking at the excellent notes he made for the sequence of three classes on theoretical statistics he has been teaching at the Unievrsity of Washington.

Parameter consistency and central limit theorems for models with increasing dimension d (but still d < n): Some central limit theorem results in increasing dimension (in the second mini we will see more specialized and stronger results).

Lecture 2, Wed Aug 31
Some references to concentration inequalities: For a comprehensive treatment of sub-gaussian variables and processes (and more) see: References for Chernoff bounds for Bernoulli (and their multiplicative forms): Finally, here is the traditional bound on the mgf of a centered bounded random variable (due to Hoeffding), implying that bounded centered variables are sub-Guassian. It should be compared to the proof given in class.

Lecture 4, Mon Sep 12
For an example of the improvement afforded by Bernstein versus Hoeffding, see Theorem 7.1 of available here. By the way, this is an excellent book. For details on the derivation of concentration inequality for quadratic forms of Gaussians, see For the Hanson-Wright inequality, see I strongly encourage to read the paper!

Lecture 7, Wed Sep 21
To read up about matrix concentration inequalities, I recommend: An excellent paper on the linear regression model. Recall: you almost never can make the assumption of linearity and the X is random!!

Lecture 9, Wed Sep 28
To read about ridge regression and lasso-type estimators a good reference is About uniqueness of the lasso (and other interesting properties): For the use of cross validation in selecting the lasso parameter see: And for the one standard error rule, which seems to work well in practice (but apparently has no theoretical justification), see these lecture by Ryan Tibshirani: pdf and pdf.

Lecture 10, Wed Oct 5
For further references on rates for the lasso, restricted eigenvalue conditions, oracle inequalities, etc, see Someone asked about references for selective inference. Here is a nicely compiled list of papers from the WHOA-PSI 2016 website, a very recent conference on this topic.

Lecture 11, Mon Oct 10
For persistence, see

Lecture 12, Wed Oct 12
Good references on perturbation theory are Below is a paper that very partially addresses the question in class about how can we know whether the eigengap condition holds.

Lecture 13, Mon Oct 17
For references on sparse PCA, see the following paper by Jing Lei and Vince Vu and references therein

Lecture 14, Wed Oct 19
Good references on ULLN:

Lecture 16, Mon Oct 26
For relative VC deviations see: For Talagrand's inequality, see, e.g.,

Lecture 19, Mon Nov 7
For Orlicz norms and processes, see:

Lecture 20, Mon Nov 7
For Local Radamacher Comlexiti Another good reference for non-parametric least squares is

Lecture 22, Mon Nov 14
To see that metric entropy of the star-hull of a class of function is, in most cases, of the same order as the metric entropy of the class itself, see, e.g., Lemma 4.5 in

Lecture 24, Mon Nov 21
References for U-Statistics (there is a huge literature on this topic; these are just few references):

Lecture 26, Mon Nov 21
Here is an article that gives a CLT for U-statistics with increasing order: For concentration inequalities for U-statistics, see