Fall 2019: Intermediate Statistics (36-705)

Instructor Information

Instructor: Siva Balakrishnan
Email: siva@stat.cmu.edu
Office Hours: Mondays 1:30PM - 2:30PM
Location: BH 132K

TA Information

TA: Aleksandr Podkopaev
Email: apodkopa@andrew.cmu.edu
Office Hours: TBA
Location: TBA

TA: Ilmun Kim
Email: ilmunk@andrew.cmu.edu
Office Hours: TBA
Location: TBA


Course Description

This course covers the fundamentals of theoretical statistics. Topics include: concentration of measure, basic empirical process theory, convergence, point and interval estimation, maximum likelihood, hypothesis testing, Bayesian inference, nonparametric statistics and bootstrap re- sampling. This course is excellent preparation for advanced work in Statistics and Machine Learning. See below for a detailed schedule.

Course Syllabus

The syllabus provides information on grading, class policies etc.

Lecture Notes

  • Lecture 1: (8/26) A very brief review
  • Lecture 2: (8/28) Concentration inequalities
  • Lecture 3: (8/30) More concentration inequalities
  • Lecture 4: (9/4) Even more concentration inequalities
  • Lecture 5: (9/6) Convergence of random variables
  • Lecture 6: (9/9) More Convergence and the CLT
  • Lecture 7: (9/11) CLT and variants
  • Lecture 8: (9/13) Uniform Laws
  • Lecture 9: (9/16) Applications of Uniform Convergence
  • Lecture 10: (9/18) Rademacher Complexity
  • Lecture 11: (9/23) Sufficient Statistics
  • Lecture 12: (9/25) Minimal Sufficiency
  • Lecture 13: (9/27) Exponential Families
  • Lecture 14: (9/30) Constructing Estimators
  • Lecture 15: (10/2) Cramer-Rao Lower Bound
  • Lecture 16: (10/4) Decision Theory Basics
  • Lecture 17: (10/7) Bounding the Minimax Risk
  • Lecture 18: (10/9) Consistency of the MLE
  • Lecture 19: (10/11) Asymptotic Normality of the MLE
  • Lecture 20: (10/14) Hypothesis Testing and the Neyman-Pearson Lemma
  • Lecture 21: (10/16) General Purpose Tests
  • Lecture 22: (10/21) LRT, Two-Sample Testing, Permutation Test
  • Lecture 23: (10/23) Multiple Testing
  • Lecture 24: (10/28) More Multiple Testing
  • Lecture 25: (10/30) Confidence Intervals
  • Lecture 26: (11/4) Confidence Intervals and the Bootstrap
  • Lecture 27: (11/6) More on the Bootstrap
  • Lecture 28: (11/8) Causal Inference
  • Lecture 29: (11/11) More Causal + Non-Parametric Regression
  • Lecture 30: (11/13) High-dimensional Statistics
  • Lecture 31: (11/15) Linear Regression
  • Lecture 32: (11/18) Non-parametric Regression
  • (11/20) Class Canceled
  • Lecture 33: (11/22) Bayesian Inference
  • (11/25)-(11/27)-(11/29) Thanksgiving Break
  • Lecture 34: (12/2) Model Selection
  • Lecture 35: (12/4) Distances Between Distributions
  • Lecture 36: (12/6) Fano's inequality and more distances

  • Assignments

  • Assignment 1: Due on 8/30 at 3pm.
  • Assignment 2: Due on 9/5 at 3pm.
  • Assignment 3: Due on 9/12 at 3pm.
  • Assignment 3 Source
  • Assignment 4: Due on 10/3 at 3pm.
  • Assignment 5: Due on 10/10 at 3pm.
  • Assignment 6: Due on 10/25 at 3pm.
  • Assignment 7: Due on 11/8 at 3pm.
  • Assignment 7 Source
  • Assignment 8: Due on 11/19 at 3pm.
  • Assignment 8 Source
  • Assignment 9: Due on 12/6 at 3pm.
  • Assignment 9 Source

  • Assignment solutions

  • Assignment 1
  • Assignment 2
  • Assignment 3
  • Assignment 4
  • Assignment 5
  • Assignment 6
  • Assignment 7
  • Assignment 8
  • Assignment 9

  • Practice Exam 1

    Solutions to Practice Exam 1

    Practice Exam 2

    Solutions to Practice Exam 2

    Practice Final (Ignore MCMC question, No Solutions this time)

    Solutions to Midterm 1

    Solutions to Midterm 2