Fall 2017: Intermediate Statistics (36-705)

Instructor Information

Instructor: Siva Balakrishnan
Email: siva@stat.cmu.edu
Office Hours: Mondays 1:30PM - 2:30PM
Location: BH 132K

TA Information

TA: Purvasha Chakravarti
Email: pchakrav@andrew.cmu.edu
Office Hours: Wednesday 6-7pm
Location: BH 132Q

TA: Zongge Liu
Email: zonggel@andrew.cmu.edu
Office Hours: Tuesday 2-3pm
Location: BH 132A


TA: Shengming Luo
Email: shengmingluo@cmu.edu
Office Hours: Tuesday 6-7pm
Location: BH 132Q


TA: Daren Wang
Email: darenw@andrew.cmu.edu
Office Hours: Wednesday 1.30-2.30pm
Location: BH 132Q

Course Description

This course covers the fundamentals of theoretical statistics. Topics include: concentration of measure, basic empirical process theory, convergence, point and interval estimation, maximum likelihood, hypothesis testing, Bayesian inference, nonparametric statistics and bootstrap re- sampling. This course is excellent preparation for advanced work in Statistics and Machine Learning. See below for a detailed schedule.

Course Syllabus

The syllabus provides information on grading, class policies etc.

Course Calendar

The calendar has an approximate week-by-week schedule. Consult this document to know when the in-class exams are.

Lecture Notes

  • Lecture 1: (8/28) A very brief review
  • Lecture 2: (8/30) Concentration inequalities
  • Lecture 3: (9/1) Concentration inequalities
  • Lecture 4: (9/6) Convergence
  • Lecture 5: (9/8) Convergence
  • Lecture 6: (9/11) Central limit theorem
  • Lecture 7: (9/13) Uniform laws and empirical process theory
  • Lecture 8: (9/15) Uniform laws and empirical process theory
  • Lecture 9: (9/18) VC dimension
  • Lecture 10: (9/20) Rademacher complexity
  • Lecture 11: (9/25) Sufficiency
  • Lecture 12: (9/27) Minimal Sufficiency and Rao-Blackwell
  • Lecture 13: (9/29) Exponential Families
  • Lecture 14: (10/2) Parametric Estimation
  • Lecture 15: (10/4) Fisher Information and Decision Theory
  • Lecture 16: (10/6) Decision Theory
  • Lecture 17: (10/9) Minimax and Consistency of MLE
  • Lecture 18: (10/11) Asymptotic Normality of MLE
  • Lecture 19: (10/13) Hypothesis Testing
  • Lecture 20: (10/16) More Hypothesis Testing
  • Lecture 21: (10/18) LRT and Permutation Cut-offs
  • Lecture 22: (10/23) Multiple Testing
  • Lecture 23: (10/25) FDR and Confidence Sets
  • Lecture 24: (10/29) Confidence Intervals
  • Lecture 25: (11/1) Confidence Intervals and Causal Inference
  • Lecture 26: (11/3) Causal Inference
  • Lecture 27: (11/6) Non-parametric Regression
  • Lecture 28: (11/8) High-dimensional Statistics
  • Lecture 29: (11/13) More High-dimensional Statistics
  • Lecture 30: (11/15) Bayesian Inference
  • Lecture 31: (11/17) MCMC
  • Lecture 32: (11/27) MCMC and Bootstrap
  • Lecture 33: (11/29) Bootstrap and Model Selection
  • Lecture 34: (12/1) Model Selection
  • Lecture 35: (12/4) Distances between Distributions
  • Lecture 36: (12/6) (Completely Optional Lecture) Research in Theoretical Statistics/Statistical Machine Learning

  • Lecture Notes Source Files

    Assignments

  • Assignment 1: Due on 8/31 at 3pm.
  • Assignment 2: Due on 9/7 at 3pm.
  • Assignment 3: Due on 9/14 at 3pm.
  • Assignment 4: Due on 10/5 at 3pm.
  • Assignment 5: Due on 10/13 at 3pm.
  • Assignment 6: Due on 10/19 at 3pm.
  • Assignment 7: Due on 11/10 at 3pm.
  • Assignment 8: Due on 11/27 at 3pm.
  • Assignment 9: Due on 12/8 at 3pm.

  • Assignment solutions

  • Assignment 1
  • Assignment 2
  • Assignment 3
  • Assignment 4
  • Assignment 5
  • Assignment 6
  • Assignment 7
  • Assignment 8
  • Assignment 9
  • Practice Exam 1

    Practice Exam 2

    Practice Final

    Exam 1

    Exam 2

    Exam 1 Solutions

    Exam 2 Solutions