---
title: Inference II --- Ergodic Theory
author: 36-467/667
date: 15 October 2020 (Lecture 14)
output: slidy_presentation
bibliography: locusts.bib
---

\[
\newcommand{\Expect}[1]{\mathbb{E}\left[ #1 \right]}
\newcommand{\Var}[1]{\mathrm{Var}\left[ #1 \right]}
\newcommand{\Cov}[1]{\mathrm{Cov}\left[ #1 \right]}
\newcommand{\Prob}[1]{\mathbb{P}\left[ #1 \right]}
\newcommand{\TrueRegFunc}{\mu}
\newcommand{\EstRegFunc}{\widehat{\TrueRegFunc}}
\DeclareMathOperator*{\argmin}{argmin}
\newcommand{\TrueNoise}{\epsilon}
\newcommand{\EstNoise}{\widehat{\TrueNoise}}
\]

```{r, include=FALSE}
# General set-up options
library(knitr)
opts_chunk$set(size="small",background="white", highlight=FALSE,
               cache=TRUE, autodep=TRUE,
               tidy=TRUE, warning=FALSE, message=FALSE,
               echo=FALSE)
```


## In our last episode...

- We typically estimate a quantity $\psi$ by minimizing a partly-random objective function $M_n(\psi)$, so $\hat{\psi}_n = \argmin_{\psi}{M_n(\psi)}$
- Generally, $\hat{\psi}_n \rightarrow \psi_0$ (consistent estimation) if two things are true:
    + $M_n(\psi) \rightarrow m(\psi)$ as $n\rightarrow\infty$
    + and the true $\psi_0 = \argmin_{m}{m(\psi)}$
- Generally, $\Var{\hat{\psi}_n} \rightarrow \mathbf{h}^{-1} \Var{\nabla M_n(\psi_0)} \mathbf{h}^{-1}$ (sandwich covariance) if
    + $\nabla M_n(\psi_0) \rightarrow \nabla m(\psi_0) = 0$
    + and $\nabla \nabla M_n(\psi_0) \rightarrow \nabla\nabla m(\psi_0) \equiv \mathbf{h}$
- With IID data, we use the law of large numbers to ensure $M_n \rightarrow m$
    + We use the central limit theorem to ensure $M_n \rightsquigarrow \mathcal{N}$

## Agenda for today

- Convergence of not-too-correlated sample averages on expectations
    + A "mean ergodic theorem"
    + Stationary and non-stationary versions
    + Notion of effective sample size and correlation time
- Inference for AR(1)
- Some glimpses at more advanced ergodic theory (without proofs)
    + Convergence of the log-likelihood
    + Weak dependence and CLTs


## Ergodic theory

- Laws of large numbers for dependent variables are called **ergodic theorems**
    + Blame Ludwig Boltzmann in the late 1800s
- This has absorbed a lot of mathematical talent over the last $\approx 150$ years [@von-Plato-modern-prob]
- We (= you) can prove a useful one over the next few minutes

## Second-order stationary and not-too-correlated

- Assume $X(1), X(2), \ldots X(t), \ldots$ is (second-order) stationary:
   + $\Expect{X(t)} = \mu$ for all $t$
   + $\Cov{X(t), X(t+h)} = \gamma(h)$ for all $t, h$
- Assume the sum of the covariances is finite:
\[
\sum_{h=-\infty}^{\infty}{\gamma(h)} \equiv \gamma(0)\tau < \infty
\]
    + **Correlation time** (also) refers to this $\tau \equiv \frac{\sum_{h=-\infty}^{\infty}{\gamma(h)}}{\gamma(0)}$
        * Other names are "autocorrelation time", "integrated autocorrelation time", "integral time scale", etc.
        * Sometimes slight variations in the definition, such as $\frac{\sum_{h=0}^{\infty}{\gamma(h)}}{\gamma(0)}$ or $\frac{\sum_{h=1}^{\infty}{\gamma(h)}}{\gamma(0)}$; check this when combining results from different authors
    + **Decay of correlations**: finite sum implies $\gamma(h) \rightarrow 0$ as $h\rightarrow\infty$


## Our first ergodic theorem {.smaller}

\begin{eqnarray}
\overline{X}_n & \equiv & \frac{1}{n}\sum_{t=1}^{n}{X(t)}\\
\Expect{\left(\overline{X}_n - \mu\right)^2} & = & \left(\Expect{\overline{X}_n - \mu}\right)^2 + \Var{\overline{X}_n}\\
\Expect{\overline{X}_n} & = & \frac{1}{n}\sum_{t=1}^{n}{\Expect{X(t)}} = \mu\\
\Var{\overline{X}_n} & = & \frac{1}{n^2}\left(\sum_{t=1}^{n}{\Var{X(t)}}
+ 2\sum_{t=1}^{n-1}{\sum_{s=t+1}^{n}{\Cov{X(t), X(s)}}}\right)\\
& = & \frac{1}{n^2}\left(n \gamma(0) + \sum_{t=1}^{n}{\sum_{s\neq t}{\gamma(t-s)}}\right)\\
& = & \frac{1}{n^2}\sum_{t=1}^{n}{\sum_{s=1}^{n}{\gamma(t-s)}}\\
& = & \frac{1}{n^2}\sum_{t=1}^{n}{\sum_{h=1-t}^{n-t}{\gamma(h)}} \\
& \rightarrow & \frac{1}{n^2}\sum_{t=1}^{n}{\sum_{h=-\infty}^{\infty}{\gamma(h)}}
 =  \frac{1}{n^2}\sum_{t=1}^{n}{\gamma(0)\tau} = \frac{\gamma(0)\tau}{n}
\end{eqnarray}

## Our first ergodic theorem {.smaller}

If $\tau < \infty$, then

\begin{eqnarray}
\Expect{\left(\overline{X}_n - \mu\right)^2} &\rightarrow & 0 + \frac{\gamma(0)\tau}{n} \rightarrow 0
\end{eqnarray}

$\Leftrightarrow$ If $\tau < \infty$, then

\begin{eqnarray}
\overline{X}_n \rightarrow \mu
\end{eqnarray}

## How sensible is $\tau < \infty$?

- Uncorrelated processes have $\tau = 1$
- Suppose $\gamma(h) = \gamma(0) \rho^{|h|}$ with $|\rho| < 1$ ([stationary AR(1) models have covariance functions like this](http://www.stat.cmu.edu/~cshalizi/dst/20/lectures/11/lecture-11.html#(11)))
\[
\sum_{h=-\infty}^{\infty}{\gamma(h)} = \gamma(0)\left(1+2\sum_{h=1}^{\infty}{\rho^h}\right) = \gamma(0)\left(1+2\frac{\rho}{1-\rho}\right)
\]
    + (sum a geometric series)
    + So $\tau = (1+\rho)/(1-\rho)$
    + Continuity with the uncorrelated case if $\rho \approx 0$
- In general, stationary AR(p) and VAR(p) models have $\tau < \infty$
    + So do lots of non-linear models we'll meet later in the course

## Not every process has $\tau < \infty$:

- An easy but extreme counter-example:
    + $X(1) \sim$ anything, $X(t+1) = X(t)$ $\Rightarrow$ $\tau = \infty$
    + "Checking a newspaper by buying more copies"
- Also bad:
    + $X(1) \sim$ anything, $X(t) = X(1) + \epsilon(t)$ with $\epsilon$s IID
    + "Checking a newspaper by buying smudged copies"
- More troublesome: very slow decay of correlations
   + $\gamma(h) \propto h^{-\alpha}$ is a "long-memory process", or one with "long-range correlations"
   + $\lim_{T\rightarrow\infty}{\sum_{h=-T}^{T}{\gamma(h)}}=\infty$ if $\alpha \leq 1$
   + But $\gamma(h)$ is summable if $\alpha > 1$


## Effective sample size {.smaller}

- For uncorrelated $X_i$, we saw [last time](http://www.stat.cmu.edu/~cshalizi/dst/20/lectures/13/lecture-13.html#(10)) that
\[
\Expect{\left(\overline{X}_n - \mu\right)^2} = \frac{\Var{X(1)}}{n}
\]
- We just showed that if $\tau < \infty$, then
\[
\Expect{\left(\overline{X}_n - \mu\right)^2} = \frac{\Var{X(1)}\tau}{n}
\]
- Equivalently,
\[
\Expect{\left(\overline{X}_n - \mu\right)^2} = \frac{\Var{X(1)}}{n/\tau}
\]
- _As though we had $n/\tau$ uncorrelated observations, instead of $n$ dependent ones_
- For the $\gamma(h) = \gamma(0) \rho^{|h|}$ situation:

```{r}
curve(1+2*x/(1-x), from=0, to=0.99, xlab=expression(rho), ylab="correlation time", main="Factor reducing effective sample size", log="y")
```



## Generalizing: non-stationary case

- We don't actually need stationarity!
- Define $\mu_n \equiv \frac{1}{n}\sum_{t=1}^{n}{\Expect{X(t)}}$
- Define $V(n) \equiv \sum_{t=1}^{n}{\sum_{s=1}^{n}{\Cov{X(t), X(s)}}}$
- Then if $V(n) = o(n^2)$
\[
\Expect{\left(\overline{X}_n - \mu_n\right)^2} \rightarrow 0
\]
    + By exactly the same proof


## Application: Stationary AR(1)

- $X(t) = a + b X(t-1) + \epsilon(t)$
- Assume stationary, so $\Expect{X(t)} = \frac{a}{1-b}$ and $\Cov{X(t), X(t+h)} = b^{|h|} \frac{\Var{\epsilon}}{1-b^2}$
- Then $\tau = 1+ 2\frac{b}{1-b}$
   + N.B.: $> 0$ for any $b \in (-1, 1)$
- So $\overline{X}_n \rightarrow \Expect{X(1)} = \frac{a}{1-b}$ (but more slowly than for uncorrelated data)
- Similarly
\[
\hat{\gamma}(h) \equiv \frac{1}{n-h}\sum_{t=1}^{n-h}{(X(t) - \overline{X}_n)(X(t+h)-\overline{X}_n)} \rightarrow \Cov{X(t), X(t+h)} = \gamma(h)
\]

## Application: Not-necessarily-stationary AR(1)

- Back in [Lecture 12](http://www.stat.cmu.edu/~cshalizi/dst/20/lectures/12/lecture-12.html#(15)), saw that if $a=0$ and we use OLS
\begin{eqnarray}
\hat{b} & = & b +  \frac{\sum_{t=0}^{n-1}{X(t)\epsilon(t+1)}}{\sum_{t=0}^{n-1}{X^2(t)}}\\
& = & b +  \frac{n^{-1}\sum_{t=0}^{n-1}{X(t)\epsilon(t+1)}}{n^{-1}\sum_{t=0}^{n-1}{X^2(t)}}
\end{eqnarray}
- But now the numerator $\rightarrow \Expect{X(t) \epsilon(t+1)} = 0$
   + because $\Cov{X(t) \epsilon(t+1), X(t+1)\epsilon(t+2)} = 0$

## Application: AR(1)

- Objective function at finite $n$:
\[
M_n(a,b) = \frac{1}{n-1}\sum_{i=1}^{n-1}{(X(t+1) - a - b X(t))^2}
\]
- Exercise (off-line): Assume stationarity and show that this goes to
\[
m(a,b) = \Expect{(X(t+1) - a-bX(t))^2}
\]
    + (Can you do this under non-stationarity?)
- So all our asymtptotic analysis from last time applies


## Looking beyond the simplest ergodic theorem

- More general "mean-square" ergodic theorem for weakly stationary processes: $\overline{X}_n \rightarrow \Expect{X(1)}$ iff $n^{-1}\sum_{h=0}^{n}{\gamma(h)} \rightarrow 0$
    + If $\tau < \infty$, the rate of convergence is $O(1/n)$ just as with uncorrelated data
    + But one can construct processes where $\overline{X}_n$ converges arbitrarily slowly
- A process is **strongly** stationary, or **strictly stationary**, when for any $k$ and any $h$,
\[ \Prob{X(1) = x_1, X(2) = x_2, \ldots X(k)=x_k} = \Prob{X(h+1) = x_1, X(h+2) = x_2, \ldots X(h+k) = x_k}
\]
- The **individual ergodic theorem**: If the process is strongly stationary, then for any $k$ an d for any function $f(X(1), X(2), \ldots X(k))$,
\[
\Prob{\frac{1}{n-k}\sum_{t=1}^{n-k}{f(X(t), \ldots X(t+k))} \rightarrow \Expect{f(X(1),  \ldots X(k))}} = 1
\]
    + i.e., sample averages converge along (almost all) _individual_ trajectories
    + Again, there are processes where the rate of convergence is arbitrarily slow, but they're a bit "pathological"
- These two theorems also work for _asymptotically_ stationary processes
    + because the long-run limit is dominated by the stationary process we're approaching
    + Also for cyclo-stationary and asymptotically cyclo-stationary processes

## Convergence of the log-likelihood

- Assume a pdf $p(x(1), x(2), \ldots x(t))$ generates $X(1), X(2), \ldots X(t)$
- Assume $X$ is stationary
- Then
\[
\lim_{n\rightarrow\infty}{\frac{1}{n}\Expect{\log{p(X(1), \ldots X(n))}}} = \lambda
\]
exists, and
\[
\frac{1}{n}\log{p(X(1), \ldots X(n))} \rightarrow \lambda
\]
    + "Asymptotic equipartiton" or "Shannon-McMillan-Breiman" property from information theory

## Convergence of the log-likelihood (II)

- Assume $X$ is generated by a distribution $p$
- Consider a _model_ pdf $f(x(1), x(2), \ldots x(t); \theta)$
- Then, under stationarity,
\begin{eqnarray}
\lim_{n\rightarrow\infty}{\frac{1}{n}\Expect{\log{f(X(1), \ldots X(n);\theta)}}} &  = & \ell(\theta)\\
\frac{1}{n}\log{f(X(1), \ldots X(n); \theta)} & \rightarrow & \ell(\theta)
\end{eqnarray}
    + And $\ell(\theta) < \lambda$, unless the model is right at some $\theta_0$
- Some non-stationary extensions, especially if asymptotically stationary

## Central limit theorems and weak dependence

- Suppose $(X(t-k), \ldots X(t-1))$ and $(X(t+h), \ldots X(t+h+k-1)$ approach independence as $h\rightarrow\infty$
- Then we've got nearly independent "blocks" of length $k$
- $\overline{X}_n$ acts like average of some nearly-independent blocks, plus some corrections/remainders/fudge
- This lets us transfer the central limit theorem to dependent data
- We need to be precise about "approaching independence" (see backup)

## Summary

- If correlations go to zero fast enough (e.g., $\tau < \infty$), then $\overline{X}_n$ converges to the expectation value
- This is enough to have a lot of estimators converge
    + And even to get asymptotic standard errors
    + Effective sample size is reduced from $n$ to $n/\tau$
- Maximum likelihood works very generally for parametric models
- Central limit theorem still holds under weak dependence / asymptotic independence

## Backup: Boltzmann

```{r, fig.retina=NULL, out.width=400, echo=FALSE}
knitr::include_graphics("Boltzmanns_Tomb.jpg")
```


(Photo credit: Tom Schneider, downloaded 2008 from an [apparently-defunct](http://www.ccrnp.ncifcrf.gov/~toms/icons/aust2002/photos-by-tds/all/index.105.html) website)


## Backup: "Ergodic", "Ergodicity" {.smaller}

- Back in the late 1800s, Boltzmann was interested  in the behavior of a physical system (e.g., the molecules of a gas) at constant energy (English translation: @Boltzmann-gas-theory)
- Write:
    + $X(t)$ for the state of the system at time $t$
    + $E(x)$ for the energy of a system in state $x$
    + $R =$ set of states with $E(x) = E(X(0))$
    + $W=$ volume of $R$ (the same $W$ as on his tomb in the previous slide)
- Boltzmann wanted to say that for any (well-behaved) set of states $A$,
\[
\frac{1}{T}\int_{t=0}^{T}{\mathbf{1}(X(t) \in A) dt} \rightarrow \int_{R}{\mathbf{1}(x \in A) \frac{1}{W} dx}
\]
    + so the average along a path $X(t)$ equals the average over the region of constant energy
    + _ergon_ = "work, energy" (as in "erg", the unit of energy), _hodos_ = "way, path" (as in "odometer") $\Rightarrow$ "ergodic" = energy-path
- The name stuck
- A process is ergodic when averages over time converge on expectation values
- An ergodic theorem is one showing some class of processes (or functions of processes, etc.) is ergodic

## Backup: More on ergodic theory

- Early history: @von-Plato-modern-prob
- Gentlest introduction, and a really good complement to this class: @Grimmett-Stirzaker, chapter 9
- Some of how ergodicity supports using probability theory in the real world: @Ruelle-chance
- The result I called "Our first ergodic theorem" goes back to @Taylor-diffusion-by-continuous-movements
    + Taylor was a physicist specializing in fluid mechanics [@Batchelor-on-Taylor]; it's the same Taylor as in the ["Taylor hypothesis" for spatio-temporal correlation functions](http://www.stat.cmu.edu/~cshalizi/dst/20/lectures/09/lecture-09.html#(19)), but not the same as in "Taylor series"
    + In continuous time, you integrate the covariance function rather than summing it, which is how I first learned the result from @Frisch-turbulence, sec. 4.4, pp. 50--51
- Ergodicity in physics: @Lebowitz-central-issues, @Castiglione-Falcioni-Lesne-Vulpiani-chaos-and-coarse-graining
- Going deeper needs advanced (measure-theoretic) probability: @Gray-ergodic-properties-2nd builds up what's needed for ergodic theory
- Ergodic properties for log-likelihood are part of information theory:
    + @Cover-and-Thomas-2nd is the best over-all textbook
    + @Gray-entropy gives detailed coverage of this topic
- @Mackey-times-arrow is really about decay-of-dependence

## Backup: Weak dependence and central limit theorems {.smaller}

- "Mixing" is a strong notion of asymptotic independence
    + A strong notion of weak dependence, you should excuse the expression
- Measure dependence between $X(-\infty), \ldots X(t-1), X(t)$ and $X(t+h), X(t+h+1), \ldots X(+\infty)$ by "total variation" from independence:
\[
\beta(h) = \int{|p(x{-\infty:t}, x_{t+h:\infty}) - p(x_{-\infty:t})p(x_{t+h:\infty})| dx_{-\infty:t} dx_{t+h:\infty}}
\]
- The process is $\beta$-mixing if $\beta(h) \rightarrow 0$ as $h\rightarrow \infty$
- Now use "blocking":
    + Divide $X_1, \ldots X_n$ into $2m$ blocks of length $h$ (plus $< h$ extra observations as remainder)
    + Take the odd-numbered blocks; they're a random variable $Z = (Z_1, Z_2, \ldots Z_m)$
    + Imagine $\tilde{Z} = (\tilde{Z}_1, \ldots \tilde{Z}_m)$, where the blocks have the same marginal distribution but are all independent
    + For any event $A$, $|\Prob{Z \in A} - \Prob{\tilde{Z} \in A}| \leq m \beta(h)$ [@Yu-rates-of-convergence]
    + Since sample averages using $\tilde{Z} \rightsquigarrow \mathcal{N}$, so do sample averages using $Z$


## References




