---
title: Causality from Time Series
date: 10 December 2020 (Lecture 27)
author: 36-467/667 (Fall 2020)
output: slidy_presentation
bibliography: locusts.bib
---

\newcommand{\Expect}[1]{\mathbb{E}\left[ #1 \right]}
\newcommand{\Prob}[1]{\mathbb{P}\left( #1 \right)}
\newcommand{\indep}{\perp}

## House-keeping

- HW 13 due tomorrow (Friday) at 6 pm
- FCEs

## Causal vs. Ordinary Prediction

> - In ordinary statistical prediction, we look at $\Expect{Y|X=x}$
>     + or $Y$'s distribution, etc.
> - This is equivalent to picking out the sub-population where $X=x$ and then calculating the average $Y$
> - Genuine differences between sub-populations can be very far from causal
>    + Sub-population of bright red cars is driven faster than the others
>    + Sub-population of middle-aged men who have recently bought bright red cars is more likely to be recently divorced
>    + Sub-population of people who have seen an ad for $Z$ online are more likely to buy $Z$ within the next month (say)
>    + Sub-population of people whose friends became obese within the last year are more likely to become obese themselves
> - Causality is **counter-factual**: what would happen if, hypothetically, something (or Someone) set $X=x$, over-riding the usual processes?

## Graphical Causal Models

- Every variable gets its own node in the graph or network
- There are _directed_ edges or arrows from node to node
- The nodes you get arrows from are your **parents**; the nodes you send arrows to are your **children**
    + Parents' parents, etc. = ancestors; childrens' children, etc = descendants
- Every variable is a function of its parents and independent noise
- Assume no cycles (**directed acyclic graph** or DAG)
- Then: every variable is independent of its non-descendants given its parents (Markov property)
\[
\Prob{V_1 = v_1, V_2=v_2, \ldots V_p=v_p} = \prod_{k=1}^{p}{\Prob{V_k = v_k| V_{\mathrm{Parents}(k)} = v_{\mathrm{Parents}(k)}}}
\]


## Some Time Series Models in DAG Form

- Markov chain:

![](mm-dag.png)

- Hidden Markov model or state-space model:

![](hmm-dag.png)

- Observation-driven model (or "chain with complete connections"):

![](ccc-dag.png)

## Expressing Causality in DAGs

- We want $\Prob{V_i|do(V_j=x)}$, i.e., we want to know the distribution of $Y$ if we somehow reached in and set $V_j$ to the value $x$, whatever $V_j$ might normally be
- We calculate this from the graph by **surgery**:
    + Delete all the edges into $V_j$
	+ Fix $V_j$ at the value $x$
	+ Leave everything else alone
	+ Calculate the joint distribution in the new, surgically-altered graph
	+ Add up over all the variables other than $V_i$
- Similarly if we set multiple variables at once
- Setting descendants of $V_i$ has no effect on its distribution
    + But conditioning on descendants _does_ give information about $V_i$



## Graphical Models for Variables Evolving over Time

- We just looked at a few!
- Usually a good idea to unpack the variables, so things look more like this:

![](unrolled-dag.png)

(from @Chu-Glymour-additive-time-series, Fig. 1)

## Graphical Models for Variables Evolving over Time

- Usually assume a constant, repetitive structure over time, so we can "roll up" the big graph into repetitions of a template:

![](rolled-dag.png)

(from @Chu-Glymour-additive-time-series, Fig. 1)



## So Where Does the Graph Come From?

> - Intuition / dreams / visions
> - Scientific theories
> - Data analysis

## Granger "Causality": A Diversion

> - Back in the 1960s, C. W. J. Granger proposed a test for causality between time series: $X$ Granger-causes $Y$ if $Y(t)$ is better predicted by the past of $Y$ _and_ $X$ than by the past of $Y$ alone
> - The original test used linear predictors but there are plenty of nonlinear variants, if you want them
>    + I helped commit one a long time ago [@Info-Coh-for-NIPS]
> - Granger was a great econometrician but this wasn't one of his better ideas
> - If $Y$ really causes $X$, but we measure $Y$ imprecisely, $X$ can Granger-cause $Y$!
> - Granger causality is (IMHO) a waste of time, _if you are interested in causality_
>    + It's exactly what you want if you just want to predict...

## What's Really Needed Is Conditional Independence

> - Remember: Any $V$ is independent of its non-descendants, given its parents (Markov property)
> - Implies: Some additional conditional independence relations (which can be read off from the graph by rules)
>    + Basically: $V_i \indep V_j | V_k$ if conditioning on $V_k$ "blocks" all the paths between $V_i$ and $V_j$ (with complications in the rules about when paths are blocked or not)
> - Assume: There are no other conditional independence relations (**faithfulness** to the graph)
> - If we can find $V_k$ where $V_i \indep V_j | V_k$, then there can't be an edge between $V_i$ and $V_j$
> - If nothing makes $V_i$ and $V_j$ independent, then there's an edge between them (or there's a hidden common cause)
> - Start with edges between everything, then remove them if we can make variables independent by conditioning [@Spirtes-Glymour-Scheines-1st; @Spirtes-Glymour-Scheines; @Kalisch-Buhlmann-on-PC-alg; @pcalg; @Maathuis-Kalisch-Buhlmann-IDA; @Colombo-et-al-RFCI]
>    + There are tricks for then getting the direction of the edges

## Unfortunately, Conditional Independence Testing Is Hard

> - Testing whether $V_i \indep V_j | V_k$ is easy for categorical variables ($\chi^2$ test)
> - Testing whether $V_i \indep V_j | V_k$ is easy if everything's Gaussian and linear (regress $V_i$ on $V_j$ and $V_k$, look at coefficient on $V_j$)
>    + Or do something more symmetric-looking with partial correlation
> - The general case is very hard, though there are ways to do it [@Zhang-et-al-kernel-based-CI-tests]

## Additive Autoregressions

- Think back to our [additive autoregressions from Lecture 24](http://www.stat.cmu.edu/~cshalizi/dst/20/lectures/24/lecture-24.html#(12))
\[
X(t+1) = \epsilon(t+1) + \alpha + \sum_{s=1}^{d}{f_i(X(t+1-s))}
\]
- It's easy to extend this to multiple variables, so we'd say something like
\[
V_i(t+1) = \epsilon_i(t+1) + \alpha_i + \sum_{s=1}^{d}{\sum_{j=1}^{p}{f_{sj}(V_j(t+1-s)}}
\]
with $f_{sj} = 0$ unless $V_j(t+1-s)$ is a parent of $V_i(t+1)$
- @Chu-Glymour-additive-time-series gave a search procedure for causal graphs for time series, assuming an additive model is right


## Additive Autoregressions

![](discovered-dag.png)


## Summing Up Today

> - Causal inference is about what-would-happen-if-this-were-changed, not about what-is-associated-with-this
> - Causal structure is represented by graphical models
> - Making this work over time involves "unrolling"
> - Causal structure can be discovered, or constrained, from observations

## Summing Up the Course

#### Spatiotemporal dependence is great

- It lets us predict _there_ and _then_ from _here_ and _now_ (or vice versa)
- It lets us learn about structure and process and how the world works
- The dependence is often what we most want to infer, statistically

#### Spatiotemporal dependence is a drag

- It messes up our basic statistical theory
- It (usually) reduces our effective sample size
- It forces us to learn more complicated theory and methods

#### Spatiotemporal dependence can be dealt with

- We need to recognize the dependence
- We need to build models of dependence
- We need to simulate those models



## References

