# Linear Prediction for Time Series

20 September 2018


# In our last episode

• General approach to optimal linear prediction
• Predict $$Y$$ from $$\vec{Z} = [Z_1, Z_2, \ldots Z_p]$$
• Prediction is $$\alpha + \vec{\beta} \cdot \vec{Z}$$
• Best $$\alpha = \Expect{Y} - \vec{\beta} \cdot \Expect{\vec{Z}}$$
• Best $$\beta = \Var{\vec{Z}}^{-1} \Cov{\vec{Z}, Y}$$
• Today: application to time series
• What can this do for us?
• How do we find the covariances?

# Optimal linear prediction for time series

• Given: $$X(t_1), X(t_2), \ldots X(t_n)$$
• Desired: prediction of $$X(t_0)$$
$\begin{eqnarray} \EstRegFunc(t_0) & = & \alpha + \vec{\beta} \cdot \left[\begin{array}{c} X(t_1) \\ X(t_2) \\ \vdots \\ X(t_n) \end{array}\right]\\ \alpha & = & \Expect{X(t_0)} - \vec{\beta} \cdot \left[\begin{array}{c} \Expect{X(t_1)}\\ \Expect{X(t_2)} \\ \vdots \\ \Expect{X(t_n)}\end{array}\right] ~ \text{(goes away if everything's centered)}\\ \vec{\beta} & = & {\left[\begin{array}{cccc} \Var{X(t_1)} & \Cov{X(t_1), X(t_2)} & \ldots & \Cov{X(t_1), X(t_n)}\\ \Cov{X(t_1), X(t_2)} & \Var{X(t_2)} & \ldots & \Cov{X(t_2), X(t_n)}\\ \vdots & \vdots & \ldots & \vdots\\ \Cov{X(t_1), X(t_n)} & \Cov{X(t_2), X(t_n)} & \ldots & \Var{X(t_n)}\end{array}\right]}^{-1} \left[\begin{array}{c} \Cov{X(t_0), X(t_1)}\\ \Cov{X(t_0), X(t_2)}\\ \vdots \\ \Cov{X(t_0), X(t_n)}\end{array}\right] \end{eqnarray}$
• What is this good for?

# Interpolation

• Time series often have gaps
• Instruments fail, people mess up, circumstances…
• What happened at the times in between the observations?

# Back to Kyoto

• A lot of this is just made up

# When did the cherries flower in 1015?

• We need $$\Cov{X(1015), X(t_i)}$$ for every year $$t_i$$ where we do have data
• We need $$\Expect{X(t_i)}$$, $$\Var{X(t_i)}$$ and $$\Cov{X(t_i), X(t_j)}$$ ditto
• We need $$\Expect{X(1015)}$$

# Similarly for extrapolation

• What was $$X(800)$$? (“retrodiction”)
• What was $$X(2016)$$?
• What will $$X(2019)$$ be? (“prediction” or “forecast” in the strictest sense)

# Getting the expectations and covariances

• We only see each $$X(t_i)$$ once
• Maybe gives us an idea of $$\Expect{X(t_i)}$$
• but not $$\Var{X(t_i)}$$ or $$\Cov{X(t_i), X(t_j)}$$
• let alone $$\Cov{X(t_0), X(t_i)}$$
• We could repeat the experiment many times
• We could make assumptions