Principal Components Analysis II

36-467/667

11 September 2018

\[ \newcommand{\X}{\mathbf{x}} \newcommand{\w}{\mathbf{w}} \newcommand{\V}{\mathbf{v}} \newcommand{\S}{\mathbf{s}} \newcommand{\Expect}[1]{\mathbb{E}\left[ #1 \right]} \newcommand{\Var}[1]{\mathrm{Var}\left[ #1 \right]} \newcommand{\SampleVar}[1]{\widehat{\mathrm{Var}}\left[ #1 \right]} \newcommand{\Cov}[1]{\mathrm{Cov}\left[ #1 \right]} \newcommand{\TrueRegFunc}{\mu} \newcommand{\EstRegFunc}{\widehat{\TrueRegFunc}} \DeclareMathOperator{\tr}{tr} \DeclareMathOperator*{\argmin}{argmin} \DeclareMathOperator{\dof}{DoF} \DeclareMathOperator{\det}{det} \newcommand{\TrueNoise}{\epsilon} \newcommand{\EstNoise}{\widehat{\TrueNoise}} \]

In our last episode…

Some properties of the PCs

Some properties of the eigenvalues

Some properties of PC scores

\[\begin{eqnarray} \Var{\text{scores}} & = & \frac{1}{n} \S^T \S\\ & = & \frac{1}{n} (\X\w)^T(\X\w)\\ & = & \frac{1}{n}\w^T \X^T \X \w\\ & = & \w^T \V\w = \w^T \mathbf{\Lambda} \w\\ & = & \mathbf{\Lambda} \w^T\w\\ & = & \mathbf{\Lambda} \end{eqnarray}\]

Some properties of PCA as a whole

Another way to think about PCA

PCA can be used for any multivariate data

PCA with spatial data

Recall the states…

state.pca <- prcomp(state.x77, scale. = TRUE)
signif(state.pca$rotation[, 1:2], 2)
##               PC1    PC2
## Population  0.130  0.410
## Income     -0.300  0.520
## Illiteracy  0.470  0.053
## Life Exp   -0.410 -0.082
## Murder      0.440  0.310
## HS Grad    -0.420  0.300
## Frost      -0.360 -0.150
## Area       -0.033  0.590

states are locations, PCs are patterns of variables

Each score is spatially distributed

Try it the other way

Turn the data on its side

state.vars.pca <- prcomp(t(scale(state.x77)))  # What's t()?
length(state.vars.pca$sdev)  # Why 8?
## [1] 8
head(signif(state.vars.pca$rotation[, 1:2]), 2)
##                PC1       PC2
## Alabama -0.2801370 0.0316183
## Alaska   0.0147876 0.5653260
signif(state.vars.pca$x[, 1], 2)
## Population     Income Illiteracy   Life Exp     Murder    HS Grad 
##      -2.60       2.90      -6.80       4.90      -6.70       4.80 
##      Frost       Area 
##       4.30      -0.69

The states turned on their sides…

Not exactly the same

… but pretty close. No coincidence! (See end)

2nd principal component

A famous example

Some maps from Cavalli-Sforza, Menozzi, and Piazza (1993)

World PC1

(\(\approx 35\%\) of between-population variance)

Some maps from Cavalli-Sforza, Menozzi, and Piazza (1993)

World PC2