Fourier Methods I

36-467/36-667

2 October 2018

\[ \newcommand{\Expect}[1]{\mathbb{E}\left[ #1 \right]} \newcommand{\Var}[1]{\mathrm{Var}\left[ #1 \right]} \newcommand{\SampleVar}[1]{\widehat{\mathrm{Var}}\left[ #1 \right]} \newcommand{\Cov}[1]{\mathrm{Cov}\left[ #1 \right]} \newcommand{\TrueRegFunc}{\mu} \newcommand{\EstRegFunc}{\widehat{\TrueRegFunc}} \DeclareMathOperator{\tr}{tr} \DeclareMathOperator*{\argmin}{argmin} \DeclareMathOperator{\det}{det} \newcommand{\TrueNoise}{\epsilon} \newcommand{\EstNoise}{\widehat{\TrueNoise}} \newcommand{\SignalNoise}{N} \]

In our previous episodes: Decomposing data into components

Today: Decomposing data into periodic components

Sine waves

We get data at times \(1, 2, \ldots n\)

curve(sin(2 * pi * x/n), xlim = c(0, n), ylim = c(-2, 2), type = "p", pch = 16, 
    cex = 0.5, col = "blue")

Sine waves

curve(sin(2 * pi * x/n), xlim = c(0, n), ylim = c(-2, 2), type = "p", pch = 16, 
    cex = 0.5, col = "blue")
curve(sin(2 * pi * 3 * x/n), type = "b", pch = 16, cex = 0.5, col = "yellow3", 
    add = TRUE)

Sine waves

curve(sin(2 * pi * x/n), xlim = c(0, n), ylim = c(-2, 2), type = "p", pch = 16, 
    cex = 0.5, col = "blue")
curve(sin(2 * pi * 3 * x/n), type = "p", pch = 16, cex = 0.5, col = "yellow3", 
    add = TRUE)
curve(sin(2 * pi * x/n) + sin(2 * pi * 3 * x/n), type = "p", pch = 16, cex = 0.5, 
    col = "green", add = TRUE)

An ambition

Some properties of sine waves

\[\begin{eqnarray} \sin{(2\pi \omega \frac{t}{n})} & \text{is} & \text{periodic, with period} ~ n/\omega ~ \text{and frequency} ~\omega/n\\ \sin{-\theta} & = & -\sin{\theta}\\ \sin{0} & = & 0\\ \sin{\pi/2} & = & 1\\ \sin{\theta+\pi} & = & -\sin{\theta} \end{eqnarray}\]

Consequences:

Some properties of cosine waves

\[\begin{eqnarray} \cos{(2\pi \omega \frac{t}{n})} & \text{is} & \text{periodic, with period} ~ n/\omega ~ \text{and frequency} ~\omega/n\\ \cos{-\theta} & = & \cos{\theta}\\ \cos{0} & = & 1\\ \cos{\pi/2} & = & 0\\ \cos{\theta+\pi} & = & -\cos{\theta} \end{eqnarray}\]

Consequences:

Some properties of combining sines and cosines

\[\begin{eqnarray} \sin{\theta+\phi} & = & \sin{\theta}\cos{\phi} + \sin{\phi}\cos{\theta}\\ \sin{\theta+\pi/2} & = & \cos{\theta}\\ \cos{\theta+\phi} & = & \cos{\theta}\cos{\phi} - \sin{\theta}\sin{\phi}\\ \cos{\theta-\pi/2} & = & \sin{\theta} \end{eqnarray}\]

\(\Rightarrow\) If we use \(\sin{(\phi_n + 2\pi \omega t/n)}\) functions, we don’t need cosines, and vice versa

Sines average out to zero

For any integer \(\omega\), \[\begin{eqnarray} \sum_{t=0}^{n-1}{\sin{2\pi\omega \frac{t}{n}}} & = & 0\\ \end{eqnarray}\] because \[ \sin{2\pi\omega \frac{t}{n}} = -\sin{2\pi\omega \frac{t+n/2\omega}{n}} \] i.e., time points come in pairs which cancel each other out in the sum

Sines of different frequencies are orthogonal

We can’t resolve either very high or very low frequency sine waves

Orthogonal vectors give us a basis

Re-writing the data in terms of sines and cosines

Re-writing the data in terms of sines

Use the trig identities to eliminate cosines in favor of sines:

\[ x(t) = \sum_{\omega=0}^{n/2-1}{\alpha_{\omega} \sin{\left(2\pi\frac{t}{n} + \phi_{\omega}\right)}} \] with \[ \alpha_{\omega} = \sqrt{a_{\omega}^2+b_{\omega}^2} \]

Re-writing the data in terms of complex exponentials

The Fourier transform

The inverse Fourier transform

An example

n <- 60
x <- sin(2 * pi * (1:60)/n) + 0.5 * sin(2 * pi * 3 * (1:60)/n)
plot(x, xlab = "t", ylab = expression(x(t)))

An example

An example

In R

fft(x)[h] = sum_{k=1}^n {x[k]*exp(-2*pi*1i*(k-1)*(h-1)/n)}

so entry h in the output of fft contains the Fourier coefficient for frequency (h-1)/n

In R

plot(x, pch = 16)

In R

plot(x, pch = 16)
points(Re(fft(fft(x), inverse = TRUE)/length(x)), col = "red", pch = 25)

Why do we care?

Convolutions

Convolutions and Fourier transforms

Autocovariance and Fourier transforms

Power spectrum

The Wiener-Khinchin theorem:

The periodogram

In R

spectrum(x)