\documentstyle[bj,txfonts]{article}

\frenchspacing
\def\vtrim{\vspace*{-0.8em}}

\begin{document}

\begin{center}
\Large
36-617: Applied Linear Models\\
Fall 2022\\
HW01 -- Due Weds Sept 07, 11:59pm
\end{center}

\begin{itemize}\parsep=0pt\itemsep=0pt\parskip=0pt

\item Please turn the homework in online in our course webspace at canvas.cmu.edu.
\begin{itemize}
\item There is a link to Gradescope in the description of this assignment on Canvas. 
\item You should submit a single pdf to Gradescope. If you need help with this, please see \\ https://www.cmu.edu/teaching/gradescope/index.html.  Also, allow yourself some extra time to create the pdf \& upload it in Gradescope.
\item Gradescope allows the TA to grade all the problem 1's together, then all the problem 2's, and so forth.  This leads to more consistent grading and better comments for you.
\end{itemize}

\item Reading:
\begin{itemize}

\item For this week: Sheather Ch 5 (supplemental: ISLR 3.1, 3.2; G\&H Ch 3)

\item For next week: ISLR 3.1, 3.2, 3.3.1, 3.3.2 (supplemental: G\&H Ch 3)

\end{itemize}

\item There are four exercises below.

~\\[-2em]

\end{itemize}


\section*{Exercises}

\begin{enumerate} % \itemsep=0pt\parsep=0pt\parskip=0pt



\item ISLR second Ed. p. 123 \#8.  

\begin{itemize}
\item Please be sure you are reading and using the {\em second edition} of the ISLR text.  I mistakenly put the first edition on canvas earlier, but now the second edition is there.
\item The {\tt Auto} data set is one of the data sets for the ISLR text.  The data sets for the ISLR book are listed briefly on pp. 13--14.  To get them, you must install the ISLR2 package from {\tt https://cran.r-project.org/}.  In Rstudio this is easiest to do by going to the {\bf Tools} menu, selecting the first option ({\bf Install packages...}) and following your nose.  You can also do this at the R command prompt, with a command like \verb|install.packages("ISLR2")|.  You only have to install a package once, but to use the package (i.e. to have access to the data sets, in this case) you must execute the command {\tt library(ISLR2)} at the beginning of your R or Rstudio session.
\end{itemize}
\item ISLR second Ed. p. 123--124, \#9.  

\begin{itemize}

\item Notes for part (e): The syntax and semantics for interactions in lm() and other modeling functions take a little getting used to.  To include the product of two variables, say $\mbox{\em weight}\times\mbox{\em horsepower}$, you would include the term {\tt weight:horsepower} on the right hand side of the regression model, so 
\begin{verbatim}
lm(mpg ~ weight:horsepower,data=Auto)
\end{verbatim}
would fit the model
\[
    \mbox{\em (mpg)} = \beta_0 + \beta_1\cdot\mbox{\em (weight)}\cdot\mbox{\em (horsepower)} + \epsilon
\]
On the other hand, the R notation {\tt weight*horsepower} includes the product {\em and all lower-order terms} (in this case, the main effects).  So for example either of the following two commands
\begin{verbatim}
lm(mpg ~ weight*horsepower,data=Auto)
lm(mpg ~ weight + horsepower + weight:horsepower,data=Auto)
\end{verbatim}
fit the same model
\[
   \mbox{\em (mpg)} = \beta_0 + \beta_1\cdot\mbox{\em (weight)} + \beta_2\cdot\mbox{\em (horsepower)} + \beta_3\cdot\mbox{\em (weight)}\cdot\mbox{\em (horsepower)} + \epsilon
\]
You should almost always prefer \verb|*| rather than \verb|:|.  Whenever you include an interaction in the model you should also include all lower-order terms from that interaction.

(Note also that the intercept is included by default in all models.  To get rid of the intercept, you have to add -1 to your model formula.)

\item Notes for part (f): You can apply a function like {\tt log(mpg)} or {\tt sqrt(horsepower)} directly to any variable in an R model statement.  Powers are a little trickier; because of the semantics of the modeling language in R (see notes for part (e) above), the command
\verb|lm(y ~ x + x^2)| produces the same model as \verb|lm(y ~ x)| (try it!).  
To get powers, it is safest to construct new variables, e.g. 
\begin{verbatim}
x2 <- x^2
lm(y ~ x + x2)
\end{verbatim}
You can accomplish the same thing with the {\tt I()} function: \verb|lm(y ~ x + I(x^2))|, though constructing a separate variable and putting it in the data frame with the other variable(s) is usually better for using the {\tt predict()} function, etc. 

\end{itemize}

\comment{

\item{} [Based on Gelman \& Hill. Ch 3, \#1, p. 49] The file {\tt pyth.dat}, in the same folder as this hw,  contains outcome {\tt y} and inputs {\tt x1}, {\tt x2} for 40 data points, with a further 20 points with the inputs but no observed outcome (for this problem we will ignore these last 20 points). Save the file to your working directory and read it into R using the {\tt read.table()} function.

\begin{enumerate}

\item Fit the two models
\begin{eqnarray*}  
	{\bf M1:~} {\tt y} & = & \beta_0 + \beta_1 {\tt x1} + \varepsilon \\
	{\bf M2:~} {\tt y} & = & \beta_0 + \beta_1 {\tt x2} + \varepsilon 
\end{eqnarray*}
Which model provides a better fit for {\tt y}?  Why?

\item Construct new variables {\tt y2 = y$^2$}, {\tt x12 = x1$^2$}, and {\tt x22 = x2$^2$} and fit the models
\begin{eqnarray*}  
	{\bf M3:~} {\tt y2} & = & \beta_0 + \beta_1 {\tt x12} + \varepsilon \\
	{\bf M4:~} {\tt y2} & = & \beta_0 + \beta_1 {\tt x22} + \varepsilon 
\end{eqnarray*}
Compare the fits of these two models to the models in part (a).  Which fits best?  Why?

\item To fit the model 
\[
	{\tt y2} ~ = ~ \beta_0 + \beta_1 {\tt x1} + \beta_2 {\tt x2} + \varepsilon ~,
\]
we just expand the R modeling language a little bit: \verb|y ~ x1 + x2|.  Fit both of the models
\begin{eqnarray*}
	{\bf M5:~} {\tt ~y} & = & \beta_0 + \beta_1 {\tt x1} + \beta_2 {\tt x2} + \varepsilon \\
	{\bf M6:~} {\tt y2} & = & \beta_0 + \beta_1 {\tt x12}  + \beta_2 {\tt x22} + \varepsilon 
\end{eqnarray*}
Compare these to the earlier models.  Which fits best?  Why?

\item Can you find a simple, recognizable function {\tt x3 = (something involving both x1 and x2)}, so that 
\[
		{\bf M7:~} {\tt y} ~ = ~ \beta_0 + \beta_1 {\tt x3} + \varepsilon
\]
provides a fit comparable to the best fitting models above?  What is going on?

\end{enumerate}

}

\item This exercise introduces some base-R notation for working with matrices, and also illustrates that the matrix formulation that we talked about in class really does produce the usual least-squares estimates for regression coefficients, standard errors, and so forth.  Here are some of the R commands and notation that you will use:
\begin{itemize}
\item {\tt cbind(x,y,z, ...)} creates a matrix with columns x,y,z, etc.  If one of the varaibles is shorter than the others, its elements are repeated in order until it is the right length.  (cbind can also do other things, like join two matrices, two data frames, etc.)
\item {\tt dim(X)} returns a vector with the dimensions of the matrix (or data frame) X: {\tt dim(X)[1]} is the number of rows, {\tt dim(X)[2]} is the number of columns.
\item \verb|%*%| does matrix multiplication.  (\verb|*| does scalar and elementwise multiplication)
\item For any matrix X, {\tt t(X)} is the transpose of X.
\item For any square matrix X, {\tt diag(X)} produces the diagonal elements of X.
\item If X is an invertible matrix, \verb|solve(X)| produces X$^{-1}$, the inverse matrix.
\item \verb|round(X,d)| rounds all the numbers in X to d decimal places. 
\item If X is a data frame with named columns, the \verb|$| operator selects columns by name, for example:
\begin{verbatim}
> X <- as.data.frame(matrix(rnorm(9),ncol=3))
> names(X) <- c("fred","bobby","sue")
> X
       fred      bobby        sue
1  0.977772 -0.5336657 -0.2353455
2  0.522910  0.4758267  1.5450506
3 -2.988834  0.7029876  0.4422550
> X$fred
[1]  0.977772  0.522910 -2.988834
> X$bobby
[1] -0.5336657  0.4758267  0.7029876
> X$sue
[1] -0.2353455  1.5450506  0.4422550
\end{verbatim}
\end{itemize}
Now, on to the exercise!
\begin{enumerate}
\item The {\tt kidiq.csv} file is in the same directory as this assigment.   Read it into a data frame in R with a command like \verb|kidiq <- read.csv("kidiq.csv",header=TRUE)|.  Use the {\tt cbind()} command to create:
\begin{itemize}
\item {\tt y} $=$ a column vector (matrix with one column), from the column {\tt kid.score} in {\tt kidiq}.
\item {\tt X} $=$ a matrix with (a) the first column containing all 1's; (b) the second column containing the column {\tt mom.hs} from {\tt kidiq}; and (c) the third colummn containing the column {\tt mom.iq} from {\tt kidiq}.
\end{itemize}
Use {\tt dim()} to verify that {\tt y} is 434$\times$1, and {\tt X} is 434$\times$3.  Use the {\tt head()} commansd to print out the first few rows of {\tt y} and {\tt X}, and turn the results of the {\tt dim()} and {\tt head()} commands in.

\item Compute $V = (X^TX)^{-1}$ in R, and show the result (V should be a $3\times3$ matrix; why?).

\item Compute $\hat\beta = (X^TX)^{-1}X^Ty$ in R, and show the result.\label{beta-hats}

\item Calculate the residual vector $y- X\hat\beta$ in R, and use the result to compute the residual variance $s^2$ (don't forget to divide by $n-k$; what are $n$ and $k$ here?).  Show the resulting $s^2$.

\item Calculate the matrix $\Var(\hat\beta) = (X^TX)^{-1} s^2$ in R, and extract the square roots of the diagonal elements of this matrix.  These are the standard errors $SE(\hat\beta_0)$, $SE(\hat\beta_1)$, and $SE(\hat\beta_2)$. \label{SEs}

\item Compare your values for $\hat\beta_0$, $\hat\beta_1$, $\hat\beta_2$ (from part \ref{beta-hats}), and their standard errors (from part \ref{SEs}) with the result of running 
\begin{verbatim}
summary(lm(kid.score ~ mom.hs + mom.iq, data=kidiq))
\end{verbatim}
Comment on any similarities or differences.  
\end{enumerate}

\item In the folder for this hw assignment you will find a pdf called ``An IMRAD paper on wine'', based on Example 1.2.4 in Sheather.  This paper is based only on EDA, not on any more sophisticated methods.
\begin{enumerate}

\item Read the slides ``IMRAD: What goes into each section'', a pdf of which is in the same folder as this assignment.  (There is nothing to turn in for this part).

\item Does the paper appropriately address each of the parts of an IMRAD paper as described the ``IMRAD: What goes into each section'' pdf?\\[-1em]  

For each section below, either say ``yes this section has the right content'', or say ``no'' and describe what is missing and/or what needs to be moved to another section of the paper or deleted.
\begin{itemize}
\item Abstract
\item Introduction
\item Methods
\item Results
\item Discussion
\end{itemize}


\end{enumerate}
Later we will see (and write) more complex versions of IMRAD and IDMRAD papers; this is just a first taste!

\end{enumerate}

\end{document}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
