% Options for packages loaded elsewhere
\PassOptionsToPackage{unicode}{hyperref}
\PassOptionsToPackage{hyphens}{url}
%
\documentclass[
]{article}
\usepackage{amsmath,amssymb}
\usepackage{lmodern}
\usepackage{iftex}
\ifPDFTeX
  \usepackage[T1]{fontenc}
  \usepackage[utf8]{inputenc}
  \usepackage{textcomp} % provide euro and other symbols
\else % if luatex or xetex
  \usepackage{unicode-math}
  \defaultfontfeatures{Scale=MatchLowercase}
  \defaultfontfeatures[\rmfamily]{Ligatures=TeX,Scale=1}
\fi
% Use upquote if available, for straight quotes in verbatim environments
\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
\IfFileExists{microtype.sty}{% use microtype if available
  \usepackage[]{microtype}
  \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
}{}
\makeatletter
\@ifundefined{KOMAClassName}{% if non-KOMA class
  \IfFileExists{parskip.sty}{%
    \usepackage{parskip}
  }{% else
    \setlength{\parindent}{0pt}
    \setlength{\parskip}{6pt plus 2pt minus 1pt}}
}{% if KOMA class
  \KOMAoptions{parskip=half}}
\makeatother
\usepackage{xcolor}
\usepackage[margin=1in]{geometry}
\usepackage{color}
\usepackage{fancyvrb}
\newcommand{\VerbBar}{|}
\newcommand{\VERB}{\Verb[commandchars=\\\{\}]}
\DefineVerbatimEnvironment{Highlighting}{Verbatim}{commandchars=\\\{\}}
% Add ',fontsize=\small' for more characters per line
\usepackage{framed}
\definecolor{shadecolor}{RGB}{248,248,248}
\newenvironment{Shaded}{\begin{snugshade}}{\end{snugshade}}
\newcommand{\AlertTok}[1]{\textcolor[rgb]{0.94,0.16,0.16}{#1}}
\newcommand{\AnnotationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\AttributeTok}[1]{\textcolor[rgb]{0.77,0.63,0.00}{#1}}
\newcommand{\BaseNTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}}
\newcommand{\BuiltInTok}[1]{#1}
\newcommand{\CharTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\CommentTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textit{#1}}}
\newcommand{\CommentVarTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\ConstantTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\ControlFlowTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{\textbf{#1}}}
\newcommand{\DataTypeTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{#1}}
\newcommand{\DecValTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}}
\newcommand{\DocumentationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\ErrorTok}[1]{\textcolor[rgb]{0.64,0.00,0.00}{\textbf{#1}}}
\newcommand{\ExtensionTok}[1]{#1}
\newcommand{\FloatTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}}
\newcommand{\FunctionTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\ImportTok}[1]{#1}
\newcommand{\InformationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\KeywordTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{\textbf{#1}}}
\newcommand{\NormalTok}[1]{#1}
\newcommand{\OperatorTok}[1]{\textcolor[rgb]{0.81,0.36,0.00}{\textbf{#1}}}
\newcommand{\OtherTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{#1}}
\newcommand{\PreprocessorTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textit{#1}}}
\newcommand{\RegionMarkerTok}[1]{#1}
\newcommand{\SpecialCharTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\SpecialStringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\StringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\VariableTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\VerbatimStringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\WarningTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\usepackage{graphicx}
\makeatletter
\def\maxwidth{\ifdim\Gin@nat@width>\linewidth\linewidth\else\Gin@nat@width\fi}
\def\maxheight{\ifdim\Gin@nat@height>\textheight\textheight\else\Gin@nat@height\fi}
\makeatother
% Scale images if necessary, so that they will not overflow the page
% margins by default, and it is still possible to overwrite the defaults
% using explicit options in \includegraphics[width, height, ...]{}
\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio}
% Set default figure placement to htbp
\makeatletter
\def\fps@figure{htbp}
\makeatother
\setlength{\emergencystretch}{3em} % prevent overfull lines
\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
\setcounter{secnumdepth}{-\maxdimen} % remove section numbering
\ifLuaTeX
  \usepackage{selnolig}  % disable illegal ligatures
\fi
\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}
\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available
\urlstyle{same} % disable monospaced font for URLs
\hypersetup{
  pdftitle={Homework 04 Solutions},
  hidelinks,
  pdfcreator={LaTeX via pandoc}}

\title{Homework 04 Solutions}
\author{}
\date{\vspace{-2.5em}2022-09-22}

\begin{document}
\maketitle

\def\red#1{\textcolor{red}{#1}}
\def\blue#1{\textcolor{blue}{#1}}
\let\bar\overline
\let\ul\underline

\frenchspacing

\begin{center}
\Large
36-617: Applied Linear Models\\
Fall 2022\\
Solutions
\end{center}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{library}\NormalTok{(arm)  }\DocumentationTok{\#\# includes lme4, MASS, Matrix}
\FunctionTok{library}\NormalTok{(ggplot2); }\FunctionTok{theme\_set}\NormalTok{(}\FunctionTok{theme\_bw}\NormalTok{())}
\FunctionTok{library}\NormalTok{(gridExtra) }\DocumentationTok{\#\# to arrange ggplots...}
\FunctionTok{library}\NormalTok{(GGally) }\DocumentationTok{\#\# for ggpairs...}

\FunctionTok{library}\NormalTok{(leaps) }\DocumentationTok{\#\# regsubsets(), summary(), coef()}
\FunctionTok{library}\NormalTok{(car)   }\DocumentationTok{\#\# subsets(), mmps(), vif(), etc.}
\end{Highlighting}
\end{Shaded}

\hypertarget{problem-1-sheather-ch-6-pp.-216221-3.}{%
\section{Problem 1: Sheather, Ch 6, pp.~216--221,
\#3.}\label{problem-1-sheather-ch-6-pp.-216221-3.}}

Continuing the analysis of data in ``cars04.csv''\ldots{} Notes:

\begin{itemize}
\item The data set is available as ``cars04.csv''.
\item There are some errors in the results for fitting the model (6.37) in the textbook, so I suggest you refit the models for this problem directly from the data in ``cars04.csv''.
\end{itemize}

The variables are:

\begin{description}
\item[$Y =$] Suggested Retail Price; 
\item[$x_1 =$] Engine size; 
\item[$x_2 =$] Cylinders;
\item[$x_3 =$] Horse power; 
\item[$x_4 =$] Highway mpg; 
\item[$x 5 =$] Weight;
\item[$x_6 =$] Wheel Base;
\item[$x_7 =$] Hybrid (1=Hybrid; 0=just gasoline-powered)
\end{description}

The first model proposed is \[
Y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \beta_3x_3 + \beta_4x_4 + \beta_5x_5 + \beta_6x_6 + \beta_7x_7 + \epsilon
\eqno{(6.36)}
\] After examining Box-Cox transformation proposals and ``rounding'' to
more interpretable values, the second model proposed is \[
\log(Y) = \beta_0 + \beta_1x_1^{0.25} + \beta_2
\log(x_2) + \beta_3\log(x_3) + \beta_4(1/x_4) + \beta_5x_5 + \beta_6\log(x_6) + \beta_7x_7 + \epsilon
\eqno{(6.37)}
\] \em

Since there are some problems with the analysis presented in Sheather,
we refit all the models and reproduce all the outputs (except that I am
just going to accept the transformations suggested in (6.37)).

\rm

\hypertarget{problem-1a}{%
\subsection{Problem 1(a)}\label{problem-1a}}

Decide whether (6.36) is a valid model. Give reasons to support your
answer.

\em

We start by refitting the model and reproducing the diagnostic plots in
Sheather.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{cars }\OtherTok{\textless{}{-}} \FunctionTok{read.csv}\NormalTok{(}\StringTok{"cars04.csv"}\NormalTok{)}
\FunctionTok{str}\NormalTok{(cars)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 'data.frame':    234 obs. of  13 variables:
##  $ Vehicle.Name        : chr  "Chevrolet Aveo 4dr" "Chevrolet Aveo LS 4dr hatch" "Chevrolet Cavalier 2dr" "Chevrolet Cavalier 4dr" ...
##  $ Hybrid              : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ SuggestedRetailPrice: int  11690 12585 14610 14810 16385 13670 15040 13270 13730 15460 ...
##  $ DealerCost          : int  10965 11802 13697 13884 15357 12849 14086 12482 12906 14496 ...
##  $ EngineSize          : num  1.6 1.6 2.2 2.2 2.2 2 2 2 2 2 ...
##  $ Cylinders           : int  4 4 4 4 4 4 4 4 4 4 ...
##  $ Horsepower          : int  103 103 140 140 140 132 132 130 110 130 ...
##  $ CityMPG             : int  28 28 26 26 26 29 29 26 27 26 ...
##  $ HighwayMPG          : int  34 34 37 37 37 36 36 33 36 33 ...
##  $ Weight              : int  2370 2348 2617 2676 2617 2581 2626 2612 2606 2606 ...
##  $ WheelBase           : int  98 98 104 104 104 105 105 103 103 103 ...
##  $ Length              : int  167 153 183 183 183 174 174 168 168 168 ...
##  $ Width               : int  66 66 69 68 69 67 67 67 67 67 ...
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{names}\NormalTok{(cars)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##  [1] "Vehicle.Name"         "Hybrid"               "SuggestedRetailPrice"
##  [4] "DealerCost"           "EngineSize"           "Cylinders"           
##  [7] "Horsepower"           "CityMPG"              "HighwayMPG"          
## [10] "Weight"               "WheelBase"            "Length"              
## [13] "Width"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{cars.red }\OtherTok{\textless{}{-}}\NormalTok{ cars[,}\FunctionTok{c}\NormalTok{(}\StringTok{"EngineSize"}\NormalTok{,}\StringTok{"Cylinders"}\NormalTok{,}
\StringTok{"Horsepower"}\NormalTok{,}\StringTok{"HighwayMPG"}\NormalTok{,}\StringTok{"Weight"}\NormalTok{,}\StringTok{"WheelBase"}\NormalTok{,}\StringTok{"Hybrid"}\NormalTok{,}\StringTok{"SuggestedRetailPrice"}\NormalTok{)]}
\FunctionTok{dim}\NormalTok{(cars.red)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 234   8
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{names}\NormalTok{(cars.red)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] "EngineSize"           "Cylinders"            "Horsepower"          
## [4] "HighwayMPG"           "Weight"               "WheelBase"           
## [7] "Hybrid"               "SuggestedRetailPrice"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{ggpairs}\NormalTok{(cars.red)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-2-1} \end{center}

\begin{Shaded}
\begin{Highlighting}[]
  \DocumentationTok{\#\# Sheather did not include the response variable "SuggestedRetailPrice",}
  \DocumentationTok{\#\# but I think it is good to include since it can help us guess transformations...}
\FunctionTok{summary}\NormalTok{(lm.}\FloatTok{6.36} \OtherTok{\textless{}{-}} \FunctionTok{lm}\NormalTok{(SuggestedRetailPrice }\SpecialCharTok{\textasciitilde{}}\NormalTok{ ., }\AttributeTok{data=}\NormalTok{cars.red))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## Call:
## lm(formula = SuggestedRetailPrice ~ ., data = cars.red)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -17436  -4134    173   3561  46392 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept) -68965.793  16180.381  -4.262 2.97e-05 ***
## EngineSize   -6957.457   1600.137  -4.348 2.08e-05 ***
## Cylinders     3564.755    969.633   3.676 0.000296 ***
## Horsepower     179.702     16.411  10.950  < 2e-16 ***
## HighwayMPG     637.939    202.724   3.147 0.001873 ** 
## Weight          11.911      2.658   4.481 1.18e-05 ***
## WheelBase       47.607    178.070   0.267 0.789444    
## Hybrid         431.759   6092.087   0.071 0.943562    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 7533 on 226 degrees of freedom
## Multiple R-squared:  0.7819, Adjusted R-squared:  0.7751 
## F-statistic: 115.7 on 7 and 226 DF,  p-value: < 2.2e-16
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{par}\NormalTok{(}\AttributeTok{mfrow=}\FunctionTok{c}\NormalTok{(}\DecValTok{2}\NormalTok{,}\DecValTok{2}\NormalTok{))}
\FunctionTok{plot}\NormalTok{(lm.}\FloatTok{6.36}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-2-2} \end{center}

Neither the scatter plot matrix nor the \verb|summary()| printout can
tell us whether the model is valid (the scatter plot matrix can help us
make guesses about transformations and/or collinearity. The
\verb|summary| output can tell us whether the fitted model will be
useful for answering questions our collaborator may have). Validity of
the model depends only on

\begin{itemize}
\item Linear relationship between $Y$ and the $X$'s
\item Normal errors $\epsilon$
\item Constant error variance $\sigma^2=Var(\epsilon)$
\end{itemize}

We can use the casewise diagnostic plots for this:

\begin{description}
\item[Linear relationship between $Y$ and the $X$'s:] The ``Residuals vs. Fitted'' plot suggests a curved relationship between the residuals $\hat e$ and the fitted values $\hat y$.  This suggests that the relationship between $Y$ and the $X$'s is \underline{nonlinear}. 
\item[Normal errors $\epsilon$:] The ``Normal Q-Q'' plot of the standardized residuals $r_i$ shows quite a bit of right-skew.  This suggests that the errors $\epsilon$ are \underline{not normally distributed}.
\item[Constant error variance $\sigma=\sqrt{Var(\epsilon)}$:] The red trend line in the ``Scale-Location'' is increasing from left to right, suggesting that the variance $Var(\epsilon)$ is not constant across the data.
\end{description}

Since there is evidence of violations of all three basic assumptions of
the model, we can say \underline{the model is not fully valid}.

\textbf{Note:} The model might still be useful in some way, e.g.~for
prediction. However it does not satisfy the basic statistical
assumptions above, and so any tests or estimates for the \(\beta\)'s,
for the \(\hat y\)'s for \(\sigma\) should be viewed with some
suspicion.

\rm

\hypertarget{problem-1b}{%
\subsection{Problem 1(b)}\label{problem-1b}}

The plot of residuals against fitted values produces a curved pattern.
Describe what, if anything can be learned about model (6.36) from this
plot.

\em

The curved pattern in the residuals suggests we should look for
transofmrations of \(X\)'s or \(Y\)'s to produce a set of raw residuals
\(\hat e\) with less functional dependence on the fitted values
\(\hat y\).

\rm

\hypertarget{problem-1c}{%
\subsection{Problem 1(c)}\label{problem-1c}}

Identify any bad leverage points for model (6.36).

\em

From the ``Residuals vs Leverage'' plot we can see:

\begin{itemize} 
\item Observation \#223 seems to be influential, with a Cook's Distance $>$ 0.5.
\item Observation \#67 may also be influention, with a Cook's Distance near 0.5.
\item Observation \#222 does not have high leverage, but its residual is a hugh outlier---more than 6 SD's from the middle of the residual distribution.
\item Two more observations, not labelled by R, have relatively high leverage ($\approx$ 0.35) but pretty small residuals.
\end{itemize}

\textbf{Note:} The usual cutoff for ``high leverage'' is \[
        h_{high} = 2\cdot\frac{p+1}{n} = 2\cdot\frac{8}{234} \approx 0.07
\] so there are actually 5 or more points with leverage
\(h_{ii}>h_{high}\), but the points I've identified above are the ones
that I might devote some worry to in practice.

\rm

\hypertarget{problem-1d}{%
\subsection{Problem 1(d)}\label{problem-1d}}

Decide whether (6.37) is a valid model.

\em

First we refit the model and reproduce the output in Sheather (this is
the part that Sheather appaers to have gotten wrong):

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{attach}\NormalTok{(cars.red) }\DocumentationTok{\#\# so I don\textquotesingle{}t have to type "cars.red$" all the time...}
\NormalTok{tSuggestedRetailPrice }\OtherTok{\textless{}{-}} \FunctionTok{log}\NormalTok{(SuggestedRetailPrice)}
\NormalTok{tEngineSize }\OtherTok{\textless{}{-}}\NormalTok{ EngineSize}\SpecialCharTok{\^{}}\NormalTok{(}\FloatTok{0.25}\NormalTok{)}
\NormalTok{tCylinders }\OtherTok{\textless{}{-}} \FunctionTok{log}\NormalTok{(Cylinders)}
\NormalTok{tHorsepower }\OtherTok{\textless{}{-}} \FunctionTok{log}\NormalTok{(Horsepower)}
\NormalTok{tHighwayMPG }\OtherTok{\textless{}{-}} \DecValTok{1}\SpecialCharTok{/}\NormalTok{HighwayMPG}
\NormalTok{tWheelBase }\OtherTok{\textless{}{-}} \FunctionTok{log}\NormalTok{(WheelBase)}
\NormalTok{cars.t }\OtherTok{\textless{}{-}} \FunctionTok{data.frame}\NormalTok{(tEngineSize,}
\NormalTok{                       tCylinders, tHorsepower, tHighwayMPG,}
\NormalTok{                       Weight, tWheelBase, Hybrid,tSuggestedRetailPrice)}
\FunctionTok{detach}\NormalTok{()}
\FunctionTok{names}\NormalTok{(cars.t)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] "tEngineSize"           "tCylinders"            "tHorsepower"          
## [4] "tHighwayMPG"           "Weight"                "tWheelBase"           
## [7] "Hybrid"                "tSuggestedRetailPrice"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{ggpairs}\NormalTok{(cars.t)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-3-1} \end{center}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lm.}\FloatTok{6.37} \OtherTok{\textless{}{-}} \FunctionTok{lm}\NormalTok{(tSuggestedRetailPrice }\SpecialCharTok{\textasciitilde{}}\NormalTok{ ., }\AttributeTok{data=}\NormalTok{cars.t))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## Call:
## lm(formula = tSuggestedRetailPrice ~ ., data = cars.t)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.42288 -0.10983 -0.00203  0.10279  0.70068 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  5.703e+00  2.010e+00   2.838  0.00496 ** 
## tEngineSize -1.575e+00  3.332e-01  -4.727 4.01e-06 ***
## tCylinders   2.335e-01  1.204e-01   1.940  0.05359 .  
## tHorsepower  8.992e-01  8.876e-02  10.130  < 2e-16 ***
## tHighwayMPG  8.029e-01  4.758e+00   0.169  0.86614    
## Weight       5.043e-04  6.367e-05   7.920 1.07e-13 ***
## tWheelBase  -6.385e-02  4.715e-01  -0.135  0.89240    
## Hybrid       6.422e-01  1.150e-01   5.582 6.78e-08 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.1789 on 226 degrees of freedom
## Multiple R-squared:  0.8621, Adjusted R-squared:  0.8578 
## F-statistic: 201.8 on 7 and 226 DF,  p-value: < 2.2e-16
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{par}\NormalTok{(}\AttributeTok{mfrow=}\FunctionTok{c}\NormalTok{(}\DecValTok{2}\NormalTok{,}\DecValTok{2}\NormalTok{))}
\FunctionTok{plot}\NormalTok{(lm.}\FloatTok{6.37}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-3-2} \end{center}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{mmps}\NormalTok{(lm.}\FloatTok{6.37}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-3-3} \end{center}

Now, following the same approach as before, we use the casewise
diagnostic plots to check: We can use the casewise diagnostic plots for
this:

\begin{description}
\item[Linear relationship between $Y$ and the $X$'s:]  The red trend-line in the ``Residuals vs Fitted'' plot is nearly flat, suggesting no relationship between $\hat e$ and $\hat y$.  This suggests that \underline{we have an approximately linear} \underline{relationship} between the new $Y$ and the new $X$'s in the transformed model.
\item[Normal errors $\epsilon$:] Except for two outliers (obs. \#222 and \#67) the ``Normal Q-Q'' plot confirms that the standardized residuals $r_i$ are approximately normally distributed, suggesting that the errors $\epsilon$ are \underline{normally distributed}.
\item[Constant error variance $\sigma=\sqrt{Var(\epsilon)}$:] The red trend line in the ``Scale-Location'' plot is nearly flat, suggesting that the error variance $Var(\epsilon)$ is constant across the data set. 
\end{description}

Of course no fit will be perfectly valid, but the evidence from the
casewise diagnosic plots suggests that (6.37) is a much more valid model
than (6.36).

\textbf{Note:} We can also see, from the scatterplot matrix and from the
marginal model plots, that we seem to have done a good job with
transforms. Especially, in the marginal model plots, the estimates of
\(E[Y|X]\) from the nonparametric regressions (blue) and the linear
model (red) line up well, so it doesn't appear that any further
transformations are needed.

\rm

\hypertarget{problem-1e}{%
\subsection{Problem 1(e)}\label{problem-1e}}

To obtain a final model, the analyst wants to simply remove the two
insignificant predictors \(1/x_4\) (i.e., \texttt{tHighwayMPG}) and
\(\log(x_6)\) (i.e., \texttt{tWheelBase}) from (6.37). Perform a partial
\(F\)-test to see if this is a sensible strategy.

\em

We will fit the smaller model (without the two variables that the
analyst wants to remove) and then compare that with \verb|lm.6.37|:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lm.smaller }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lm.}\FloatTok{6.37}\NormalTok{, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ tHighwayMPG }\SpecialCharTok{{-}}\NormalTok{ tWheelBase)}
\FunctionTok{anova}\NormalTok{(lm.smaller, lm.}\FloatTok{6.37}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Analysis of Variance Table
## 
## Model 1: tSuggestedRetailPrice ~ tEngineSize + tCylinders + tHorsepower + 
##     Weight + Hybrid
## Model 2: tSuggestedRetailPrice ~ tEngineSize + tCylinders + tHorsepower + 
##     tHighwayMPG + Weight + tWheelBase + Hybrid
##   Res.Df    RSS Df Sum of Sq     F Pr(>F)
## 1    228 7.2358                          
## 2    226 7.2337  2 0.0021769 0.034 0.9666
\end{verbatim}

With a \(p\)-value of 0.97, there reall is no evidence in favor of
keeping these two variables in the model, and so \verb|lm.smaller| seems
like a sufficient model:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lm.smaller)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## Call:
## lm(formula = tSuggestedRetailPrice ~ tEngineSize + tCylinders + 
##     tHorsepower + Weight + Hybrid, data = cars.t)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.42224 -0.11001 -0.00099  0.10191  0.70205 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  5.422e+00  3.291e-01  16.474  < 2e-16 ***
## tEngineSize -1.591e+00  3.157e-01  -5.041 9.45e-07 ***
## tCylinders   2.375e-01  1.186e-01   2.003   0.0463 *  
## tHorsepower  9.049e-01  8.305e-02  10.896  < 2e-16 ***
## Weight       5.029e-04  5.203e-05   9.666  < 2e-16 ***
## Hybrid       6.340e-01  1.080e-01   5.870 1.53e-08 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.1781 on 228 degrees of freedom
## Multiple R-squared:  0.862,  Adjusted R-squared:  0.859 
## F-statistic: 284.9 on 5 and 228 DF,  p-value: < 2.2e-16
\end{verbatim}

We probably should go ahead and check casewise diagnostic plots,
marginal model plots, etc., to be sure that we still have a valid model
and no more transformations are needed, but I will stop here.

\rm

\hypertarget{problem-1f}{%
\subsection{Problem 1(f)}\label{problem-1f}}

The analyst's boss has complained about model (6.37) saying that it
fails to take account of the manufacturer of the vehicle (e.g., BMW vs
Toyota). Describe how model (6.37) could be expanded in order to
estimate the effect of manufacturer on suggested retail price.

\em

This is slightly tricky to do well. The basic idea (and, really, all you
have to say to answer the question) is that you want to extract the
manufacturer name from the variable \verb|Vehicle.Name| in the orignal
data set \verb|cars|, and add that to the regressions above, and see
what happens\ldots{}

Doing the extraction is a little tricky but here's a way to proceed, if
you are curious:

The \verb|Vehicle.Name| variable in the original data set is nearly
unique for each car in the data set:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{dim}\NormalTok{(cars)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 234  13
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{length}\NormalTok{(}\FunctionTok{unique}\NormalTok{(cars}\SpecialCharTok{$}\NormalTok{Vehicle.Name))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 232
\end{verbatim}

(There are two cars duplicated in the data set,

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{tmp }\OtherTok{\textless{}{-}} \FunctionTok{table}\NormalTok{(cars}\SpecialCharTok{$}\NormalTok{Vehicle.Name)}
\NormalTok{tmp[tmp}\SpecialCharTok{\textgreater{}}\DecValTok{1}\NormalTok{]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##       Infiniti G35 4dr Mercedes-Benz C240 4dr 
##                      2                      2
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{cars[}\FunctionTok{grep}\NormalTok{(}\FunctionTok{names}\NormalTok{(tmp[tmp}\SpecialCharTok{\textgreater{}}\DecValTok{1}\NormalTok{])[}\DecValTok{1}\NormalTok{],cars}\SpecialCharTok{$}\NormalTok{Vehicle.Name),]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##         Vehicle.Name Hybrid SuggestedRetailPrice DealerCost EngineSize
## 116 Infiniti G35 4dr      0                28495      26157        3.5
## 161 Infiniti G35 4dr      0                32445      29783        3.5
##     Cylinders Horsepower CityMPG HighwayMPG Weight WheelBase Length Width
## 116         6        260      18         26   3336       112    187    69
## 161         6        260      18         26   3677       112    187    69
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{cars[}\FunctionTok{grep}\NormalTok{(}\FunctionTok{names}\NormalTok{(tmp[tmp}\SpecialCharTok{\textgreater{}}\DecValTok{1}\NormalTok{])[}\DecValTok{2}\NormalTok{],cars}\SpecialCharTok{$}\NormalTok{Vehicle.Name),]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##               Vehicle.Name Hybrid SuggestedRetailPrice DealerCost EngineSize
## 169 Mercedes-Benz C240 4dr      0                32280      30071        2.6
## 170 Mercedes-Benz C240 4dr      0                33480      31187        2.6
##     Cylinders Horsepower CityMPG HighwayMPG Weight WheelBase Length Width
## 169         6        168      20         25   3360       107    178    68
## 170         6        168      19         25   3360       107    178    68
\end{verbatim}

but this doesn't change the problem.)

With 232 unique vehicle names for 234 observation in the data set, if we
just include vehicle name in a regression, it will soak up all of the
variability explained by the other variables, and we won't get any
useful information out.

We note that the make (i.e.~the manufacturer name) of the car always
comes first in the vehicle name, followed by a space. So we can make a
new variable, \verb|Make|, that just has the manufacturers' names:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{splits }\OtherTok{\textless{}{-}} \FunctionTok{strsplit}\NormalTok{(cars}\SpecialCharTok{$}\NormalTok{Vehicle.Name,}\StringTok{" "}\NormalTok{)}
\FunctionTok{head}\NormalTok{(splits,}\DecValTok{10}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [[1]]
## [1] "Chevrolet" "Aveo"      "4dr"      
## 
## [[2]]
## [1] "Chevrolet" "Aveo"      "LS"        "4dr"       "hatch"    
## 
## [[3]]
## [1] "Chevrolet" "Cavalier"  "2dr"      
## 
## [[4]]
## [1] "Chevrolet" "Cavalier"  "4dr"      
## 
## [[5]]
## [1] "Chevrolet" "Cavalier"  "LS"        "2dr"      
## 
## [[6]]
## [1] "Dodge" "Neon"  "SE"    "4dr"  
## 
## [[7]]
## [1] "Dodge" "Neon"  "SXT"   "4dr"  
## 
## [[8]]
## [1] "Ford"  "Focus" "ZX3"   "2dr"   "hatch"
## 
## [[9]]
## [1] "Ford"  "Focus" "LX"    "4dr"  
## 
## [[10]]
## [1] "Ford"  "Focus" "SE"    "4dr"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{mfr.name }\OtherTok{\textless{}{-}} \FunctionTok{sapply}\NormalTok{(splits, }\ControlFlowTok{function}\NormalTok{(x) x[}\DecValTok{1}\NormalTok{])}
\FunctionTok{head}\NormalTok{(mfr.name,}\DecValTok{10}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##  [1] "Chevrolet" "Chevrolet" "Chevrolet" "Chevrolet" "Chevrolet" "Dodge"    
##  [7] "Dodge"     "Ford"      "Ford"      "Ford"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{cars}\SpecialCharTok{$}\NormalTok{Make }\OtherTok{\textless{}{-}}\NormalTok{ mfr.name}
\FunctionTok{length}\NormalTok{(}\FunctionTok{unique}\NormalTok{(cars}\SpecialCharTok{$}\NormalTok{Make))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 33
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(cars}\SpecialCharTok{$}\NormalTok{Make)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##         Acura          Audi           BMW         Buick      Cadillac 
##             5            13            13             7             4 
##     Chevrolet      Chrvsler      Chrysler         Dodge          Ford 
##            13             1            10             6            10 
##         Honda       Hyundai      Infiniti        Jaguar           Kia 
##            11            10             6             8             7 
##         Lexus       Lincoln        Mazda6 Mercedes-Benz       Mercury 
##             6             7             1            15             6 
##          Mini    Mitsubishi        Nissan    Oldsmobile       Pontiac 
##             2             2             7             2             5 
##          Saab        Saturn         Scion        Subaru        Suzuki 
##             6             6             1             6             5 
##        Toyota    Volkswagen         Volvo 
##            15             9             9
\end{verbatim}

Now we could introduce the \verb|Make| variable into any of the
regressions we fitted above, to see if the car maker affect prediction
of the \verb|SuggestedRetailPrice|, separately from the other features
of the car.

\rm

\hypertarget{problem-2-sheather-ch-7-p.-261-3.}{%
\section{Problem 2: Sheather, Ch 7, p.~261,
\#3.}\label{problem-2-sheather-ch-7-p.-261-3.}}

Continuing with analysis of ``pgatour2006.csv''\ldots{} Note:

\begin{itemize}
\item The data is available in ``pgatour2006.csv''.
\end{itemize}

The variables are:

\begin{description}
\item[$Y =$] PrizeMoney; 
\item[$x_1 =$] DrivingAccuracy; 
\item[$x_2 =$] GIR;
\item[$x_3 =$] PuttingAverage; 
\item[$x_4 =$] BirdieConversion; 
\item[$x 5 =$] SandSaves; 
\item[$x_6 =$] Scrambling;
\item[$x_7 =$] PuttsPerRound
\end{description}

and the model we are considering is \[
\log(Y) = \beta_0 + \beta_1x_1 + \beta_2x_2 + \beta_3x_3 + \beta_4x_4 + \beta_5x_5 + \beta_6x_6 + \beta_7x_7 + \epsilon
\eqno{(7.10)}
\] We want to do variable selection to choose a subset of the predictors
to model \(\log(Y)\).

\hypertarget{problem-2a}{%
\subsection{Problem 2(a)}\label{problem-2a}}

Identify the optimal model or models based on \(R^2_{adj}\), AIC,
AIC\(_C\), BIC from the approach based on all possible subsets.

\em

Remember that ``all subsets'' selection divides the possible models into
groups according to the number of predictors, up to the maximum number
of predictors (seven, in our case):

\begin{itemize}
\item First find the 1-predictor model with the smallest RSS.
\item Then find the 2-predictor model with the smallest RSS.
\item \ldots{} and so forth up to the 7-predictor model with the smallest RSS.
\end{itemize}

These seven models can then be compared with AIC, BIC, etc.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{golf }\OtherTok{\textless{}{-}} \FunctionTok{read.csv}\NormalTok{(}\StringTok{"pgatour2006.csv"}\NormalTok{)}

\NormalTok{golf.red }\OtherTok{\textless{}{-}}\NormalTok{ golf[,}\FunctionTok{c}\NormalTok{(}\StringTok{"PrizeMoney"}\NormalTok{,}\StringTok{"DrivingAccuracy"}\NormalTok{,}\StringTok{"GIR"}\NormalTok{,}\StringTok{"PuttingAverage"}\NormalTok{,}
                    \StringTok{"BirdieConversion"}\NormalTok{,}\StringTok{"SandSaves"}\NormalTok{,}\StringTok{"Scrambling"}\NormalTok{,}\StringTok{"PuttsPerRound"}\NormalTok{)]}

\NormalTok{all.subsets }\OtherTok{\textless{}{-}} \FunctionTok{regsubsets}\NormalTok{(}\FunctionTok{log}\NormalTok{(PrizeMoney) }\SpecialCharTok{\textasciitilde{}}\NormalTok{ ., }\AttributeTok{data=}\NormalTok{golf.red)}
\end{Highlighting}
\end{Shaded}

We can get some of the results we want from the ``subsets()'' function:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{subsets}\NormalTok{(all.subsets,}\AttributeTok{statistic=}\StringTok{"adjr2"}\NormalTok{,}\AttributeTok{legend=}\ConstantTok{FALSE}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-10-1} \end{center}

\begin{verbatim}
##                  Abbreviation
## DrivingAccuracy             D
## GIR                         G
## PuttingAverage             PA
## BirdieConversion            B
## SandSaves                  SS
## Scrambling                 Sc
## PuttsPerRound              PP
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{subsets}\NormalTok{(all.subsets,}\AttributeTok{statistic=}\StringTok{"bic"}\NormalTok{,}\AttributeTok{legend=}\ConstantTok{FALSE}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-10-2} \end{center}

\begin{verbatim}
##                  Abbreviation
## DrivingAccuracy             D
## GIR                         G
## PuttingAverage             PA
## BirdieConversion            B
## SandSaves                  SS
## Scrambling                 Sc
## PuttsPerRound              PP
\end{verbatim}

But to completely do the problem we need another strategy, since the
``subsets()'' function does not know AIC or CAIC (see the ``statistic''
parameter in Figure 1 below):

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{help}\NormalTok{(subsets)  }
\DocumentationTok{\#\# opens a browser window with the help, but I\textquotesingle{}ve included part of the }
\DocumentationTok{\#\# contents of the browser window in Figure 1 below.}
\end{Highlighting}
\end{Shaded}

\begin{figure}
\centering
\includegraphics[width=1\textwidth,height=\textheight]{"help(subsets).jpg"}
\caption{help(subsets)}
\end{figure}

So, we have to calculate AIC and CAIC (and in the process we'll also see
how to do BIC)

Note that from lecture 08, slide 7, at the bottom, we can write the
log-likelihood as \[
  \mbox{(log-likelihood)} = c_1(n) - c_2(n)\log(RSS) = -\frac{n}{2}\log(2\pi) - \frac{n}{2}\log(RSS)
\] which means we can calculate AIC by hand as\\

\begin{verbatim}  
    AIC = -2(log-liklihood) + 2*(p+2) = [n*log(2pi)] + n*log(RSS) + 2*(p+2)
\end{verbatim}

and since the \verb|[n*log(2pi)]| term will cancel when we subtract
AIC's we can ignore it and just write

\begin{verbatim}
    AIC = n*log(RSS) + 2*(p+2)
\end{verbatim}

Similarly,

\begin{verbatim}
    CAIC = AIC + 2*(p+2)*(p+3)/(n-p-1) = n*log(RSS) + 2*(p+2) + 2*(p+2)*(p+3)/(n-p-1)
\end{verbatim}

and

\begin{verbatim}
    BIC = n*log(RSS) + log(n)*(p+2)
\end{verbatim}

We can get the RSS for each model from

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{tmp }\OtherTok{\textless{}{-}} \FunctionTok{summary}\NormalTok{(all.subsets)  }
\FunctionTok{names}\NormalTok{(tmp)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] "which"  "rsq"    "rss"    "adjr2"  "cp"     "bic"    "outmat" "obj"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{p }\OtherTok{\textless{}{-}} \DecValTok{1}\SpecialCharTok{:}\DecValTok{7} \DocumentationTok{\#\# number of parameters in each subset model}
\NormalTok{n }\OtherTok{\textless{}{-}} \FunctionTok{dim}\NormalTok{(golf.red)[}\DecValTok{1}\NormalTok{]}

\FunctionTok{attach}\NormalTok{(tmp)}
\NormalTok{results }\OtherTok{\textless{}{-}} \FunctionTok{data.frame}\NormalTok{(which,rss,adjr2,}\AttributeTok{bic=}\NormalTok{n}\SpecialCharTok{*}\FunctionTok{log}\NormalTok{(rss)}\SpecialCharTok{+}\FunctionTok{log}\NormalTok{(n)}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{2}\NormalTok{),}\AttributeTok{aic=}\NormalTok{n}\SpecialCharTok{*}\FunctionTok{log}\NormalTok{(rss)}\SpecialCharTok{+}\DecValTok{2}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{2}\NormalTok{),}
                      \AttributeTok{caic=}\NormalTok{n}\SpecialCharTok{*}\FunctionTok{log}\NormalTok{(rss) }\SpecialCharTok{+} \DecValTok{2}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{2}\NormalTok{) }\SpecialCharTok{+} \DecValTok{2}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{2}\NormalTok{)}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{1}\NormalTok{)}\SpecialCharTok{/}\NormalTok{(n}\SpecialCharTok{{-}}\NormalTok{p}\DecValTok{{-}1}\NormalTok{))}
\FunctionTok{detach}\NormalTok{()}
\end{Highlighting}
\end{Shaded}

Note that different authors will use somewhat different definitions of
AIC and BIC. The differences are usually just in what is done with the
constants \(c_1(n)\) and \(c_2(n)\), so the value of the criterion
changes, but the model that minimizes the criterion does not change.

We can print the ``results'' data frame and get the minumum by hand:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{results}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 1         TRUE           FALSE TRUE          FALSE            FALSE     FALSE
## 2         TRUE           FALSE TRUE          FALSE            FALSE     FALSE
## 3         TRUE           FALSE TRUE          FALSE             TRUE     FALSE
## 4         TRUE           FALSE TRUE          FALSE             TRUE      TRUE
## 5         TRUE           FALSE TRUE          FALSE             TRUE      TRUE
## 6         TRUE            TRUE TRUE          FALSE             TRUE      TRUE
## 7         TRUE            TRUE TRUE           TRUE             TRUE      TRUE
##   Scrambling PuttsPerRound       rss     adjr2      bic      aic     caic
## 1      FALSE         FALSE 139.59511 0.2510765 983.8286 973.9943 974.0561
## 2      FALSE          TRUE  95.35465 0.4857746 914.4027 901.2902 901.4146
## 3       TRUE         FALSE  85.19106 0.5381917 897.5903 881.1997 881.4080
## 4       TRUE         FALSE  83.90549 0.5427792 899.8881 880.2195 880.5336
## 5       TRUE          TRUE  82.90524 0.5458520 902.8157 879.8689 880.3110
## 6       TRUE          TRUE  82.86756 0.5436566 908.0047 881.7798 882.3724
## 7       TRUE          TRUE  82.86555 0.5412404 913.2780 883.7750 884.5410
\end{verbatim}

Or we can write a function to identify the row that minimizes each
criteron:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{minimize }\OtherTok{\textless{}{-}} \ControlFlowTok{function}\NormalTok{(res,col) \{}
\NormalTok{  obj }\OtherTok{\textless{}{-}}\NormalTok{ res[,col]}
\NormalTok{  k }\OtherTok{\textless{}{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{:}\FunctionTok{length}\NormalTok{(obj))[obj}\SpecialCharTok{==}\FunctionTok{min}\NormalTok{(obj)]}
  \FunctionTok{return}\NormalTok{(res[k,])}
\NormalTok{\}}

\NormalTok{maximize }\OtherTok{\textless{}{-}} \ControlFlowTok{function}\NormalTok{ (res,col) \{}
\NormalTok{  obj }\OtherTok{\textless{}{-}}\NormalTok{ res[,col]}
\NormalTok{  k }\OtherTok{\textless{}{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{:}\FunctionTok{length}\NormalTok{(obj))[obj}\SpecialCharTok{==}\FunctionTok{max}\NormalTok{(obj)]}
  \FunctionTok{return}\NormalTok{(res[k,])}
\NormalTok{\}}

\FunctionTok{maximize}\NormalTok{(results,}\StringTok{"adjr2"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 5         TRUE           FALSE TRUE          FALSE             TRUE      TRUE
##   Scrambling PuttsPerRound      rss    adjr2      bic      aic    caic
## 5       TRUE          TRUE 82.90524 0.545852 902.8157 879.8689 880.311
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{minimize}\NormalTok{(results,}\StringTok{"bic"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 3         TRUE           FALSE TRUE          FALSE             TRUE     FALSE
##   Scrambling PuttsPerRound      rss     adjr2      bic      aic    caic
## 3       TRUE         FALSE 85.19106 0.5381917 897.5903 881.1997 881.408
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{minimize}\NormalTok{(results,}\StringTok{"aic"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 5         TRUE           FALSE TRUE          FALSE             TRUE      TRUE
##   Scrambling PuttsPerRound      rss    adjr2      bic      aic    caic
## 5       TRUE          TRUE 82.90524 0.545852 902.8157 879.8689 880.311
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{minimize}\NormalTok{(results,}\StringTok{"caic"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 5         TRUE           FALSE TRUE          FALSE             TRUE      TRUE
##   Scrambling PuttsPerRound      rss    adjr2      bic      aic    caic
## 5       TRUE          TRUE 82.90524 0.545852 902.8157 879.8689 880.311
\end{verbatim}

Reading the ``TRUE'' and ``FALSE'' values as when to include or not
include a variable in the model, we see that for ``all subsets''
selection:

\begin{itemize}
\item The optimal $R^2_{adj}$ model is\\ \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + SandSaves + PuttPerRound|
\item The optimal BIC model is\\ \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling|
\item The optimal AIC model is\\ \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + SandSaves + PuttsPerRound|
\item The optimal CAIC model is\\ \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + SandSaves + PuttsPerRound|
\end{itemize}

It's a bit hard to read the graph that \verb|subsets()| gives us for
\(R^2_{adj}\) but clearly the best BIC model we identified is the same
as the one that \verb|subsets()| gives us for BIC.

\rm

\hypertarget{problem-2b}{%
\subsection{Problem 2(b)}\label{problem-2b}}

Identify the optimal model or models based on AIC and BIC from the
approach based on backward selection.

\em

We can specify a version of ``backward selection'' with the ``method''
parameter in the \verb|regsubsets()| function. Instead of looking at all
models at each subset size, this

\begin{itemize}
\item First calculates RSS for the largest model, 7 predictors, in our case 
\item Then takes the 6-predictor model that increases RSS the least from the 7-predictor model
\item Then takes the 5-predictor model that increases RSS the least from the 6-predictor model
\item \ldots{} and so forth down to the 1-predictor model
\end{itemize}

We can then compare these seven models with AIC, BIC, etc.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{backward }\OtherTok{\textless{}{-}} \FunctionTok{regsubsets}\NormalTok{(}\FunctionTok{log}\NormalTok{(PrizeMoney) }\SpecialCharTok{\textasciitilde{}}\NormalTok{ ., }\AttributeTok{data=}\NormalTok{golf.red, }\AttributeTok{method =}\StringTok{"backward"}\NormalTok{)}

\NormalTok{tmp }\OtherTok{\textless{}{-}} \FunctionTok{summary}\NormalTok{(backward)}
\FunctionTok{names}\NormalTok{(tmp)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] "which"  "rsq"    "rss"    "adjr2"  "cp"     "bic"    "outmat" "obj"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{attach}\NormalTok{(tmp)}
\NormalTok{results }\OtherTok{\textless{}{-}} \FunctionTok{data.frame}\NormalTok{(which,}\AttributeTok{bic=}\NormalTok{n}\SpecialCharTok{*}\FunctionTok{log}\NormalTok{(rss)}\SpecialCharTok{+}\FunctionTok{log}\NormalTok{(n)}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{2}\NormalTok{),}\AttributeTok{aic=}\NormalTok{n}\SpecialCharTok{*}\FunctionTok{log}\NormalTok{(rss)}\SpecialCharTok{+}\DecValTok{2}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{2}\NormalTok{))}
\FunctionTok{detach}\NormalTok{()}

\NormalTok{results}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 1         TRUE           FALSE TRUE          FALSE            FALSE     FALSE
## 2         TRUE           FALSE TRUE          FALSE             TRUE     FALSE
## 3         TRUE           FALSE TRUE          FALSE             TRUE     FALSE
## 4         TRUE           FALSE TRUE          FALSE             TRUE      TRUE
## 5         TRUE           FALSE TRUE          FALSE             TRUE      TRUE
## 6         TRUE            TRUE TRUE          FALSE             TRUE      TRUE
## 7         TRUE            TRUE TRUE           TRUE             TRUE      TRUE
##   Scrambling PuttsPerRound      bic      aic
## 1      FALSE         FALSE 983.8286 973.9943
## 2      FALSE         FALSE 925.6317 912.5193
## 3       TRUE         FALSE 897.5903 881.1997
## 4       TRUE         FALSE 899.8881 880.2195
## 5       TRUE          TRUE 902.8157 879.8689
## 6       TRUE          TRUE 908.0047 881.7798
## 7       TRUE          TRUE 913.2780 883.7750
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{minimize}\NormalTok{(results,}\StringTok{"bic"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 3         TRUE           FALSE TRUE          FALSE             TRUE     FALSE
##   Scrambling PuttsPerRound      bic      aic
## 3       TRUE         FALSE 897.5903 881.1997
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{minimize}\NormalTok{(results,}\StringTok{"aic"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 5         TRUE           FALSE TRUE          FALSE             TRUE      TRUE
##   Scrambling PuttsPerRound      bic      aic
## 5       TRUE          TRUE 902.8157 879.8689
\end{verbatim}

We see that for ``backward selection''

\begin{itemize}
\item The optimal BIC model is\\ \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling|
\item The optimal AIC model is\\ \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + SandSaves + PuttsPerRound|
\end{itemize}

\rm

\hypertarget{problem-2c}{%
\subsection{Problem 2(c)}\label{problem-2c}}

Identify the optimal model or models based on AIC and BIC from the
approach based on forward selection.

\em
We can specify a version of "forward selection" with the "method" parameter in the
\verb|regsubsets()| function.  Instead of looking at all models at each subset size, this 

\begin{itemize}
\item First finds the 1-predictor model with the smallest RSS
\item Then takes the 2-predictor model that decreases RSS the most from the 1-predictor model
\item Then takes the 3-predictor model that decreases RSS the most from the 2-predictor model
\item \ldots{} and so forth up to the 7-predictor model
\end{itemize}

We can then compare these seven models with AIC, BIC, etc.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{forward }\OtherTok{\textless{}{-}} \FunctionTok{regsubsets}\NormalTok{(}\FunctionTok{log}\NormalTok{(PrizeMoney) }\SpecialCharTok{\textasciitilde{}}\NormalTok{ ., }\AttributeTok{data=}\NormalTok{golf.red, }\AttributeTok{method =}\StringTok{"forward"}\NormalTok{)}

\NormalTok{tmp }\OtherTok{\textless{}{-}} \FunctionTok{summary}\NormalTok{(forward)}
\FunctionTok{names}\NormalTok{(tmp)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] "which"  "rsq"    "rss"    "adjr2"  "cp"     "bic"    "outmat" "obj"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{attach}\NormalTok{(tmp)}
\NormalTok{results }\OtherTok{\textless{}{-}} \FunctionTok{data.frame}\NormalTok{(which,}\AttributeTok{bic=}\NormalTok{n}\SpecialCharTok{*}\FunctionTok{log}\NormalTok{(rss)}\SpecialCharTok{+}\FunctionTok{log}\NormalTok{(n)}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{2}\NormalTok{),}\AttributeTok{aic=}\NormalTok{n}\SpecialCharTok{*}\FunctionTok{log}\NormalTok{(rss)}\SpecialCharTok{+}\DecValTok{2}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{2}\NormalTok{))}
\FunctionTok{detach}\NormalTok{()}

\NormalTok{results}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 1         TRUE           FALSE TRUE          FALSE            FALSE     FALSE
## 2         TRUE           FALSE TRUE          FALSE            FALSE     FALSE
## 3         TRUE           FALSE TRUE          FALSE             TRUE     FALSE
## 4         TRUE           FALSE TRUE          FALSE             TRUE     FALSE
## 5         TRUE           FALSE TRUE          FALSE             TRUE      TRUE
## 6         TRUE            TRUE TRUE          FALSE             TRUE      TRUE
## 7         TRUE            TRUE TRUE           TRUE             TRUE      TRUE
##   Scrambling PuttsPerRound      bic      aic
## 1      FALSE         FALSE 983.8286 973.9943
## 2      FALSE          TRUE 914.4027 901.2902
## 3      FALSE          TRUE 902.1169 885.7263
## 4       TRUE          TRUE 900.1392 880.4705
## 5       TRUE          TRUE 902.8157 879.8689
## 6       TRUE          TRUE 908.0047 881.7798
## 7       TRUE          TRUE 913.2780 883.7750
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{minimize}\NormalTok{(results,}\StringTok{"bic"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 4         TRUE           FALSE TRUE          FALSE             TRUE     FALSE
##   Scrambling PuttsPerRound      bic      aic
## 4       TRUE          TRUE 900.1392 880.4705
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{minimize}\NormalTok{(results,}\StringTok{"aic"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. DrivingAccuracy  GIR PuttingAverage BirdieConversion SandSaves
## 5         TRUE           FALSE TRUE          FALSE             TRUE      TRUE
##   Scrambling PuttsPerRound      bic      aic
## 5       TRUE          TRUE 902.8157 879.8689
\end{verbatim}

We see that for ``forward selection''

\begin{itemize}
\item The optimal BIC model is\\ \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + PuttsPerRound|
\item The optimal AIC model is\\ \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + SandSaves + PuttsPerRound|
\end{itemize}

\rm

\hypertarget{problem-2d}{%
\subsection{Problem 2(d)}\label{problem-2d}}

Carefully explain why the models chosen in (a) \& (c) are not the same
while those in (a) and (b) are the same.

\em

Here are the results that I got from model selection:

\begin{description}

\item[Best Models from 2(a) ``all subsets'':] ~
\begin{itemize}
\item BIC: \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling|
\item AIC: \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + SandSaves + PuttsPerRound|
\end{itemize}

\item[Best Models from 2(b)``backward'':] ~
\begin{itemize}
\item BIC: \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling|
\item AIC: \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + SandSaves + PuttsPerRound|
\end{itemize}

\item[Best Models from 2(c) ``forward'':] ~
\begin{itemize}
\item BIC: \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + PuttsPerRound|
\item AIC: \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + SandSaves + PuttsPerRound|
\end{itemize}

\end{description}

We know that the ``all subsets'' models are the best of all possible
models on the seven variables we have to work with, whereas the
``backward'' and ``forward'' models come from heuristics that may or may
not find the best of all possible models on these seven variables.

\begin{itemize}

\item The BIC model for "backward" is the same as for "all subsets" because the "backward" selection method stumbled on the best overall model for BIC (look at line 3 of the ``results'' data frame in parts (a) and (b) to verify this).

\item The BIC model for "forward" selection is different because the "forward" heuristic didn't identify the best overall model for BIC (look at line 3 of the ``results'' output in part (a), vs line 4 of the ``results'' output in part (c) to verify this. Note also that the ``line 3'' model in part (c) is \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + PuttsPerRound| instead of the BIC-optimal model \verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling|.

\item The AIC models are the same in each case, because both the "forward" and the "backward" heuristics stumbled on the best overall model for AIC (look at line 5 of the ``results'' data frame in parts (a), (b) and (c) to verify this).
\end{itemize}

\rm

\hypertarget{problem-2e}{%
\subsection{Problem 2(e)}\label{problem-2e}}

Recommend a final model. Give detailed reasons to support your choice.

\em

There are really only two models to consider, from the model selection
above. They are:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lm.AIC }\OtherTok{\textless{}{-}} \FunctionTok{lm}\NormalTok{(}\FunctionTok{log}\NormalTok{(PrizeMoney) }\SpecialCharTok{\textasciitilde{}} \DecValTok{1} \SpecialCharTok{+}\NormalTok{ GIR }\SpecialCharTok{+}\NormalTok{ BirdieConversion }\SpecialCharTok{+}\NormalTok{ Scrambling }\SpecialCharTok{+} 
\NormalTok{               SandSaves }\SpecialCharTok{+}\NormalTok{ PuttsPerRound, }\AttributeTok{data=}\NormalTok{golf.red)}

\NormalTok{lm.BIC }\OtherTok{\textless{}{-}} \FunctionTok{lm}\NormalTok{(}\FunctionTok{log}\NormalTok{(PrizeMoney) }\SpecialCharTok{\textasciitilde{}} \DecValTok{1} \SpecialCharTok{+}\NormalTok{ GIR }\SpecialCharTok{+}\NormalTok{ BirdieConversion }\SpecialCharTok{+}\NormalTok{ Scrambling,}\AttributeTok{data=}\NormalTok{golf.red)}
\end{Highlighting}
\end{Shaded}

Looking at the casewise diagnostic plots,

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{par}\NormalTok{(}\AttributeTok{mfrow=}\FunctionTok{c}\NormalTok{(}\DecValTok{1}\NormalTok{,}\DecValTok{4}\NormalTok{))}
\FunctionTok{plot}\NormalTok{(lm.AIC)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-18-1} \end{center}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{plot}\NormalTok{(lm.BIC)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-18-2} \end{center}

we see that both models seem to satisfy the assumptions of linear
regression equally well.

\begin{description}
\item[Linearity:] There is no real trend in the Residuals vs Fitted plot for either model, so no transforms seem to be needed for either model.
\item[Normality:] The Normal Q-Q plots for both models look about the same.
\item[Constant Variance:] Except for ``edge effects'' the trend lines in both Scale-Location plots are pretty horizontal, supporting the idea of constant variance for both models.
\end{description}

In addition the Residuals vs Leverage plots look about the same for both
models.

Now let's look at summaries and, \textit{since the models are nested}, a
partial F-test to see whether the additional two variables in
\verb|lm.AIC| are provide an improvement over \verb|lm.BIC|.

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lm.AIC)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## Call:
## lm(formula = log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + 
##     SandSaves + PuttsPerRound, data = golf.red)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.71291 -0.48168 -0.09097  0.44843  2.15763 
## 
## Coefficients:
##                   Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      -0.583181   7.158721  -0.081   0.9352    
## GIR               0.197022   0.028711   6.862 9.31e-11 ***
## BirdieConversion  0.162752   0.032672   4.981 1.41e-06 ***
## Scrambling        0.049635   0.024738   2.006   0.0462 *  
## SandSaves         0.015524   0.009743   1.593   0.1127    
## PuttsPerRound    -0.349738   0.230995  -1.514   0.1317    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.6606 on 190 degrees of freedom
## Multiple R-squared:  0.5575, Adjusted R-squared:  0.5459 
## F-statistic: 47.88 on 5 and 190 DF,  p-value: < 2.2e-16
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lm.BIC)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## Call:
## lm(formula = log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling, 
##     data = golf.red)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.71081 -0.50717 -0.06683  0.41975  2.04147 
## 
## Coefficients:
##                   Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      -11.08314    1.45712  -7.606 1.23e-12 ***
## GIR                0.15658    0.01787   8.761 1.01e-15 ***
## BirdieConversion   0.20625    0.02164   9.531  < 2e-16 ***
## Scrambling         0.09178    0.01539   5.965 1.16e-08 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.6661 on 192 degrees of freedom
## Multiple R-squared:  0.5453, Adjusted R-squared:  0.5382 
## F-statistic: 76.75 on 3 and 192 DF,  p-value: < 2.2e-16
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{anova}\NormalTok{(lm.BIC,lm.AIC)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Analysis of Variance Table
## 
## Model 1: log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling
## Model 2: log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling + SandSaves + 
##     PuttsPerRound
##   Res.Df    RSS Df Sum of Sq      F  Pr(>F)  
## 1    192 85.191                              
## 2    190 82.905  2    2.2858 2.6193 0.07548 .
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
\end{verbatim}

Both the \(t\)-statistics in \verb|summary(lm.AIC)| and the partial
\(F\)-test suggest that neither \verb|SandSaves| nor
\verb|PuttsPerRound| is needed.

\textbf{Conclusion:} The smaller model, \verb|lm.BIC|,
\verb|log(PrizeMoney) ~ 1 + GIR + BirdieConversion + Scrambling|, seems
to be the preferable model.

\textbf{Important Note:} Because we are repeatedly using the data, for
variable selection, and to conduct \(t\)-tests and an \(F\)-test, the
results of the tests may be misleading because of capitalization on
chance. The usual result of testing ``significance'\,' after model
selection on the same data is that the results are
\textit{more significant} than they should be.

In this case, we are seeing non-significance for the two variables
\verb|SandSaves| and \verb|PuttsPerRound|, despite this tendency toward
inflated significance. So these two variables probably really are
non-significant predictors.

\rm

\hypertarget{problem-2f}{%
\subsection{Problem 2(f)}\label{problem-2f}}

Interpret the regression coefficients in the final model. Is it
necessary to be cautious about taking these results to literally?

\em

\textbf{The final model is} \verb|lm.BIC|.

Since we are considering \verb|log(PrizeMoney)| we can interpret
coefficient estimates in \verb|summary(lm.BIC)| in terms of percent
change in prize money:

\begin{itemize}
\item An increase of 1 unit in \verb|GIR| (1 extra percent of the time the player was able to hit the greens in regulation) is associated with a $100\times0.15658\%\approx 16\%$ increase in \verb|PrizeMoney|.
\item An increase of 1 unit in \verb|BirdieConversion| (1 extra percent of the time the player makes a ``birdie'' or better after hitting the green in regulation) is associated with a $100\times0.20625\%\approx 21\%$ increase in \verb|PrizeMoney|.
\item An increase of 1 unit in \verb|Scrambling| (1 extra percent of the time the player misses the green in regulation but still makes ``par'') is associated with a $100\times0.09178\%\approx 9\%$ increase in \verb|PrizeMoney|.
\end{itemize}

So it looks like all three of \verb|GIR|, \verb|BirdieConversion| and
\verb|Scrambling| are importantly associated with a golfer's prize
money. We don't know if increases in \verb|BirdieConversion| and
\verb|Scrambling| \textit{cause} a player to earn more prize money, or
if there is a third lurking variable (like natural talent) that
influences all three variables, \verb|BirdieConversion|,
\verb|Scrambling| and \verb|PrizeMoney|.

\textbf{Important Note:} All three variables have small SE's and are
highly significant, but just like with testing after variable selection,
calculating SE's after variable selection usually results in SE's that
are too small, and so predictors look more significant than they
actually are.

In this case, all the \(p\)-values are very very small, so we might
guess that these variable probably really are significant predictors.

\rm

\hypertarget{problem-3-based-on-gelman-hill-2009-p.-51-5}{%
\section{Problem 3: {[}Based on Gelman \& Hill (2009), p.~51,
\#5{]}}\label{problem-3-based-on-gelman-hill-2009-p.-51-5}}

The subfolder \texttt{beauty} in the hw04 folder in the ``Files'' area
for our course on canvas contains data from Hamermesh and Parker (2005)
on student evaluations of instructors' beauty and teaching quality for
several courses at the University of Texas. The teaching evaluations
were conducted at the end of the semester, and the beauty judgments were
made later, by six students who had not attended the classes and were
not aware of the course evaluations. Various documents in the folder
give background and some variable definitions (some variables are
defined in the ``.log'' file there, others' definitions you will have to
deduce from pdf's in the subfolder).

\renewcommand{\labelenumi}{\roman{enumi}}

\hypertarget{problem-3a}{%
\subsection{Problem 3(a)}\label{problem-3a}}

Fit a regression model predicting \texttt{courseevaluation} (average
student evaluations) from \texttt{btystdave} (the average of 6
standardized beauty ratings for each instructor) and \texttt{female}.
Then fit the same model with the interaction between \texttt{btystdave}
and \texttt{female} added in.

\begin{enumerate}
\item Graph each fitted model on a scatter plot of \texttt{courseevaluation} vs \texttt{btystdave}.  Indicate clearly in the graph what the various parameters in the model represent geometrically.
\item Display the four standard diagnostic plots in R and comment on their features, for each model.  Comment on whether the fit seems adequate from the evidence in these plots, for either model.  In case there are problems with the fit, indicate what they are and how you might improve things.
\item Produce summaries of the two fitted models; comment on the coefficient estimates and their standard errors, and on $R^2$, for each model  Use a partial $F$ test to determine whether the interaction should be kept. \textit{ Your comments should include not only technical points (``$B$'' in the ``$ABA^{-1}$'' metaphor for applied statistics from the course syllabus), but also what it means for understanding how factors may influence course evaluations (``$A^{-1}$'').}   
\end{enumerate}

\em
\underline{\bf Part (i):}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{beauty }\OtherTok{\textless{}{-}} \FunctionTok{read.csv}\NormalTok{(}\StringTok{"ProfEvaltnsBeautyPublic.csv"}\NormalTok{)}
\DocumentationTok{\#\# str(beauty) \#\# too long to include here...}
\NormalTok{lm}\FloatTok{.1} \OtherTok{\textless{}{-}} \FunctionTok{lm}\NormalTok{(courseevaluation }\SpecialCharTok{\textasciitilde{}}\NormalTok{ btystdave }\SpecialCharTok{+}\NormalTok{ female,}\AttributeTok{data=}\NormalTok{beauty)}
\FunctionTok{print}\NormalTok{(beta.lm}\FloatTok{.1} \OtherTok{\textless{}{-}} \FunctionTok{coef}\NormalTok{(lm}\FloatTok{.1}\NormalTok{))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## (Intercept)   btystdave      female 
##   4.0947104   0.1485876  -0.1978096
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lm}\FloatTok{.2} \OtherTok{\textless{}{-}} \FunctionTok{lm}\NormalTok{(courseevaluation }\SpecialCharTok{\textasciitilde{}}\NormalTok{ btystdave }\SpecialCharTok{*}\NormalTok{ female,}\AttributeTok{data=}\NormalTok{beauty)}
\FunctionTok{print}\NormalTok{(beta.lm}\FloatTok{.2} \OtherTok{\textless{}{-}} \FunctionTok{coef}\NormalTok{(lm}\FloatTok{.2}\NormalTok{))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##      (Intercept)        btystdave           female btystdave:female 
##        4.1036435        0.2002743       -0.2050501       -0.1126579
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{par}\NormalTok{(}\AttributeTok{mfrow=}\FunctionTok{c}\NormalTok{(}\DecValTok{1}\NormalTok{,}\DecValTok{2}\NormalTok{))}
\FunctionTok{plot}\NormalTok{(courseevaluation }\SpecialCharTok{\textasciitilde{}}\NormalTok{ btystdave, }\AttributeTok{data=}\NormalTok{beauty,}\AttributeTok{main=}\StringTok{"lm.1"}\NormalTok{,}
     \AttributeTok{col=}\FunctionTok{c}\NormalTok{(}\StringTok{"blue"}\NormalTok{,}\StringTok{"red"}\NormalTok{)[beauty}\SpecialCharTok{$}\NormalTok{female}\SpecialCharTok{+}\DecValTok{1}\NormalTok{])}
\FunctionTok{abline}\NormalTok{(beta.lm}\FloatTok{.1}\NormalTok{[}\DecValTok{1}\NormalTok{],beta.lm}\FloatTok{.1}\NormalTok{[}\DecValTok{2}\NormalTok{],}\AttributeTok{col=}\StringTok{"blue"}\NormalTok{)}
\FunctionTok{abline}\NormalTok{(beta.lm}\FloatTok{.1}\NormalTok{[}\DecValTok{1}\NormalTok{]}\SpecialCharTok{+}\NormalTok{beta.lm}\FloatTok{.1}\NormalTok{[}\DecValTok{3}\NormalTok{],beta.lm}\FloatTok{.1}\NormalTok{[}\DecValTok{2}\NormalTok{],}\AttributeTok{col=}\StringTok{"red"}\NormalTok{)}
\FunctionTok{legend}\NormalTok{(}\FloatTok{0.5}\NormalTok{,}\FloatTok{2.5}\NormalTok{,}\AttributeTok{lty=}\DecValTok{1}\NormalTok{,}\AttributeTok{col=}\FunctionTok{c}\NormalTok{(}\StringTok{"blue"}\NormalTok{,}\StringTok{"red"}\NormalTok{),}
       \AttributeTok{legend=}\FunctionTok{c}\NormalTok{(}\StringTok{"male instructor"}\NormalTok{,}\StringTok{"female instructor"}\NormalTok{),}\AttributeTok{cex=}\FloatTok{0.55}\NormalTok{)}
\FunctionTok{plot}\NormalTok{(courseevaluation }\SpecialCharTok{\textasciitilde{}}\NormalTok{ btystdave, }\AttributeTok{data=}\NormalTok{beauty,}\AttributeTok{main=}\StringTok{"lm.2"}\NormalTok{,}
     \AttributeTok{col=}\FunctionTok{c}\NormalTok{(}\StringTok{"blue"}\NormalTok{,}\StringTok{"red"}\NormalTok{)[beauty}\SpecialCharTok{$}\NormalTok{female}\SpecialCharTok{+}\DecValTok{1}\NormalTok{])}
\FunctionTok{abline}\NormalTok{(beta.lm}\FloatTok{.2}\NormalTok{[}\DecValTok{1}\NormalTok{],beta.lm}\FloatTok{.2}\NormalTok{[}\DecValTok{2}\NormalTok{],}\AttributeTok{col=}\StringTok{"blue"}\NormalTok{)}
\FunctionTok{abline}\NormalTok{(beta.lm}\FloatTok{.2}\NormalTok{[}\DecValTok{1}\NormalTok{]}\SpecialCharTok{+}\NormalTok{beta.lm}\FloatTok{.2}\NormalTok{[}\DecValTok{3}\NormalTok{],beta.lm}\FloatTok{.2}\NormalTok{[}\DecValTok{2}\NormalTok{]}\SpecialCharTok{+}\NormalTok{beta.lm}\FloatTok{.2}\NormalTok{[}\DecValTok{4}\NormalTok{],}\AttributeTok{col=}\StringTok{"red"}\NormalTok{)}
\FunctionTok{legend}\NormalTok{(}\FloatTok{0.5}\NormalTok{,}\FloatTok{2.5}\NormalTok{,}\AttributeTok{lty=}\DecValTok{1}\NormalTok{,}\AttributeTok{col=}\FunctionTok{c}\NormalTok{(}\StringTok{"blue"}\NormalTok{,}\StringTok{"red"}\NormalTok{),}
       \AttributeTok{legend=}\FunctionTok{c}\NormalTok{(}\StringTok{"male instructor"}\NormalTok{,}\StringTok{"female instructor"}\NormalTok{),}\AttributeTok{cex=}\FloatTok{0.55}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-20-1} \end{center}

For the additive model (\verb|lm.1|, the plot on the left), the slope of
the line relating \verb|courseevaluation| to \verb|btystdave| is 0.15,
for both male and female instructors. The intercept for male instructors
is 4.09, and for female instructors it's 3.9. In this model, male
instructors get about a 0.2 boost in course evaluations, vs.~female
instructors.

For the interactive model (\verb|lm.2|, the plot on the right), the
slope of the line relating \verb|courseevaluation| to \verb|btystdave|
is 0.2 for male instructors and 0.09 for female instructors. The
intercept for male instructors is 4.1 whereas for female instructors it
is 3.9. Thus according to this model, evaluations for courses with male
instructors benefit by about 0.21 to start with, and the slope as a
function of \verb|btystave| is about 0.11 higher for male instructors,
than for female instructors.

\underline{\bf Part (ii):}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{par}\NormalTok{(}\AttributeTok{mfrow=}\FunctionTok{c}\NormalTok{(}\DecValTok{1}\NormalTok{,}\DecValTok{4}\NormalTok{))}
\FunctionTok{plot}\NormalTok{(lm}\FloatTok{.1}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-21-1} \end{center}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{plot}\NormalTok{(lm}\FloatTok{.2}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-21-2} \end{center}

There is very little difference between the two sets of casewise
diagnostic plots. In both models, there is no functional pattern in the
raw residual plot, slightly short tails in the residuals in the Normal
Q-Q plots, approximately constant variance in the Scale-Location plots,
and very similar, and unconcerning residuals vs.~leverage plots.

So, the fits of both models seem adequate (except perhaps for short
tails in the Normal Q-Q plots, which don't concern me very much), and I
would not do any transformations, etc. to improve either fit.

About the only difference I see is that the dsitribution of fitted
values appear to be slightly higher for \verb|lm.2| than \verb|lm.1|
(e.g.~see the x-axis of the residuals vs fitted plots).

\underline{\bf Part (iii):}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lm}\FloatTok{.1}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## Call:
## lm(formula = courseevaluation ~ btystdave + female, data = beauty)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.87196 -0.36913  0.03493  0.39919  1.03237 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  4.09471    0.03328  123.03  < 2e-16 ***
## btystdave    0.14859    0.03195    4.65 4.34e-06 ***
## female      -0.19781    0.05098   -3.88  0.00012 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5373 on 460 degrees of freedom
## Multiple R-squared:  0.0663, Adjusted R-squared:  0.06224 
## F-statistic: 16.33 on 2 and 460 DF,  p-value: 1.407e-07
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lm}\FloatTok{.2}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## Call:
## lm(formula = courseevaluation ~ btystdave * female, data = beauty)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.83820 -0.37387  0.04551  0.39876  1.06764 
## 
## Coefficients:
##                  Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       4.10364    0.03359 122.158  < 2e-16 ***
## btystdave         0.20027    0.04333   4.622 4.95e-06 ***
## female           -0.20505    0.05103  -4.018 6.85e-05 ***
## btystdave:female -0.11266    0.06398  -1.761   0.0789 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.5361 on 459 degrees of freedom
## Multiple R-squared:  0.07256,    Adjusted R-squared:  0.0665 
## F-statistic: 11.97 on 3 and 459 DF,  p-value: 1.471e-07
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{anova}\NormalTok{(lm}\FloatTok{.1}\NormalTok{,lm}\FloatTok{.2}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Analysis of Variance Table
## 
## Model 1: courseevaluation ~ btystdave + female
## Model 2: courseevaluation ~ btystdave * female
##   Res.Df    RSS Df Sum of Sq     F  Pr(>F)  
## 1    460 132.81                             
## 2    459 131.92  1   0.89124 3.101 0.07891 .
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
\end{verbatim}

The most noticable thing about both models is that their \(R^2\)'s are
pitiful: 0.066 for \verb|lm.1| and 0.073 for \verb|lm.2|. So, even
though we have some significant predictors in both models, they are not
doing a very good job of accounting for the variation in
\verb|courseevaluation|.

In both models, \verb|btystave| and \verb|female| are highly significant
predictors, and in \verb|lm.2|, the interaction term
\verb|btystdave:female| is just barely non-significant (\(p\)-value
\(=\) 0.0789).

Because we are only adding one more variable, the interaction term, the
\(F\)-test statistic is just the square of the \(t\)-statistic for the
interaction in the \verb|summary()| output, and hence there is no new
information: the \(p\)-value is again 0.0789. So, we would probably
prefer the simpler model \verb|lm.1|.

In practical terms, model \verb|lm.1| says that a course with a male
instructor starts off with a course evaluation of about 4.09, but this
drops to about 3.9 if the course has a female instructor. The increase
in course evaluation per unit increase in beauty rating is the same for
both male and female instructors, about 0.15.

The standard errors for the coefficients are all quite small (but
perhaps a bit smaller than they should be since we used the data twice:
once to select a model and once to estimate coefficients), and generally
they are reassuring that the effects quoted in the last paragraph are
``real''.

\rm

\hypertarget{problem-3b}{%
\subsection{Problem 3(b)}\label{problem-3b}}

Now let's look at \textit{ all} of the variables in the data set. Should
any of the variables in the data set be transformed before being used in
a regression model? List each variable that is not a dummy variable, and
for each of these,

\begin{itemize}
\item Say whether the variable should be transformed (yes or no)
\item If yes, indicate what transformation you would make
\item Justify these two answers, using both evidence from the data and other considerations
\end{itemize}

Note: being able to communicate with a client or collaborator matters,
so there may be instances where either (a) a transformation might help,
but you decide against it since it would be difficult to explain to a
client/collaborator, or (b) an automatic method like Box-Cox might
suggest one power, but you pick a simpler power ``nearby'\,' because it
is easier to explain to a collaborator/client.

\em

Since we're not going to consider \texttt{profnumber},
\texttt{multipleclass} and \texttt{class1} through \texttt{class30} in
part (c), I will eliminate them now before we consider transformations.

I will use Box-Cox to suggest transformations, and then choose more
interpretable transformations based on Box-Cox.

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{names}\NormalTok{(beauty)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##  [1] "tenured"           "profnumber"        "minority"         
##  [4] "age"               "beautyf2upper"     "beautyflowerdiv"  
##  [7] "beautyfupperdiv"   "beautym2upper"     "beautymlowerdiv"  
## [10] "beautymupperdiv"   "btystdave"         "btystdf2u"        
## [13] "btystdfl"          "btystdfu"          "btystdm2u"        
## [16] "btystdml"          "btystdmu"          "class1"           
## [19] "class2"            "class3"            "class4"           
## [22] "class5"            "class6"            "class7"           
## [25] "class8"            "class9"            "class10"          
## [28] "class11"           "class12"           "class13"          
## [31] "class14"           "class15"           "class16"          
## [34] "class17"           "class18"           "class19"          
## [37] "class20"           "class21"           "class22"          
## [40] "class23"           "class24"           "class25"          
## [43] "class26"           "class27"           "class28"          
## [46] "class29"           "class30"           "courseevaluation" 
## [49] "didevaluation"     "female"            "formal"           
## [52] "fulldept"          "lower"             "multipleclass"    
## [55] "nonenglish"        "onecredit"         "percentevaluating"
## [58] "profevaluation"    "students"          "tenuretrack"      
## [61] "blkandwhite"       "btystdvariance"    "btystdavepos"     
## [64] "btystdaveneg"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{prof.loc }\OtherTok{\textless{}{-}} \FunctionTok{grep}\NormalTok{(}\StringTok{"profnumber"}\NormalTok{,}\FunctionTok{names}\NormalTok{(beauty))}
\NormalTok{multiclass.loc }\OtherTok{\textless{}{-}} \FunctionTok{grep}\NormalTok{(}\StringTok{"multipleclass"}\NormalTok{,}\FunctionTok{names}\NormalTok{(beauty))}
\NormalTok{class.locs }\OtherTok{\textless{}{-}} \FunctionTok{grep}\NormalTok{(}\StringTok{"class"}\NormalTok{,}\FunctionTok{names}\NormalTok{(beauty))}
\NormalTok{beauty.red }\OtherTok{\textless{}{-}}\NormalTok{ beauty[,}\SpecialCharTok{{-}}\FunctionTok{c}\NormalTok{(prof.loc,multiclass.loc,class.locs)]}
\end{Highlighting}
\end{Shaded}

From the \verb|str()| command, our background knowledge of college, and
some examination of the papers and variable glossary in the
\verb|beauty| subdirectory, we find \small

\begin{description}\itemsep=0pt\parsep=0pt\parskip=0pt
\item[tenured          ]  dummy: instructor tenured? 0=no, 1=yes
\item[minority         ]  dummy: instructor minority? 0=no, 1=yes
\item[age              ]  continuous: 36 59 51 40 31 62 33 51 33 47 ...
\item[beautyf2upper    ]  continuous: 6 2 5 4 9 5 5 6 5 6 ...
\item[beautyflowerdiv  ]  continuous: 5 4 5 2 7 6 4 4 3 5 ...
\item[beautyfupperdiv  ]  continuous: 7 4 2 5 9 6 4 6 7 7 ...
\item[beautym2upper    ]  continuous: 6 3 3 2 6 6 4 3 5 6 ...
\item[beautymlowerdiv  ]  continuous: 2 2 2 3 7 5 4 2 5 3 ...
\item[beautymupperdiv  ]  continuous: 4 3 3 3 6 5 4 3 3 6 ...
\item[btystdave        ]  continuous: 0.202 -0.826 -0.66 -0.766 1.421 ...
\item[btystdf2u        ]  continuous: 0.289 -1.619 -0.188 -0.665 1.721 ...
\item[btystdfl         ]  continuous: 0.458 -0.0735 0.458 -1.1365 1.521 ...
\item[btystdfu         ]  continuous: 0.8758 -0.577 -1.5456 -0.0927 1.8444 ...
\item[btystdm2u        ]  continuous: 0.682 -1.132 -1.132 -1.736 0.682 ...
\item[btystdml         ]  continuous: -0.9 -0.9 -0.9 -0.313 2.038 ...
\item[btystdmu         ]  continuous: -0.195 -0.655 -0.655 -0.655 0.723 ...
\item[courseevaluation ]  continuous: 4.3 4.5 3.7 4.3 4.4 4.2 4 3.4 4.5 3.9 ...
\item[didevaluation    ]  continuous: 24 17 55 40 42 182 33 25 48 16 ...
\item[female           ]  dummy: instructor female? 0=no, 1=yes
\item[formal           ]  dummy: web pic of instructor wears tie/jacket (dress)? 0=no, 1=yes
\item[fulldept         ]  dummy: everyone in dept has a web pic? 0=no, 1=yes
\item[lower            ]  dummy: lower division (freshman/sophomre)? 0=no, 1=yes
\item[nonenglish       ]  dummy: instructor non-native English speaker? 0=no, 1=yes
\item[onecredit        ]  dummy: a one-credit course? 0=no, 1=yes
\item[percentevaluating]  continuous: 55.8 85 100 87 87.5 ...
\item[profevaluation   ]  continuous: 4.7 4.6 4.1 4.5 4.8 4.4 4.4 3.4 4.8 4 ...
\item[students         ]  continuous: 43 20 55 46 48 282 41 41 60 19 ...
\item[tenuretrack      ]  dummy: is instructor in tenure-track? 0=no, 1=yes
\item[blkandwhite      ]  dummy: (don't know what this is!)
\item[btystdvariance   ]  continuous: 2.13 1.39 2.54 1.76 1.69 ...
\item[btystdavepos     ]  continuous: 0.202 0 0 0 1.421 ...
\item[btystdaveneg     ]  continuous: 0 -0.826 -0.66 -0.766 0 ...
\end{description}
\normalsize

There are a few relationships we can see or guess immediately:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{attach}\NormalTok{(beauty.red)}
\FunctionTok{sum}\NormalTok{(}\FunctionTok{abs}\NormalTok{(btystdave }\SpecialCharTok{{-}}\NormalTok{ (btystdf2u}\SpecialCharTok{+}\NormalTok{btystdfl}\SpecialCharTok{+}\NormalTok{btystdfu}\SpecialCharTok{+}\NormalTok{btystdm2u}\SpecialCharTok{+}\NormalTok{btystdml}\SpecialCharTok{+}\NormalTok{btystdmu)}\SpecialCharTok{/}\DecValTok{6}\NormalTok{))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 4.693333e-05
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{sum}\NormalTok{(}\FunctionTok{abs}\NormalTok{(btystdave }\SpecialCharTok{{-}}\NormalTok{ (btystdavepos}\SpecialCharTok{+}\NormalTok{btystdaveneg)))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 0.0004629
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{detach}\NormalTok{()}
\end{Highlighting}
\end{Shaded}

So,up to rounding error, we see that

\begin{itemize}
\item \verb|btystdave| is in fact \verb|(btystdf2u+btystdfl+btystdfu+btystdm2u+btystdml+btystdmu)/6|
\item \verb|btystdave| is also \verb|btystdavepos+btystdaveneg|
\end{itemize}

So we will keep \verb|btystdave| and eliminate the variables that sum up
to it. (There is no loss in eliminating \verb|btystdavepos| and
\verb|btystdaveneg|, since these are just the ``positive part'' and
``negative part'' of \verb|btystdave|. There is a little loss in
eliminating \verb|btystdf2u, btystdfl, btystdfu, btystdm2u, btystdml|
and \verb|btystdmu|, but hopefully not too much since we will keep the
unstandardized versions.)

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{btystd.locs }\OtherTok{\textless{}{-}} \FunctionTok{grep}\NormalTok{(}\StringTok{"btystd"}\NormalTok{,}\FunctionTok{names}\NormalTok{(beauty.red))[}\SpecialCharTok{{-}}\FunctionTok{c}\NormalTok{(}\DecValTok{1}\NormalTok{,}\DecValTok{8}\NormalTok{)] }
\DocumentationTok{\#\# we are keeping btystdave, which comes first in this list,}
\DocumentationTok{\#\# and btystdvariance, which is eighth...}
\NormalTok{beauty.red }\OtherTok{\textless{}{-}}\NormalTok{ beauty.red[,}\SpecialCharTok{{-}}\NormalTok{btystd.locs]}
\FunctionTok{names}\NormalTok{(beauty.red)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##  [1] "tenured"           "minority"          "age"              
##  [4] "beautyf2upper"     "beautyflowerdiv"   "beautyfupperdiv"  
##  [7] "beautym2upper"     "beautymlowerdiv"   "beautymupperdiv"  
## [10] "btystdave"         "courseevaluation"  "didevaluation"    
## [13] "female"            "formal"            "fulldept"         
## [16] "lower"             "nonenglish"        "onecredit"        
## [19] "percentevaluating" "profevaluation"    "students"         
## [22] "tenuretrack"       "blkandwhite"       "btystdvariance"
\end{verbatim}

The variables we want to consider for transformation are all of the
non-dummies that are left:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{trans.names }\OtherTok{\textless{}{-}} \FunctionTok{names}\NormalTok{(beauty.red)[}\FunctionTok{c}\NormalTok{(}\DecValTok{3}\SpecialCharTok{:}\DecValTok{12}\NormalTok{,}\DecValTok{19}\SpecialCharTok{:}\DecValTok{21}\NormalTok{,}\DecValTok{24}\NormalTok{)]}

\NormalTok{powers }\OtherTok{\textless{}{-}} \ConstantTok{NULL}
\ControlFlowTok{for}\NormalTok{ (i }\ControlFlowTok{in}\NormalTok{ trans.names) \{}
\NormalTok{  x }\OtherTok{\textless{}{-}}\NormalTok{ beauty.red[,i]}
  \ControlFlowTok{if}\NormalTok{ (}\FunctionTok{min}\NormalTok{(x)}\SpecialCharTok{==}\DecValTok{0}\NormalTok{) \{ }
\NormalTok{    x }\OtherTok{\textless{}{-}}\NormalTok{ x }\SpecialCharTok{+} \FloatTok{0.01}\SpecialCharTok{*}\FunctionTok{max}\NormalTok{(x)}
\NormalTok{    \} }\ControlFlowTok{else} \ControlFlowTok{if}\NormalTok{ (}\FunctionTok{min}\NormalTok{(x)}\SpecialCharTok{\textless{}}\DecValTok{0}\NormalTok{) \{}
\NormalTok{      x }\OtherTok{\textless{}{-}}\NormalTok{ x }\SpecialCharTok{{-}} \FunctionTok{min}\NormalTok{(x)}\SpecialCharTok{*}\FloatTok{1.01}
\NormalTok{      \}}
  \DocumentationTok{\#\# forces each variable to be strictly positive, for Box{-}Cox}
\NormalTok{  powers }\OtherTok{\textless{}{-}} \FunctionTok{c}\NormalTok{(powers,}\FunctionTok{powerTransform}\NormalTok{(x)}\SpecialCharTok{$}\NormalTok{lambda)}
\NormalTok{\}}
\FunctionTok{names}\NormalTok{(powers) }\OtherTok{\textless{}{-}}\NormalTok{ trans.names}
\NormalTok{powers }\OtherTok{\textless{}{-}} \FunctionTok{round}\NormalTok{(powers,}\DecValTok{2}\NormalTok{)}
\NormalTok{powers}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##               age     beautyf2upper   beautyflowerdiv   beautyfupperdiv 
##              0.74              0.77              0.54              0.87 
##     beautym2upper   beautymlowerdiv   beautymupperdiv         btystdave 
##              0.82              0.37              0.44              0.61 
##  courseevaluation     didevaluation percentevaluating    profevaluation 
##              2.20             -0.35              1.94              3.09 
##          students    btystdvariance 
##             -0.47              0.29
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{par}\NormalTok{(}\AttributeTok{mfrow=}\FunctionTok{c}\NormalTok{(}\DecValTok{5}\NormalTok{,}\DecValTok{3}\NormalTok{))}
\ControlFlowTok{for}\NormalTok{ (i }\ControlFlowTok{in}\NormalTok{ trans.names) \{}
  \FunctionTok{plot}\NormalTok{(}\FunctionTok{density}\NormalTok{(beauty.red[,i]),}\AttributeTok{main=}\NormalTok{i,}\AttributeTok{xlab=}\FunctionTok{paste}\NormalTok{(}\StringTok{"Suggested power ="}\NormalTok{,powers[i]))}
\NormalTok{\}}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-26-1} \end{center}

I'm going to round these suggested powers as follows:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{bjpowers }\OtherTok{\textless{}{-}}\NormalTok{ powers}
\NormalTok{bjpowers[}\StringTok{"age"}\NormalTok{] }\OtherTok{\textless{}{-}} \DecValTok{1} \DocumentationTok{\#\# already fairly symmetric}
\NormalTok{bjpowers[}\StringTok{"beautyf2upper"}\NormalTok{]   }\OtherTok{\textless{}{-}} \FloatTok{0.5}  \DocumentationTok{\#\# right skewed, sqrt() easier to talk about }
\NormalTok{bjpowers[}\StringTok{"beautyflowerdiv"}\NormalTok{] }\OtherTok{\textless{}{-}} \FloatTok{0.5}  \DocumentationTok{\#\# right skewed, sqrt() easier to talk about }
\NormalTok{bjpowers[}\StringTok{"beautyfupperdiv"}\NormalTok{] }\OtherTok{\textless{}{-}} \FloatTok{0.5}  \DocumentationTok{\#\# right skewed, sqrt() easier to talk about }
\NormalTok{bjpowers[}\StringTok{"beautym2upper"}\NormalTok{]   }\OtherTok{\textless{}{-}} \FloatTok{0.5}  \DocumentationTok{\#\# right skewed, sqrt() easier to talk about }
\NormalTok{bjpowers[}\StringTok{"beautymlowerdiv"}\NormalTok{] }\OtherTok{\textless{}{-}} \FloatTok{0.5}  \DocumentationTok{\#\# right skewed, sqrt() easier to talk about }
\NormalTok{bjpowers[}\StringTok{"beautymupperdiv"}\NormalTok{] }\OtherTok{\textless{}{-}} \FloatTok{0.5}  \DocumentationTok{\#\# right skewed, sqrt() easier to talk about }
\NormalTok{bjpowers[}\StringTok{"btystdave"}\NormalTok{] }\OtherTok{\textless{}{-}} \DecValTok{1} \DocumentationTok{\#\# some right skew but reasonably bounded between {-}2 and +2}
\NormalTok{bjpowers[}\StringTok{"courseevaluation"}\NormalTok{] }\OtherTok{\textless{}{-}} \DecValTok{1}   \DocumentationTok{\#\# even though 2.2 is suggested, I\textquotesingle{}m leaving this }
                                    \DocumentationTok{\#\# alone, to make the linear regression coefficients}
                                    \DocumentationTok{\#\# more interpretable}
\NormalTok{bjpowers[}\StringTok{"didevaluation"}\NormalTok{]   }\OtherTok{\textless{}{-}} \SpecialCharTok{{-}}\FloatTok{0.5} \DocumentationTok{\#\# I may try a log here too.}
\NormalTok{bjpowers[}\StringTok{"percentevaluating"}\NormalTok{] }\OtherTok{\textless{}{-}} \DecValTok{2}  \DocumentationTok{\#\# close, perhaps more interpretable}
\NormalTok{bjpowers[}\StringTok{"profevaluation"}\NormalTok{]   }\OtherTok{\textless{}{-}} \DecValTok{3}   \DocumentationTok{\#\# I may try 2 later..}
\NormalTok{bjpowers[}\StringTok{"students"}\NormalTok{]        }\OtherTok{\textless{}{-}} \SpecialCharTok{{-}}\FloatTok{0.5} \DocumentationTok{\#\# I may try a log here too.}
\NormalTok{bjpowers[}\StringTok{"btystdvariance"}\NormalTok{]  }\OtherTok{\textless{}{-}} \FloatTok{0.5}  \DocumentationTok{\#\# 0.29 is called for but hard to talk about}
\end{Highlighting}
\end{Shaded}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{par}\NormalTok{(}\AttributeTok{mfrow=}\FunctionTok{c}\NormalTok{(}\DecValTok{5}\NormalTok{,}\DecValTok{3}\NormalTok{))}
\ControlFlowTok{for}\NormalTok{ (i }\ControlFlowTok{in}\NormalTok{ trans.names) \{}
  \FunctionTok{plot}\NormalTok{(}\FunctionTok{density}\NormalTok{(beauty.red[,i]}\SpecialCharTok{\^{}}\NormalTok{bjpowers[i]),}\AttributeTok{main=}\FunctionTok{paste}\NormalTok{(i,bjpowers[i],}\AttributeTok{sep=}\StringTok{"\^{}"}\NormalTok{),}\AttributeTok{xlab=}\StringTok{""}\NormalTok{)}
\NormalTok{\}}
\end{Highlighting}
\end{Shaded}

\begin{center}\includegraphics{hw04-solutions_files/figure-latex/unnamed-chunk-28-1} \end{center}

We'll go with these transformations, though you can see in my comments
above that some others might be better\ldots

\rm

\hypertarget{problem-3c}{%
\subsection{Problem 3(c)}\label{problem-3c}}

Fit the model that regresses \texttt{courseevaluation} onto all other
variables, except for \texttt{profnumber}, \texttt{multipleclass}, and
the 30 \texttt{class} variables (\texttt{class1} through
\texttt{class30}). Use the transformations you recommended in part (b).
Make a table indicating

\begin{itemize}
\item The t-statistics for each variable
\item The VIFs for each variable
\end{itemize}

in your model.

\em

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{beauty.trans }\OtherTok{\textless{}{-}}\NormalTok{ beauty.red}
\ControlFlowTok{for}\NormalTok{ (i }\ControlFlowTok{in}\NormalTok{ trans.names) \{}
\NormalTok{  beauty.trans[,i] }\OtherTok{\textless{}{-}}\NormalTok{ beauty.red[,i]}\SpecialCharTok{\^{}}\NormalTok{bjpowers[i]}
  \FunctionTok{names}\NormalTok{(beauty.trans)[}\FunctionTok{grep}\NormalTok{(i,}\FunctionTok{names}\NormalTok{(beauty.trans))] }\OtherTok{\textless{}{-}} \FunctionTok{paste}\NormalTok{(}\StringTok{"t"}\NormalTok{,i,}\AttributeTok{sep=}\StringTok{"."}\NormalTok{)}
\NormalTok{\}}
\FunctionTok{names}\NormalTok{(beauty.trans)[}\FunctionTok{grep}\NormalTok{(}\StringTok{"t.btystdave"}\NormalTok{,}\FunctionTok{names}\NormalTok{(beauty.trans))] }\OtherTok{\textless{}{-}} \StringTok{"btystdave"} 
\DocumentationTok{\#\# didn\textquotesingle{}t transform btystdave...}
\FunctionTok{names}\NormalTok{(beauty.trans)[}\FunctionTok{grep}\NormalTok{(}\StringTok{"t.courseevaluation"}\NormalTok{,}\FunctionTok{names}\NormalTok{(beauty.trans))] }\OtherTok{\textless{}{-}} \StringTok{"courseevaluation"} 
\DocumentationTok{\#\# didn\textquotesingle{}t transform courseevaluation...}
\FunctionTok{names}\NormalTok{(beauty.trans)[}\FunctionTok{grep}\NormalTok{(}\StringTok{"t.age"}\NormalTok{,}\FunctionTok{names}\NormalTok{(beauty.trans))] }\OtherTok{\textless{}{-}} \StringTok{"age"} 
\DocumentationTok{\#\# didn\textquotesingle{}t transform age...}
\FunctionTok{names}\NormalTok{(beauty.trans)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##  [1] "tenured"             "minority"            "age"                
##  [4] "t.beautyf2upper"     "t.beautyflowerdiv"   "t.beautyfupperdiv"  
##  [7] "t.beautym2upper"     "t.beautymlowerdiv"   "t.beautymupperdiv"  
## [10] "btystdave"           "courseevaluation"    "t.didevaluation"    
## [13] "female"              "formal"              "fulldept"           
## [16] "lower"               "nonenglish"          "onecredit"          
## [19] "t.percentevaluating" "t.profevaluation"    "t.students"         
## [22] "tenuretrack"         "blkandwhite"         "t.btystdvariance"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lm}\FloatTok{.1} \OtherTok{\textless{}{-}} \FunctionTok{lm}\NormalTok{(courseevaluation }\SpecialCharTok{\textasciitilde{}}\NormalTok{ ., }\AttributeTok{data=}\NormalTok{beauty.trans)}
\NormalTok{tab }\OtherTok{\textless{}{-}} \FunctionTok{cbind}\NormalTok{(}\FunctionTok{summary}\NormalTok{(lm}\FloatTok{.1}\NormalTok{)}\SpecialCharTok{$}\NormalTok{coef, }\AttributeTok{vif=}\FunctionTok{c}\NormalTok{(}\ConstantTok{NA}\NormalTok{,}\FunctionTok{vif}\NormalTok{(lm}\FloatTok{.1}\NormalTok{)))}

\FunctionTok{round}\NormalTok{(tab,}\DecValTok{4}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                     Estimate Std. Error t value Pr(>|t|)      vif
## (Intercept)           2.8629     0.5349  5.3523   0.0000       NA
## tenured               0.0326     0.0307  1.0598   0.2898   2.7083
## minority             -0.0194     0.0324 -0.5994   0.5492   1.4469
## age                   0.0014     0.0014  0.9948   0.3204   2.1224
## t.beautyf2upper      -0.0285     0.0533 -0.5351   0.5928   7.0688
## t.beautyflowerdiv    -0.0282     0.0534 -0.5279   0.5978   7.8156
## t.beautyfupperdiv     0.0131     0.0481  0.2726   0.7853   5.6271
## t.beautym2upper      -0.0432     0.0633 -0.6825   0.4953   6.5295
## t.beautymlowerdiv    -0.0592     0.0576 -1.0282   0.3044   7.7566
## t.beautymupperdiv    -0.0298     0.0499 -0.5967   0.5510   8.0266
## btystdave             0.0996     0.1190  0.8367   0.4032 101.6065
## t.didevaluation      -0.7943     0.6439 -1.2335   0.2181  25.6187
## female               -0.0331     0.0233 -1.4186   0.1567   1.5344
## formal                0.0152     0.0287  0.5285   0.5974   1.3193
## fulldept             -0.0038     0.0364 -0.1055   0.9160   1.4497
## lower                 0.0051     0.0236  0.2163   0.8289   1.4452
## nonenglish           -0.0592     0.0464 -1.2765   0.2025   1.4141
## onecredit             0.0655     0.0497  1.3189   0.1879   1.5669
## t.percentevaluating   0.0000     0.0000 -0.3371   0.7362   5.2175
## t.profevaluation      0.0187     0.0004 46.5582   0.0000   1.3235
## t.students            1.2703     0.8063  1.5755   0.1159  31.9203
## tenuretrack          -0.0400     0.0340 -1.1783   0.2393   2.2934
## blkandwhite           0.0174     0.0313  0.5550   0.5792   1.5860
## t.btystdvariance     -0.0167     0.0235 -0.7090   0.4787   1.3595
\end{verbatim}

\rm

\hypertarget{problem-3d}{%
\subsection{Problem 3(d)}\label{problem-3d}}

On the basis of this table, and what you know about the definitions of
the variables, would you eliminate any variables in your model? Why or
why not?

\em

\begin{itemize}

\item Since none of them are individually significant, all have moderately large vif's, and they are approximately summarized by \verb|btystdave|, I would eliminate the six students' ratings \verb|t.beautyf2upper, t.beautyflowerdiv, t.beautyfupperdiv, t.beautym2upper, t.beautymlowerdiv| and \verb|t.beautymupperdiv|.
\item Even though \verb|t.profevaluation| is highly significant with a low vif, I would probably remove it.  In my experience, evalation of the instructor and evaluation of the course are highly correlated, and so this variable is probably soaking up variation in course evaluation that I'd like to see explained by other variables in the model.
\item The variables \verb|t.didevaluation,  t.percentevaluating|, and  \verb|t.students| are interesting: In untransformed form there is  clear (nonlinear) relationship among these variables: \texttt{percentevaluating = 100*didevaluation/students}.  Since \verb|t.percentevaluating| has almost no practical effect on course evaluation, I will eliminate that one, and see what the effect is.
\item I would not get rid of \verb|btystave| even though it has a sky-high vif and is only marginally significant: I'd hope that removing the other variables above would allow us to see the effect of \verb|btystdave| on course evaluation.
\end{itemize}

In a ``first pass'', this is all the farther I'd go; removing these
variables is going to strongly affect the \(t\) statistics and the vifs.
If I refit the model without these variables, I get

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lm}\FloatTok{.2} \OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lm}\FloatTok{.1}\NormalTok{, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ t.beautyf2upper }\SpecialCharTok{{-}}\NormalTok{ t.beautyflowerdiv }\SpecialCharTok{{-}}\NormalTok{ t.beautyfupperdiv }\SpecialCharTok{{-}}
\NormalTok{                 t.beautym2upper }\SpecialCharTok{{-}}\NormalTok{ t.beautymlowerdiv }\SpecialCharTok{{-}}\NormalTok{ t.beautymupperdiv }\SpecialCharTok{{-}} 
\NormalTok{                 t.profevaluation }\SpecialCharTok{{-}}\NormalTok{ t.percentevaluating)}
\NormalTok{tab }\OtherTok{\textless{}{-}} \FunctionTok{cbind}\NormalTok{(}\FunctionTok{summary}\NormalTok{(lm}\FloatTok{.2}\NormalTok{)}\SpecialCharTok{$}\NormalTok{coef, }\AttributeTok{vif=}\FunctionTok{c}\NormalTok{(}\ConstantTok{NA}\NormalTok{,}\FunctionTok{vif}\NormalTok{(lm}\FloatTok{.2}\NormalTok{)))}

\FunctionTok{round}\NormalTok{(tab,}\DecValTok{4}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                  Estimate Std. Error t value Pr(>|t|)    vif
## (Intercept)        4.1781     0.2056 20.3167   0.0000     NA
## tenured            0.0702     0.0735  0.9552   0.3400 2.5303
## minority          -0.1584     0.0780 -2.0313   0.0428 1.3673
## age               -0.0073     0.0032 -2.2880   0.0226 1.8399
## btystdave          0.0915     0.0338  2.7080   0.0070 1.3384
## t.didevaluation   -1.5430     0.8365 -1.8446   0.0658 7.0660
## female            -0.1825     0.0528 -3.4591   0.0006 1.2819
## formal             0.1426     0.0691  2.0635   0.0396 1.2507
## fulldept           0.1997     0.0842  2.3717   0.0181 1.2669
## lower              0.0181     0.0568  0.3189   0.7500 1.3633
## nonenglish        -0.3116     0.1110 -2.8065   0.0052 1.3229
## onecredit          0.5339     0.1155  4.6245   0.0000 1.3827
## t.students         2.6575     0.9420  2.8210   0.0050 7.1209
## tenuretrack       -0.1461     0.0818 -1.7874   0.0746 2.1684
## blkandwhite        0.2023     0.0717  2.8225   0.0050 1.3589
## t.btystdvariance  -0.0437     0.0528 -0.8281   0.4081 1.1185
\end{verbatim}

Things are starting to come into focus:

\begin{itemize}\renewcommand{\labelitemii}{$\circ$}
\item \verb|bytstdave| is a significant predictor now
\item Most of the predictors are significant or very nearly significant; the exceptions are \verb|tenured,  lower| and \verb| t.btystdvariance|.  (I would probably try to remove these in a further round of variable selection\ldots)
\item The variables \verb|t.didevaluation| and \verb|t.students| still have somewhat high vifs, but both are significant or nearly so.  I might try to take one of these out (I think they are just collinear with each other: their correlation is  0.92), or I might try to leave them in and see if the rather large effects they have are ``real''.
\item The other variables have coefficients with signs that we might expect:
\begin{itemize}
\item minority instructors have somewhat lower course evaluations ($\hat\beta_{minority}=-0.1584$)
\item each year older an instructor is, the course evaluation lowers a bit ($\hat\beta_{age}=-0.0073$)
\item increasing beauty rating has a somewhat positive effect on course rating ($\hat\beta_{btystdave}=-0.0915$)
\item keeping in mind that \verb|t.didevaluation = 1/sqrt(didevaluation)|, the \underline{fewer} students that evaluate the class, the lower the course rating ($\hat\beta_{t.didevaluation}=-1.5430$)
\item similarly, since \verb|t.students = 1/sqrt(students)|, the \underline{fewer} students in the class, the higher the course rating ($\hat\beta_{t.students}=2.6575$)
\item female instructors take a hit a bit larger than minority instructors in course evaluation ($\hat\beta_{female}=-0.1825$)
\item If the instructor dresses formally for their web photo, or if the department faculty all have web photos, course evaluation increases enough to essentially offset the effect of being female or minority ($\hat\beta_{formal}=0.1426$, $\hat\beta_{fulldept}=0.1997$)
\item If the instructor is not a native speaker of English, the course evaluation takes a hit that's about twice as big as the minority instructor hit ($\hat\beta_{nonenglish}=-0.3116$)
\item One credit course get over a half-point advantage in course evaluation ($\hat\beta_{onecredit}=0.5339$)
\item We have no idea what \verb|blkandwhite| is, but it has a positive effect on course evaluation! ($\hat\beta_{onecredit}=0.2023$)
\end{itemize}
Of course, all the interpretations above sound like we are saying each predictor ``causes'' a change in course evaluation, but we really don't know about lurking variables, common causes, etc., so we can't be sure of ``causes'' here.
\end{itemize}

\rm

\hypertarget{problem-3e}{%
\subsection{Problem 3(e)}\label{problem-3e}}

Why might the methods used in parts (c) and (d) not be adequate for
deciding which variables to keep, and which ones to eliminate, in a
regression model?

\em

Essentially, the question is, ``what could go wrong with selecting
variables on the basis of \(t\)-statistics and vif's?''

There are a few potential difficulties with this approach:

\begin{enumerate}\renewcommand{\labelenumi}{\arabic{enumi}.}
\item Removing variables one-at-a-time can change other coefficients and $t$-statistics, so if you remove the variables in one order based on $t$-statistics, and I remove them in another order, we could end up with very different models.
\item High vif's do not mean that the model is invalid; they just mean that the estimated coefficients will be hard to interpret.  Removing variables on the basis of vif's can remove important predictor variables from a model
\item One-variable-at-a-time approaches are examples of ``greedy'' algorithms: since they do not consider all variables at each step of variable selection, they can miss the very best models (forward selection with BIC in Problem \#2 is an example of this).
\end{enumerate}

In the case of this problem, these dificulties are offset by ``subject
matter knowledge'': since we know a lot about college or university
life, we can make better guesses about what order to remove variables
in, what groups of variables to remove all at once, etc.

Just for kicks, I tried ``all subsets'' variable selection on the model
fitted in part (c) here (except that I took \verb|profevaluation| out,
because of its close relationship with \verb|courseevaluation|). This
approach uses no subject matter knowledge (none of our knowledge about
university life!), and replaces it with just mathematical optimality:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{all.subsets }\OtherTok{\textless{}{-}} \FunctionTok{regsubsets}\NormalTok{(courseevaluation }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ t.profevaluation, }\AttributeTok{data=}\NormalTok{beauty.trans,}
                          \AttributeTok{nvmax=}\FunctionTok{dim}\NormalTok{(beauty.trans)}\SpecialCharTok{{-}}\DecValTok{1}\NormalTok{)  }\CommentTok{\# {-}1 for t.profevaluation}
\NormalTok{tmp }\OtherTok{\textless{}{-}} \FunctionTok{summary}\NormalTok{(all.subsets)}
\FunctionTok{attach}\NormalTok{(tmp)}
\NormalTok{p }\OtherTok{\textless{}{-}} \DecValTok{1}\SpecialCharTok{:}\FunctionTok{dim}\NormalTok{(tmp}\SpecialCharTok{$}\NormalTok{which)[}\DecValTok{1}\NormalTok{]}
\NormalTok{n }\OtherTok{\textless{}{-}} \FunctionTok{dim}\NormalTok{(beauty.trans)[}\DecValTok{1}\NormalTok{]}
\NormalTok{results }\OtherTok{\textless{}{-}} \FunctionTok{data.frame}\NormalTok{(which, }\AttributeTok{bic=}\NormalTok{n}\SpecialCharTok{*}\FunctionTok{log}\NormalTok{(rss)}\SpecialCharTok{+}\FunctionTok{log}\NormalTok{(n)}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{2}\NormalTok{), }\AttributeTok{aic=}\NormalTok{n}\SpecialCharTok{*}\FunctionTok{log}\NormalTok{(rss)}\SpecialCharTok{+}\DecValTok{2}\SpecialCharTok{*}\NormalTok{(p}\SpecialCharTok{+}\DecValTok{2}\NormalTok{))}
\FunctionTok{detach}\NormalTok{()}
\end{Highlighting}
\end{Shaded}

\tiny

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{oldwidth }\OtherTok{\textless{}{-}} \FunctionTok{options}\NormalTok{()}\SpecialCharTok{$}\NormalTok{width}
\FunctionTok{options}\NormalTok{(}\AttributeTok{width=}\DecValTok{200}\NormalTok{)}
\NormalTok{results}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##    X.Intercept. tenured minority   age t.beautyf2upper t.beautyflowerdiv t.beautyfupperdiv t.beautym2upper t.beautymlowerdiv t.beautymupperdiv btystdave t.didevaluation female formal fulldept lower
## 1          TRUE   FALSE    FALSE FALSE           FALSE             FALSE             FALSE           FALSE             FALSE             FALSE     FALSE           FALSE  FALSE  FALSE    FALSE FALSE
## 2          TRUE   FALSE    FALSE FALSE           FALSE             FALSE              TRUE           FALSE             FALSE             FALSE     FALSE           FALSE  FALSE  FALSE    FALSE FALSE
## 3          TRUE   FALSE    FALSE FALSE           FALSE             FALSE              TRUE           FALSE             FALSE             FALSE     FALSE           FALSE   TRUE  FALSE    FALSE FALSE
## 4          TRUE   FALSE    FALSE FALSE           FALSE             FALSE              TRUE           FALSE             FALSE             FALSE     FALSE           FALSE   TRUE  FALSE    FALSE FALSE
## 5          TRUE   FALSE     TRUE FALSE           FALSE             FALSE              TRUE           FALSE             FALSE             FALSE     FALSE           FALSE   TRUE  FALSE    FALSE FALSE
## 6          TRUE   FALSE    FALSE FALSE           FALSE             FALSE              TRUE           FALSE             FALSE             FALSE     FALSE           FALSE   TRUE  FALSE    FALSE FALSE
## 7          TRUE   FALSE     TRUE FALSE           FALSE             FALSE              TRUE           FALSE             FALSE             FALSE     FALSE           FALSE   TRUE  FALSE    FALSE FALSE
## 8          TRUE   FALSE     TRUE FALSE           FALSE             FALSE              TRUE            TRUE             FALSE             FALSE     FALSE           FALSE   TRUE  FALSE    FALSE FALSE
## 9          TRUE   FALSE     TRUE FALSE            TRUE             FALSE              TRUE            TRUE             FALSE             FALSE     FALSE           FALSE   TRUE  FALSE    FALSE FALSE
## 10         TRUE   FALSE     TRUE FALSE            TRUE             FALSE              TRUE            TRUE             FALSE             FALSE     FALSE            TRUE   TRUE  FALSE    FALSE FALSE
## 11         TRUE   FALSE     TRUE FALSE            TRUE             FALSE              TRUE            TRUE             FALSE             FALSE     FALSE            TRUE   TRUE  FALSE    FALSE FALSE
## 12         TRUE   FALSE     TRUE FALSE            TRUE             FALSE              TRUE            TRUE             FALSE             FALSE     FALSE            TRUE   TRUE  FALSE     TRUE FALSE
## 13         TRUE    TRUE     TRUE  TRUE            TRUE             FALSE              TRUE            TRUE             FALSE             FALSE     FALSE            TRUE   TRUE  FALSE    FALSE FALSE
## 14         TRUE    TRUE     TRUE  TRUE            TRUE             FALSE              TRUE            TRUE             FALSE             FALSE     FALSE            TRUE   TRUE   TRUE    FALSE FALSE
## 15         TRUE    TRUE     TRUE  TRUE            TRUE              TRUE              TRUE           FALSE             FALSE             FALSE      TRUE            TRUE   TRUE   TRUE    FALSE FALSE
## 16         TRUE    TRUE     TRUE  TRUE            TRUE              TRUE              TRUE           FALSE             FALSE              TRUE      TRUE            TRUE   TRUE   TRUE    FALSE FALSE
## 17         TRUE    TRUE     TRUE  TRUE            TRUE              TRUE              TRUE           FALSE             FALSE              TRUE      TRUE            TRUE   TRUE   TRUE     TRUE FALSE
## 18         TRUE    TRUE     TRUE  TRUE            TRUE              TRUE              TRUE           FALSE             FALSE              TRUE      TRUE            TRUE   TRUE   TRUE     TRUE  TRUE
## 19         TRUE    TRUE     TRUE  TRUE            TRUE              TRUE              TRUE           FALSE              TRUE              TRUE      TRUE            TRUE   TRUE   TRUE     TRUE  TRUE
## 20         TRUE    TRUE     TRUE  TRUE            TRUE              TRUE              TRUE           FALSE              TRUE              TRUE      TRUE            TRUE   TRUE   TRUE     TRUE  TRUE
## 21         TRUE    TRUE     TRUE  TRUE            TRUE              TRUE              TRUE           FALSE              TRUE              TRUE      TRUE            TRUE   TRUE   TRUE     TRUE  TRUE
## 22         TRUE    TRUE     TRUE  TRUE            TRUE              TRUE              TRUE            TRUE              TRUE              TRUE      TRUE            TRUE   TRUE   TRUE     TRUE  TRUE
##    nonenglish onecredit t.percentevaluating t.students tenuretrack blkandwhite t.btystdvariance      bic      aic
## 1       FALSE      TRUE               FALSE      FALSE       FALSE       FALSE            FALSE 2287.381 2274.968
## 2       FALSE      TRUE               FALSE      FALSE       FALSE       FALSE            FALSE 2268.701 2252.150
## 3       FALSE      TRUE               FALSE      FALSE       FALSE       FALSE            FALSE 2254.834 2234.145
## 4       FALSE      TRUE                TRUE      FALSE       FALSE       FALSE            FALSE 2245.415 2220.588
## 5       FALSE      TRUE                TRUE      FALSE       FALSE       FALSE            FALSE 2235.815 2206.851
## 6        TRUE      TRUE                TRUE      FALSE       FALSE        TRUE            FALSE 2229.402 2196.300
## 7        TRUE      TRUE                TRUE      FALSE       FALSE        TRUE            FALSE 2229.766 2192.526
## 8        TRUE      TRUE                TRUE      FALSE       FALSE        TRUE            FALSE 2233.035 2191.658
## 9        TRUE      TRUE                TRUE      FALSE       FALSE        TRUE            FALSE 2236.075 2190.560
## 10       TRUE      TRUE                TRUE      FALSE       FALSE        TRUE            FALSE 2238.882 2189.229
## 11       TRUE      TRUE                TRUE      FALSE        TRUE        TRUE            FALSE 2243.019 2189.229
## 12       TRUE      TRUE                TRUE      FALSE        TRUE        TRUE            FALSE 2247.324 2189.396
## 13       TRUE      TRUE                TRUE      FALSE        TRUE        TRUE            FALSE 2251.593 2189.528
## 14       TRUE      TRUE                TRUE      FALSE        TRUE        TRUE            FALSE 2255.547 2189.344
## 15       TRUE      TRUE                TRUE      FALSE        TRUE        TRUE            FALSE 2259.621 2189.280
## 16       TRUE      TRUE                TRUE      FALSE        TRUE        TRUE            FALSE 2264.658 2190.179
## 17       TRUE      TRUE                TRUE      FALSE        TRUE        TRUE            FALSE 2270.045 2191.428
## 18       TRUE      TRUE                TRUE      FALSE        TRUE        TRUE            FALSE 2275.607 2192.852
## 19       TRUE      TRUE                TRUE      FALSE        TRUE        TRUE            FALSE 2281.385 2194.493
## 20       TRUE      TRUE                TRUE       TRUE        TRUE        TRUE            FALSE 2287.338 2196.308
## 21       TRUE      TRUE                TRUE       TRUE        TRUE        TRUE             TRUE 2293.425 2198.257
## 22       TRUE      TRUE                TRUE       TRUE        TRUE        TRUE             TRUE 2299.528 2200.222
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{minimize}\NormalTok{(results,}\StringTok{"bic"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##   X.Intercept. tenured minority   age t.beautyf2upper t.beautyflowerdiv t.beautyfupperdiv t.beautym2upper t.beautymlowerdiv t.beautymupperdiv btystdave t.didevaluation female formal fulldept lower
## 6         TRUE   FALSE    FALSE FALSE           FALSE             FALSE              TRUE           FALSE             FALSE             FALSE     FALSE           FALSE   TRUE  FALSE    FALSE FALSE
##   nonenglish onecredit t.percentevaluating t.students tenuretrack blkandwhite t.btystdvariance      bic    aic
## 6       TRUE      TRUE                TRUE      FALSE       FALSE        TRUE            FALSE 2229.402 2196.3
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{minimize}\NormalTok{(results,}\StringTok{"aic"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##    X.Intercept. tenured minority   age t.beautyf2upper t.beautyflowerdiv t.beautyfupperdiv t.beautym2upper t.beautymlowerdiv t.beautymupperdiv btystdave t.didevaluation female formal fulldept lower
## 11         TRUE   FALSE     TRUE FALSE            TRUE             FALSE              TRUE            TRUE             FALSE             FALSE     FALSE            TRUE   TRUE  FALSE    FALSE FALSE
##    nonenglish onecredit t.percentevaluating t.students tenuretrack blkandwhite t.btystdvariance      bic      aic
## 11       TRUE      TRUE                TRUE      FALSE        TRUE        TRUE            FALSE 2243.019 2189.229
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{options}\NormalTok{(}\AttributeTok{width=}\NormalTok{oldwidth)}
\end{Highlighting}
\end{Shaded}

\normalsize

The best BIC model here is

\begin{verbatim}
courseevaluation ~ 1 +  t.beautyfupperdiv +  female + nonenglish + onecredit +
    t.percentevaluating + blkandwhite
\end{verbatim}

and the best AIC model is

\begin{verbatim}
courseevaluation ~ 1 + minority + t.beautyf2upper + t.beautyfupperdiv + t.beautym2upper +
     t.didevaluation + female + nonenglish + onecredit + t.percentevaluating +
     blkandwhite
\end{verbatim}

This turns out to be a nice illustration of how mathematical optimality
may not be what makes the most sense substantively:

\begin{itemize}
\item Each model tries to make a new ``beauty'' variable instead of taking \verb|btystdave|.  There isn't really much substantive sense to the beauty variables that the models pick out.
\item The other variables in these models are a subset of the model from part (d).  They are fine, but they also seem to be missing significant predictors with intepretable coefficients.  
\end{itemize}

If I were advising the university about what (besides teaching quality)
affects course evaluation, I would not want to miss some of the
variables that the optimal AIC and BIC models miss! \rm

\end{document}
