% Options for packages loaded elsewhere
\PassOptionsToPackage{unicode}{hyperref}
\PassOptionsToPackage{hyphens}{url}
%
\documentclass[
]{article}
\usepackage{amsmath,amssymb}
\usepackage{lmodern}
\usepackage{iftex}
\ifPDFTeX
  \usepackage[T1]{fontenc}
  \usepackage[utf8]{inputenc}
  \usepackage{textcomp} % provide euro and other symbols
\else % if luatex or xetex
  \usepackage{unicode-math}
  \defaultfontfeatures{Scale=MatchLowercase}
  \defaultfontfeatures[\rmfamily]{Ligatures=TeX,Scale=1}
\fi
% Use upquote if available, for straight quotes in verbatim environments
\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
\IfFileExists{microtype.sty}{% use microtype if available
  \usepackage[]{microtype}
  \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
}{}
\makeatletter
\@ifundefined{KOMAClassName}{% if non-KOMA class
  \IfFileExists{parskip.sty}{%
    \usepackage{parskip}
  }{% else
    \setlength{\parindent}{0pt}
    \setlength{\parskip}{6pt plus 2pt minus 1pt}}
}{% if KOMA class
  \KOMAoptions{parskip=half}}
\makeatother
\usepackage{xcolor}
\usepackage[margin=1in]{geometry}
\usepackage{color}
\usepackage{fancyvrb}
\newcommand{\VerbBar}{|}
\newcommand{\VERB}{\Verb[commandchars=\\\{\}]}
\DefineVerbatimEnvironment{Highlighting}{Verbatim}{commandchars=\\\{\}}
% Add ',fontsize=\small' for more characters per line
\usepackage{framed}
\definecolor{shadecolor}{RGB}{248,248,248}
\newenvironment{Shaded}{\begin{snugshade}}{\end{snugshade}}
\newcommand{\AlertTok}[1]{\textcolor[rgb]{0.94,0.16,0.16}{#1}}
\newcommand{\AnnotationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\AttributeTok}[1]{\textcolor[rgb]{0.77,0.63,0.00}{#1}}
\newcommand{\BaseNTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}}
\newcommand{\BuiltInTok}[1]{#1}
\newcommand{\CharTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\CommentTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textit{#1}}}
\newcommand{\CommentVarTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\ConstantTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\ControlFlowTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{\textbf{#1}}}
\newcommand{\DataTypeTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{#1}}
\newcommand{\DecValTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}}
\newcommand{\DocumentationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\ErrorTok}[1]{\textcolor[rgb]{0.64,0.00,0.00}{\textbf{#1}}}
\newcommand{\ExtensionTok}[1]{#1}
\newcommand{\FloatTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}}
\newcommand{\FunctionTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\ImportTok}[1]{#1}
\newcommand{\InformationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\KeywordTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{\textbf{#1}}}
\newcommand{\NormalTok}[1]{#1}
\newcommand{\OperatorTok}[1]{\textcolor[rgb]{0.81,0.36,0.00}{\textbf{#1}}}
\newcommand{\OtherTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{#1}}
\newcommand{\PreprocessorTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textit{#1}}}
\newcommand{\RegionMarkerTok}[1]{#1}
\newcommand{\SpecialCharTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\SpecialStringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\StringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\VariableTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\VerbatimStringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\WarningTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\usepackage{graphicx}
\makeatletter
\def\maxwidth{\ifdim\Gin@nat@width>\linewidth\linewidth\else\Gin@nat@width\fi}
\def\maxheight{\ifdim\Gin@nat@height>\textheight\textheight\else\Gin@nat@height\fi}
\makeatother
% Scale images if necessary, so that they will not overflow the page
% margins by default, and it is still possible to overwrite the defaults
% using explicit options in \includegraphics[width, height, ...]{}
\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio}
% Set default figure placement to htbp
\makeatletter
\def\fps@figure{htbp}
\makeatother
\setlength{\emergencystretch}{3em} % prevent overfull lines
\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
\setcounter{secnumdepth}{-\maxdimen} % remove section numbering
\usepackage{longtable}
\usepackage{booktabs}
\usepackage{dcolumn}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{hyperref}
\usepackage{xcolor}
\DeclareMathOperator{\Cov}{Cov}
\DeclareMathOperator{\Corr}{Corr}
\DeclareMathOperator{\Var}{Var}
\DeclareMathOperator{\upmodels}{\perp \!\!\! \perp}
\ifLuaTeX
  \usepackage{selnolig}  % disable illegal ligatures
\fi
\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}
\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available
\urlstyle{same} % disable monospaced font for URLs
\hypersetup{
  pdftitle={Homework 09 Solutions},
  hidelinks,
  pdfcreator={LaTeX via pandoc}}

\title{Homework 09 Solutions}
\author{}
\date{\vspace{-2.5em}2022-11-15}

\begin{document}
\maketitle

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{library}\NormalTok{(arm)}
\FunctionTok{library}\NormalTok{(ggplot2)}
\FunctionTok{library}\NormalTok{(HLMdiag)    }\DocumentationTok{\#\# easy extraction of MLM residuals}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Warning: package 'HLMdiag' was built under R version 4.1.2
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{library}\NormalTok{(gridExtra)  }\DocumentationTok{\#\# For the grid.arrange() function}

\FunctionTok{library}\NormalTok{(texreg)  }\DocumentationTok{\#\# for pretty regression tables}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Warning: package 'texreg' was built under R version 4.1.3
\end{verbatim}

\def\red#1{\textcolor{red}{#1}}
\def\blue#1{\textcolor{blue}{#1}}
\let\bar\overline
\let\ul\underline

\def\theenumi{\roman{enumi}}

\hypertarget{problem-1.}{%
\section{Problem 1.}\label{problem-1.}}

Explore the data, looking for usual or unusual relationships, usual or
unusual data values, etc. Perform any data cleaning, transformations,
etc. that may be needed; justify your actions. Submit a brief but
interesting summary, with justifications.

\it

This summary is a bit long, but hopefully it is still clear\ldots{}
\label{p:1}

I begin by reading in the data\ldots

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{data }\OtherTok{\textless{}{-}} \FunctionTok{read.csv}\NormalTok{(}\StringTok{"ratings.csv"}\NormalTok{,}\AttributeTok{header=}\NormalTok{T)}

\FunctionTok{str}\NormalTok{(data)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 'data.frame':    2520 obs. of  28 variables:
##  $ X                           : int  1 2 3 4 5 6 7 8 9 10 ...
##  $ Subject                     : chr  "15" "15" "15" "15" ...
##  $ Harmony                     : chr  "I-IV-V" "I-IV-V" "I-IV-V" "I-IV-V" ...
##  $ Instrument                  : chr  "guitar" "guitar" "guitar" "piano" ...
##  $ Voice                       : chr  "contrary" "par3rd" "par5th" "contrary" ...
##  $ Selfdeclare                 : int  5 5 5 5 5 5 5 5 5 5 ...
##  $ OMSI                        : int  734 734 734 734 734 734 734 734 734 734 ...
##  $ X16.minus.17                : num  5 5 5 5 5 5 5 5 5 5 ...
##  $ ConsInstr                   : num  4.33 4.33 4.33 4.33 4.33 4.33 4.33 4.33 4.33 4.33 ...
##  $ ConsNotes                   : int  5 5 5 5 5 5 5 5 5 5 ...
##  $ Instr.minus.Notes           : num  -0.67 -0.67 -0.67 -0.67 -0.67 -0.67 -0.67 -0.67 -0.67 -0.67 ...
##  $ PachListen                  : int  5 5 5 5 5 5 5 5 5 5 ...
##  $ ClsListen                   : int  4 4 4 4 4 4 4 4 4 4 ...
##  $ KnowRob                     : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ KnowAxis                    : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ X1990s2000s                 : int  5 5 5 5 5 5 5 5 5 5 ...
##  $ X1990s2000s.minus.1960s1970s: int  2 2 2 2 2 2 2 2 2 2 ...
##  $ CollegeMusic                : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ NoClass                     : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ APTheory                    : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ Composing                   : int  4 4 4 4 4 4 4 4 4 4 ...
##  $ PianoPlay                   : int  1 1 1 1 1 1 1 1 1 1 ...
##  $ GuitarPlay                  : int  5 5 5 5 5 5 5 5 5 5 ...
##  $ X1stInstr                   : int  4 4 4 4 4 4 4 4 4 4 ...
##  $ X2ndInstr                   : int  NA NA NA NA NA NA NA NA NA NA ...
##  $ first12                     : chr  "string" "string" "string" "string" ...
##  $ Classical                   : num  3 3 1 3 2 8 10 6 5 1 ...
##  $ Popular                     : num  9 7 8 7 8 3 1 4 5 8 ...
\end{verbatim}

The following tables show that the researchers really did produce a
balanced design:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(data}\SpecialCharTok{$}\NormalTok{Harmony)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## I-IV-V I-V-IV I-V-VI IV-I-V 
##    630    630    630    630
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(data}\SpecialCharTok{$}\NormalTok{Instrument)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## guitar  piano string 
##    840    840    840
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(data}\SpecialCharTok{$}\NormalTok{Voice)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## contrary   par3rd   par5th 
##      840      840      840
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{with}\NormalTok{(data, }\FunctionTok{table}\NormalTok{(}\FunctionTok{paste}\NormalTok{(Harmony,Voice,Instrument)))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## I-IV-V contrary guitar  I-IV-V contrary piano I-IV-V contrary string 
##                     70                     70                     70 
##   I-IV-V par3rd guitar    I-IV-V par3rd piano   I-IV-V par3rd string 
##                     70                     70                     70 
##   I-IV-V par5th guitar    I-IV-V par5th piano   I-IV-V par5th string 
##                     70                     70                     70 
## I-V-IV contrary guitar  I-V-IV contrary piano I-V-IV contrary string 
##                     70                     70                     70 
##   I-V-IV par3rd guitar    I-V-IV par3rd piano   I-V-IV par3rd string 
##                     70                     70                     70 
##   I-V-IV par5th guitar    I-V-IV par5th piano   I-V-IV par5th string 
##                     70                     70                     70 
## I-V-VI contrary guitar  I-V-VI contrary piano I-V-VI contrary string 
##                     70                     70                     70 
##   I-V-VI par3rd guitar    I-V-VI par3rd piano   I-V-VI par3rd string 
##                     70                     70                     70 
##   I-V-VI par5th guitar    I-V-VI par5th piano   I-V-VI par5th string 
##                     70                     70                     70 
## IV-I-V contrary guitar  IV-I-V contrary piano IV-I-V contrary string 
##                     70                     70                     70 
##   IV-I-V par3rd guitar    IV-I-V par3rd piano   IV-I-V par3rd string 
##                     70                     70                     70 
##   IV-I-V par5th guitar    IV-I-V par5th piano   IV-I-V par5th string 
##                     70                     70                     70
\end{verbatim}

While exploring the distribution of the response variables we find that,
although they are supposed to be integer ratings from 1 to 10, in fact
we find some unusual values for them:

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\#}
\DocumentationTok{\#\# Counts of Classical responses:}
\FunctionTok{rbind}\NormalTok{(}\AttributeTok{count=}\FunctionTok{table}\NormalTok{(data}\SpecialCharTok{$}\NormalTok{Classical))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##       0   1   2   3 3.5   4 4.2 4.6   5   6   7   8   9 9.5  10 19
## count 8 111 231 254   1 239   1   1 297 266 334 305 203   1 240  1
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\#}
\DocumentationTok{\#\# Counts of Popular responses:}
\FunctionTok{rbind}\NormalTok{(}\AttributeTok{count=}\FunctionTok{table}\NormalTok{(data}\SpecialCharTok{$}\NormalTok{Popular))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##        0   1   2   3 3.5   4 4.2 4.6   5   6 6.8   7   8   9  10 19
## count 25 171 197 218   1 282   1   1 371 314   1 350 312 136 112  1
\end{verbatim}

We see

\begin{itemize}
\item A Classical rating of 0 was given 8 times, and a Popular rating of 0 was given 25 times
\item There are single ratings of 3.5, 4.2,4.6, 6.8, 9.5 and 19 in each response variable
\end{itemize}

I decided not to recode the ratings of 0 or the non-integer ratings
between 1 and 10. It is reasonable to guess that they reflect the
intended responses of listeners who may not have entirely understood the
``rate 1 to 10'' instructions.

I will recode the 19's to 10's, for the following reasons:

\begin{itemize}
\item The Classical rating is paired with a modest Popular rating, and vice versa, so it's likely that the respondents intended to respond with 10's for these ratings:
\end{itemize}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# The code is a little complicated because of the way R treats NA\textquotesingle{}s in expressions. }
\DocumentationTok{\#\#}
\DocumentationTok{\#\# Basically, if any part of an expression evaluates to NA, then the whole expression}
\DocumentationTok{\#\# evaluates to NA.  In the code below, I first filter out the NA\textquotesingle{}s with}
\DocumentationTok{\#\# !is.na(data$Classical) for example, and then test the condition that I\textquotesingle{}m }
\DocumentationTok{\#\# interested in (data$Classical=="19") on the remaining rows.  Otherwise I would }
\DocumentationTok{\#\# get a whole set of rows with NA\textquotesingle{}s instead of selecting the row with the "19" in it.}
\DocumentationTok{\#\#}
\DocumentationTok{\#\# Similarly for the "Popular" code.}

\DocumentationTok{\#\#}
\DocumentationTok{\#\# The Classical rating of 19:}
\NormalTok{c19 }\OtherTok{\textless{}{-}}\NormalTok{data}\SpecialCharTok{$}\NormalTok{Subject[}\SpecialCharTok{!}\FunctionTok{is.na}\NormalTok{(data}\SpecialCharTok{$}\NormalTok{Classical) }\SpecialCharTok{\&}\NormalTok{ data}\SpecialCharTok{$}\NormalTok{Classical}\SpecialCharTok{==}\StringTok{"19"}\NormalTok{]}
\NormalTok{x }\OtherTok{\textless{}{-}}\NormalTok{ data[data}\SpecialCharTok{$}\NormalTok{Subject}\SpecialCharTok{==}\NormalTok{c19,}\FunctionTok{c}\NormalTok{(}\StringTok{"Subject"}\NormalTok{, }\StringTok{"Classical"}\NormalTok{,}\StringTok{"Popular"}\NormalTok{)]}
\NormalTok{x[}\SpecialCharTok{!}\FunctionTok{is.na}\NormalTok{(x}\SpecialCharTok{$}\NormalTok{Classical) }\SpecialCharTok{\&}\NormalTok{ x}\SpecialCharTok{$}\NormalTok{Classical}\SpecialCharTok{==}\StringTok{"19"}\NormalTok{,]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##      Subject Classical Popular
## 1978      73        19       6
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\#}
\DocumentationTok{\#\# The Popular rating of 19:}
\NormalTok{p19 }\OtherTok{\textless{}{-}}\NormalTok{ data}\SpecialCharTok{$}\NormalTok{Subject[}\SpecialCharTok{!}\FunctionTok{is.na}\NormalTok{(data}\SpecialCharTok{$}\NormalTok{Popular) }\SpecialCharTok{\&}\NormalTok{ data}\SpecialCharTok{$}\NormalTok{Popular}\SpecialCharTok{==}\StringTok{"19"}\NormalTok{]}
\NormalTok{x }\OtherTok{\textless{}{-}}\NormalTok{ data[data}\SpecialCharTok{$}\NormalTok{Subject}\SpecialCharTok{==}\NormalTok{p19,}\FunctionTok{c}\NormalTok{(}\StringTok{"Subject"}\NormalTok{, }\StringTok{"Classical"}\NormalTok{,}\StringTok{"Popular"}\NormalTok{)]}
\NormalTok{x[}\SpecialCharTok{!}\FunctionTok{is.na}\NormalTok{(x}\SpecialCharTok{$}\NormalTok{Popular) }\SpecialCharTok{\&}\NormalTok{ x}\SpecialCharTok{$}\NormalTok{Popular}\SpecialCharTok{==}\StringTok{"19"}\NormalTok{,]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##      Subject Classical Popular
## 1166      47         5      19
\end{verbatim}

\begin{itemize}
\item The "9" key is next to the "0" key on a computer keyboard, so it seems likely that the 19's are mis-typed 10's.
\end{itemize}

Next we see that several variables have a significant portion of missing
values:

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\#}
\DocumentationTok{\#\# Count and Percent of Missing Values:}
\NormalTok{x }\OtherTok{\textless{}{-}} \FunctionTok{data.frame}\NormalTok{(}\AttributeTok{count=}\FunctionTok{sapply}\NormalTok{(data,}\ControlFlowTok{function}\NormalTok{(x) }\FunctionTok{sum}\NormalTok{(}\FunctionTok{is.na}\NormalTok{(x))),}
      \AttributeTok{percent=}\FunctionTok{sapply}\NormalTok{(data,}\ControlFlowTok{function}\NormalTok{(x) }\FunctionTok{round}\NormalTok{(}\DecValTok{100}\SpecialCharTok{*}\FunctionTok{mean}\NormalTok{(}\FunctionTok{is.na}\NormalTok{(x)),}\DecValTok{2}\NormalTok{)))}
\NormalTok{x[x}\SpecialCharTok{$}\NormalTok{percent}\SpecialCharTok{!=}\DecValTok{0}\NormalTok{,]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                              count percent
## ConsNotes                      360   14.29
## PachListen                      72    2.86
## ClsListen                       36    1.43
## KnowRob                        180    7.14
## KnowAxis                       288   11.43
## X1990s2000s                    144    5.71
## X1990s2000s.minus.1960s1970s   180    7.14
## CollegeMusic                   108    4.29
## NoClass                        288   11.43
## APTheory                       216    8.57
## Composing                       72    2.86
## X1stInstr                     1512   60.00
## X2ndInstr                     2196   87.14
## Classical                       27    1.07
## Popular                         27    1.07
\end{verbatim}

\begin{itemize}

\item The missing values in the response variables Classical and Popular cannot (and should not) be imputed, so I will delete those rows.  Further exploration (not shown here) shows that the same rows with missing Classical values are missing the Popular values, and vice versa.
\item The variables X1stInstr and X2ndInstr have 60\% and 80\% missing values, respectively, which makes them hard to use and hard to impute so I will delete those variables.
\end{itemize}

Taking into account all of the above, our full data set can be
constructed as follows

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{fulldata }\OtherTok{\textless{}{-}}\NormalTok{ data[,}\SpecialCharTok{!}\NormalTok{(}\FunctionTok{names}\NormalTok{(data) }\SpecialCharTok{\%in\%} \FunctionTok{c}\NormalTok{(}\StringTok{"X1stInstr"}\NormalTok{,}\StringTok{"X2ndInstr"}\NormalTok{))]}
\NormalTok{fulldata }\OtherTok{\textless{}{-}}\NormalTok{ fulldata[}\SpecialCharTok{!}\FunctionTok{is.na}\NormalTok{(fulldata}\SpecialCharTok{$}\NormalTok{Classical),]}
\NormalTok{fulldata }\OtherTok{\textless{}{-}}\NormalTok{ fulldata[}\SpecialCharTok{!}\FunctionTok{is.na}\NormalTok{(fulldata}\SpecialCharTok{$}\NormalTok{Popular),]}
\NormalTok{fulldata}\SpecialCharTok{$}\NormalTok{Classical[fulldata}\SpecialCharTok{$}\NormalTok{Classical}\SpecialCharTok{==}\StringTok{"19"}\NormalTok{]}\OtherTok{\textless{}{-}}\StringTok{"10"}
\NormalTok{fulldata}\SpecialCharTok{$}\NormalTok{Popular[fulldata}\SpecialCharTok{$}\NormalTok{Popular}\SpecialCharTok{==}\StringTok{"19"}\NormalTok{]}\OtherTok{\textless{}{-}}\StringTok{"10"}
\end{Highlighting}
\end{Shaded}

From the tables and counts below we see that we have lost 27 rows
(deleting rows with missing responses) and 2 columns (deleting the two
variables X1stInstr and X2ndInstr), and we no longer have a perfectly
balanced experiment.

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{dim}\NormalTok{(data)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 2520   28
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{dim}\NormalTok{(fulldata)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 2493   26
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(fulldata}\SpecialCharTok{$}\NormalTok{Harmony)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## I-IV-V I-V-IV I-V-VI IV-I-V 
##    625    622    622    624
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(fulldata}\SpecialCharTok{$}\NormalTok{Instrument)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## guitar  piano string 
##    833    820    840
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(fulldata}\SpecialCharTok{$}\NormalTok{Voice)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## contrary   par3rd   par5th 
##      831      830      832
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{with}\NormalTok{(fulldata, }\FunctionTok{table}\NormalTok{(}\FunctionTok{paste}\NormalTok{(Harmony,Voice,Instrument)))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## I-IV-V contrary guitar  I-IV-V contrary piano I-IV-V contrary string 
##                     70                     69                     70 
##   I-IV-V par3rd guitar    I-IV-V par3rd piano   I-IV-V par3rd string 
##                     69                     70                     70 
##   I-IV-V par5th guitar    I-IV-V par5th piano   I-IV-V par5th string 
##                     70                     67                     70 
## I-V-IV contrary guitar  I-V-IV contrary piano I-V-IV contrary string 
##                     69                     67                     70 
##   I-V-IV par3rd guitar    I-V-IV par3rd piano   I-V-IV par3rd string 
##                     70                     69                     70 
##   I-V-IV par5th guitar    I-V-IV par5th piano   I-V-IV par5th string 
##                     69                     68                     70 
## I-V-VI contrary guitar  I-V-VI contrary piano I-V-VI contrary string 
##                     69                     68                     70 
##   I-V-VI par3rd guitar    I-V-VI par3rd piano   I-V-VI par3rd string 
##                     69                     67                     70 
##   I-V-VI par5th guitar    I-V-VI par5th piano   I-V-VI par5th string 
##                     70                     69                     70 
## IV-I-V contrary guitar  IV-I-V contrary piano IV-I-V contrary string 
##                     70                     69                     70 
##   IV-I-V par3rd guitar    IV-I-V par3rd piano   IV-I-V par3rd string 
##                     69                     67                     70 
##   IV-I-V par5th guitar    IV-I-V par5th piano   IV-I-V par5th string 
##                     69                     70                     70
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(fulldata}\SpecialCharTok{$}\NormalTok{Classical)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##   0   1  10   2   3 3.5   4 4.2 4.6   5   6   7   8   9 9.5 
##   8 111 241 231 254   1 239   1   1 297 266 334 305 203   1
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(fulldata}\SpecialCharTok{$}\NormalTok{Popular)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##   0   1  10   2   3 3.5   4 4.2 4.6   5   6 6.8   7   8   9 
##  25 171 113 197 218   1 282   1   1 371 314   1 350 312 136
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{length}\NormalTok{(}\FunctionTok{unique}\NormalTok{(fulldata}\SpecialCharTok{$}\NormalTok{Subject))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 70
\end{verbatim}

The ``fulldata'' set still has many missing values. For model selection,
when I need to compare models on the same data set, it may be useful to
have a data set with no missing values. A simple approach is to
construct a reduced data set by removing rows with missing values:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{nomissdata }\OtherTok{\textless{}{-}}\NormalTok{ fulldata[}\FunctionTok{apply}\NormalTok{(fulldata,}\DecValTok{1}\NormalTok{,}\ControlFlowTok{function}\NormalTok{(x) }\SpecialCharTok{!}\FunctionTok{any}\NormalTok{(}\FunctionTok{is.na}\NormalTok{(x))),]}
\FunctionTok{dim}\NormalTok{(fulldata)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 2493   26
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{dim}\NormalTok{(nomissdata)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 1541   26
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata}\SpecialCharTok{$}\NormalTok{Harmony)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## I-IV-V I-V-IV I-V-VI IV-I-V 
##    386    385    384    386
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata}\SpecialCharTok{$}\NormalTok{Instrument)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## guitar  piano string 
##    515    510    516
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata}\SpecialCharTok{$}\NormalTok{Voice)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## contrary   par3rd   par5th 
##      513      514      514
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{length}\NormalTok{(}\FunctionTok{unique}\NormalTok{(nomissdata}\SpecialCharTok{$}\NormalTok{Subject))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] 43
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{with}\NormalTok{(nomissdata, }\FunctionTok{table}\NormalTok{(}\FunctionTok{paste}\NormalTok{(Harmony,Voice,Instrument)))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## I-IV-V contrary guitar  I-IV-V contrary piano I-IV-V contrary string 
##                     43                     43                     43 
##   I-IV-V par3rd guitar    I-IV-V par3rd piano   I-IV-V par3rd string 
##                     43                     43                     43 
##   I-IV-V par5th guitar    I-IV-V par5th piano   I-IV-V par5th string 
##                     43                     42                     43 
## I-V-IV contrary guitar  I-V-IV contrary piano I-V-IV contrary string 
##                     43                     42                     43 
##   I-V-IV par3rd guitar    I-V-IV par3rd piano   I-V-IV par3rd string 
##                     43                     43                     43 
##   I-V-IV par5th guitar    I-V-IV par5th piano   I-V-IV par5th string 
##                     43                     42                     43 
## I-V-VI contrary guitar  I-V-VI contrary piano I-V-VI contrary string 
##                     42                     42                     43 
##   I-V-VI par3rd guitar    I-V-VI par3rd piano   I-V-VI par3rd string 
##                     43                     42                     43 
##   I-V-VI par5th guitar    I-V-VI par5th piano   I-V-VI par5th string 
##                     43                     43                     43 
## IV-I-V contrary guitar  IV-I-V contrary piano IV-I-V contrary string 
##                     43                     43                     43 
##   IV-I-V par3rd guitar    IV-I-V par3rd piano   IV-I-V par3rd string 
##                     43                     42                     43 
##   IV-I-V par5th guitar    IV-I-V par5th piano   IV-I-V par5th string 
##                     43                     43                     43
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata}\SpecialCharTok{$}\NormalTok{Classical)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##   0   1  10   2   3   4   5   6   7   8   9 9.5 
##   8  93 147 154 163 158 176 147 195 175 124   1
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata}\SpecialCharTok{$}\NormalTok{Popular)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##   0   1  10   2   3 3.5   4   5   6   7   8   9 
##  23  99  88 126 135   1 156 210 191 224 195  93
\end{verbatim}

We have lost nearly 1000 observations and we are down to 43 subjects
from the original 70, but the data is still fairly balanced among the
experimental conditions, and most of the 43 subjects saw all 36
combinations of Harmony, Instrument and Voice.

\begin{itemize}
\item I will use ``fulldata'' for Problem \#2 below, since it only involves response variables and design variables, and there is no missingness in these variables in the ``fulldata'' data set.
\item I will begin with ``nomissdata'' for Problems \#3 and \#4 below but if it becomes obvious that some variables with missing values are not useful in variable selection, I may be able to delete those variables and have a somewhat larger data set for model comparisons.
\end{itemize}

Finally it is worthwhile to look at histograms of the variables (a) to
see if the distributions are similar in ``fulldata'' and ``nomissdata''
and (b) to check for unusual distributions of any kind.

In the plots on the next two pages, we see that

\begin{itemize}

\item The distributions of the numeric variables are broadly similar across the two data sets

\item Most variables do not seem to need transformations, either because they are really discrete/categorical variables (e.g. all the variables on a scale from 0 to 5) or they are not sufficiently skewed to warrant transformation.

\item Possible exceptions to the previous point are the first three variables plotted, Selfdeclare, OMSI and X16.minus.17.  Selfdeclare looks like another categorical variable so I would probably leave it alone.  
\begin{itemize}

\item  OMSI looks continuous and right-skewed so I could consider a log or similar transformation.  

\item X16.minus.17 could be shifted and then transformed.
\end{itemize}
but it's not clear how meaningful the transformed variables would be, and so I'm inclined to leave these two variables alone as well.

\end{itemize}

\clearpage

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{par}\NormalTok{(}\AttributeTok{mfrow=}\FunctionTok{c}\NormalTok{(}\DecValTok{5}\NormalTok{,}\DecValTok{4}\NormalTok{))}
\ControlFlowTok{for}\NormalTok{ (i }\ControlFlowTok{in} \FunctionTok{names}\NormalTok{(fulldata)[}\SpecialCharTok{{-}}\FunctionTok{c}\NormalTok{(}\DecValTok{1}\SpecialCharTok{:}\DecValTok{5}\NormalTok{,}\DecValTok{24}\NormalTok{)]) \{ }\DocumentationTok{\#\# select only the numeric variables}
  \FunctionTok{hist}\NormalTok{(}\FunctionTok{as.numeric}\NormalTok{(fulldata[,i]),}\AttributeTok{xlab=}\NormalTok{i,}\AttributeTok{main=}\StringTok{""}\NormalTok{,}\AttributeTok{cex=}\FloatTok{0.5}\NormalTok{)}
\NormalTok{\}}
\end{Highlighting}
\end{Shaded}

\includegraphics{hw09-solutions_files/figure-latex/unnamed-chunk-10-1.pdf}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{par}\NormalTok{(}\AttributeTok{mfrow=}\FunctionTok{c}\NormalTok{(}\DecValTok{5}\NormalTok{,}\DecValTok{4}\NormalTok{))}
\ControlFlowTok{for}\NormalTok{ (i }\ControlFlowTok{in} \FunctionTok{names}\NormalTok{(nomissdata)[}\SpecialCharTok{{-}}\FunctionTok{c}\NormalTok{(}\DecValTok{1}\SpecialCharTok{:}\DecValTok{5}\NormalTok{,}\DecValTok{24}\NormalTok{)]) \{  }\DocumentationTok{\#\# select only the numeric variables}
  \FunctionTok{hist}\NormalTok{(}\FunctionTok{as.numeric}\NormalTok{(nomissdata[,i]),}\AttributeTok{xlab=}\NormalTok{i,}\AttributeTok{main=}\StringTok{""}\NormalTok{,}\AttributeTok{cex=}\FloatTok{0.5}\NormalTok{)}
\NormalTok{\}}
\end{Highlighting}
\end{Shaded}

\includegraphics{hw09-solutions_files/figure-latex/unnamed-chunk-11-1.pdf}

\rm

In your work below, \texttt{Classical} should not appear in any models
for \texttt{Popular}, nor vice-versa. When evaluating models, please
remember to look at appropriate plots as well as test statistics.

\hypertarget{problem-2.}{%
\section{Problem 2.}\label{problem-2.}}

\underline{\textit{The three main experimental factors.}}

\hypertarget{problem-2a.}{%
\subsection{Problem 2(a).}\label{problem-2a.}}

Examine the influence of the three main experimental factors
(Instrument, Harmony \& Voice) on Classical ratings, using conventional
linear models and/or analysis of variance models.\\
Comment briefly on your findings, providing suitable brief evidence for
each result.
\textit{Hint: Since there are only three factors here, it is worth considering interactions of all orders.}
\label{p:2a}

\it

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# Build some basic models to check whether we need 3{-}way or 2{-}way interactions }

\NormalTok{lm.all.interactions }\OtherTok{\textless{}{-}} \FunctionTok{lm}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{*}\NormalTok{ Harmony }\SpecialCharTok{*}\NormalTok{ Voice, }\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lm.no.threeway }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lm.all.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ Instrument }\SpecialCharTok{:}\NormalTok{ Harmony }\SpecialCharTok{:}\NormalTok{ Voice)}

\NormalTok{lm.no.interactions }\OtherTok{\textless{}{-}} \FunctionTok{lm}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+}\NormalTok{ Voice, }\AttributeTok{data=}\NormalTok{fulldata)}

\DocumentationTok{\#\# Here\textquotesingle{}s the "full model" will all interactions.   Not too much is significant}

\FunctionTok{display}\NormalTok{(lm.all.interactions)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lm(formula = Classical ~ Instrument * Harmony * Voice, data = fulldata)
##                                            coef.est coef.se
## (Intercept)                                 4.49     0.27  
## Instrumentpiano                             1.01     0.39  
## Instrumentstring                            2.79     0.39  
## HarmonyI-V-IV                              -0.41     0.39  
## HarmonyI-V-VI                               1.05     0.39  
## HarmonyIV-I-V                              -0.43     0.39  
## Voicepar3rd                                -0.24     0.39  
## Voicepar5th                                -0.64     0.39  
## Instrumentpiano:HarmonyI-V-IV               0.82     0.55  
## Instrumentstring:HarmonyI-V-IV              0.88     0.55  
## Instrumentpiano:HarmonyI-V-VI               0.21     0.55  
## Instrumentstring:HarmonyI-V-VI              0.07     0.55  
## Instrumentpiano:HarmonyIV-I-V               0.46     0.55  
## Instrumentstring:HarmonyIV-I-V              0.30     0.55  
## Instrumentpiano:Voicepar3rd                -0.05     0.55  
## Instrumentstring:Voicepar3rd               -0.05     0.55  
## Instrumentpiano:Voicepar5th                 0.64     0.55  
## Instrumentstring:Voicepar5th                0.59     0.55  
## HarmonyI-V-IV:Voicepar3rd                   0.07     0.55  
## HarmonyI-V-VI:Voicepar3rd                  -1.41     0.55  
## HarmonyIV-I-V:Voicepar3rd                   0.52     0.55  
## HarmonyI-V-IV:Voicepar5th                   0.56     0.55  
## HarmonyI-V-VI:Voicepar5th                  -0.11     0.55  
## HarmonyIV-I-V:Voicepar5th                   0.77     0.55  
## Instrumentpiano:HarmonyI-V-IV:Voicepar3rd  -0.67     0.78  
## Instrumentstring:HarmonyI-V-IV:Voicepar3rd -0.62     0.77  
## Instrumentpiano:HarmonyI-V-VI:Voicepar3rd   0.94     0.78  
## Instrumentstring:HarmonyI-V-VI:Voicepar3rd  1.24     0.77  
## Instrumentpiano:HarmonyIV-I-V:Voicepar3rd  -0.22     0.78  
## Instrumentstring:HarmonyIV-I-V:Voicepar3rd  0.26     0.77  
## Instrumentpiano:HarmonyI-V-IV:Voicepar5th  -0.97     0.78  
## Instrumentstring:HarmonyI-V-IV:Voicepar5th -1.28     0.77  
## Instrumentpiano:HarmonyI-V-VI:Voicepar5th  -0.50     0.78  
## Instrumentstring:HarmonyI-V-VI:Voicepar5th -0.48     0.77  
## Instrumentpiano:HarmonyIV-I-V:Voicepar5th  -0.97     0.78  
## Instrumentstring:HarmonyIV-I-V:Voicepar5th -1.03     0.77  
## ---
## n = 2493, k = 36
## residual sd = 2.28, R-Squared = 0.27
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# Let\textquotesingle{}s compare the models with F tests...}

\FunctionTok{anova}\NormalTok{(}\FunctionTok{lm}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}} \DecValTok{1}\NormalTok{, }\AttributeTok{data=}\NormalTok{fulldata),lm.no.interactions,lm.no.threeway,lm.all.interactions)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Analysis of Variance Table
## 
## Model 1: Classical ~ 1
## Model 2: Classical ~ Instrument + Harmony + Voice
## Model 3: Classical ~ Instrument + Harmony + Voice + Instrument:Harmony + 
##     Instrument:Voice + Harmony:Voice
## Model 4: Classical ~ Instrument * Harmony * Voice
##   Res.Df   RSS Df Sum of Sq        F Pr(>F)    
## 1   2492 17438                                 
## 2   2485 12982  7    4456.1 122.0707 <2e-16 ***
## 3   2469 12875 16     107.0   1.2819 0.1992    
## 4   2457 12813 12      61.8   0.9869 0.4587    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# From this it seems like just the additive model with no interactions is best, }
\DocumentationTok{\#\# but let\textquotesingle{}s at least check the pairwise two{-}way interactions... }

\FunctionTok{anova}\NormalTok{(lm.no.interactions, }\FunctionTok{update}\NormalTok{(lm.no.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{:}\NormalTok{Harmony))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Analysis of Variance Table
## 
## Model 1: Classical ~ Instrument + Harmony + Voice
## Model 2: Classical ~ Instrument + Harmony + Voice + Instrument:Harmony
##   Res.Df   RSS Df Sum of Sq      F Pr(>F)
## 1   2485 12982                           
## 2   2479 12971  6    10.906 0.3474 0.9117
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{anova}\NormalTok{(lm.no.interactions, }\FunctionTok{update}\NormalTok{(lm.no.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{:}\NormalTok{Voice))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Analysis of Variance Table
## 
## Model 1: Classical ~ Instrument + Harmony + Voice
## Model 2: Classical ~ Instrument + Harmony + Voice + Instrument:Voice
##   Res.Df   RSS Df Sum of Sq      F Pr(>F)
## 1   2485 12982                           
## 2   2481 12972  4    9.6793 0.4628 0.7631
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{anova}\NormalTok{(lm.no.interactions, }\FunctionTok{update}\NormalTok{(lm.no.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{+}\NormalTok{ Voice}\SpecialCharTok{:}\NormalTok{Harmony))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Analysis of Variance Table
## 
## Model 1: Classical ~ Instrument + Harmony + Voice
## Model 2: Classical ~ Instrument + Harmony + Voice + Harmony:Voice
##   Res.Df   RSS Df Sum of Sq      F  Pr(>F)  
## 1   2485 12982                              
## 2   2479 12895  6    86.448 2.7698 0.01097 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# It seems like having Voice:Harmony is a good idea; we can summarize as follows:}


\NormalTok{lm.vh }\OtherTok{\textless{}{-}}  \FunctionTok{update}\NormalTok{(lm.no.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{+}\NormalTok{ Voice}\SpecialCharTok{:}\NormalTok{Harmony)}

\FunctionTok{anova}\NormalTok{(}\FunctionTok{lm}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}} \DecValTok{1}\NormalTok{, }\AttributeTok{data=}\NormalTok{fulldata),lm.no.interactions,lm.vh,lm.no.threeway,lm.all.interactions)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Analysis of Variance Table
## 
## Model 1: Classical ~ 1
## Model 2: Classical ~ Instrument + Harmony + Voice
## Model 3: Classical ~ Instrument + Harmony + Voice + Harmony:Voice
## Model 4: Classical ~ Instrument + Harmony + Voice + Instrument:Harmony + 
##     Instrument:Voice + Harmony:Voice
## Model 5: Classical ~ Instrument * Harmony * Voice
##   Res.Df   RSS Df Sum of Sq        F  Pr(>F)    
## 1   2492 17438                                  
## 2   2485 12982  7    4456.1 122.0707 < 2e-16 ***
## 3   2479 12895  6      86.4   2.7629 0.01115 *  
## 4   2469 12875 10      20.5   0.3933 0.95018    
## 5   2457 12813 12      61.8   0.9869 0.45872    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# We could also look at AIC and BIC, where we don\textquotesingle{}t need to worry }
\DocumentationTok{\#\# so much about nested models....}

\NormalTok{lm}\FloatTok{.0} \OtherTok{\textless{}{-}} \FunctionTok{lm}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}} \DecValTok{1}\NormalTok{, }\AttributeTok{data=}\NormalTok{fulldata)}
\NormalTok{lm.iv }\OtherTok{\textless{}{-}}  \FunctionTok{update}\NormalTok{(lm.no.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{:}\NormalTok{Voice)}
\NormalTok{lm.ih }\OtherTok{\textless{}{-}}  \FunctionTok{update}\NormalTok{(lm.no.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{:}\NormalTok{Harmony)}

\NormalTok{IC }\OtherTok{\textless{}{-}} \ControlFlowTok{function}\NormalTok{(M) \{}
\NormalTok{    x }\OtherTok{\textless{}{-}}\FunctionTok{c}\NormalTok{(}\AttributeTok{AIC=}\FunctionTok{AIC}\NormalTok{(M),}\AttributeTok{BIC=}\FunctionTok{BIC}\NormalTok{(M))}
    \FunctionTok{return}\NormalTok{(x)}
\NormalTok{\}}

\FunctionTok{print}\NormalTok{(x }\OtherTok{\textless{}{-}} \FunctionTok{rbind}\NormalTok{(}\AttributeTok{lm.0=}\FunctionTok{IC}\NormalTok{(lm}\FloatTok{.0}\NormalTok{),}
      \AttributeTok{lm.no.interactions=}\FunctionTok{IC}\NormalTok{(lm.no.interactions),}
      \AttributeTok{lm.iv=}\FunctionTok{IC}\NormalTok{(lm.iv),}
      \AttributeTok{lm.ih=}\FunctionTok{IC}\NormalTok{(lm.ih),}
      \AttributeTok{lm.vh=}\FunctionTok{IC}\NormalTok{(lm.vh),}
      \AttributeTok{lm.no.threway=}\FunctionTok{IC}\NormalTok{(lm.no.threeway),}
      \AttributeTok{lm.all.interactions=}\FunctionTok{IC}\NormalTok{(lm.all.interactions))}
\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                          AIC      BIC
## lm.0                11928.08 11939.73
## lm.no.interactions  11206.40 11258.79
## lm.iv               11212.54 11288.22
## lm.ih               11216.31 11303.62
## lm.vh               11201.74 11289.06
## lm.no.threway       11217.77 11363.31
## lm.all.interactions 11229.79 11445.17
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{x }\OtherTok{\textless{}{-}} \FunctionTok{as.data.frame}\NormalTok{(x)}

\NormalTok{x[x}\SpecialCharTok{$}\NormalTok{AIC}\SpecialCharTok{==}\FunctionTok{min}\NormalTok{(x}\SpecialCharTok{$}\NormalTok{AIC),]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##            AIC      BIC
## lm.vh 11201.74 11289.06
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{x[x}\SpecialCharTok{$}\NormalTok{BIC}\SpecialCharTok{==}\FunctionTok{min}\NormalTok{(x}\SpecialCharTok{$}\NormalTok{BIC),]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                        AIC      BIC
## lm.no.interactions 11206.4 11258.79
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# So the result is much the same as for the F{-}test comparisons. }
\DocumentationTok{\#\# The best BIC model is the no{-}interactions model, and the best }
\DocumentationTok{\#\# AIC model adds the Voice:Harmony interaction to that model.}
\end{Highlighting}
\end{Shaded}

There is some evidence in favor of keeping the Voice:Harmony interaction
(AIC, and ANOVA comparing this model with the no-interactions model),
but also some evidence against (BIC, and ANOVA comparing the model with
all two-way interactions with the no-interactions model).

One reason for keeping the interaction is that it give you more to talk
about with Dr.~Jimenez. A reason for not keeping it is that the evidence
is that, in the face of ambiguous evidence, simpler is better.

In what follows I will drop the Voice:Harmony interaction term, but you
could argue it either way.

\rm

\hypertarget{problem-2b.}{%
\subsection{Problem 2(b).}\label{problem-2b.}}

Since we have approximately 36 ratings from each participant, we can
expand the model in part\textasciitilde2(a) by including a random
intercept for each participant. Such a model is sometimes called a
``repeated measures'' model.

\hypertarget{bi}{%
\subsubsection{2(b)(i)}\label{bi}}

Write this expanded model in mathematical terms as a multi-level model,
using notation we have used in class. Be sure to carefully identify what
are the groups, and what are the individual observations. \label{p:2b.i}

\it

In this case the groups \(j\) are the listeners in the experiment and
the individual observations \(i\) are the individual ratings made by
each listener. The two-level model is then
\def\c#1{\mbox{\scriptsize\it #1}} \begin{eqnarray*}
\mbox{Classical}_i & = & \alpha_{0j[i]} + 
\beta_{\mbox{\scriptsize\it guitar}} \cdot 1_{\{\mbox{\scriptsize\it Instrument}_i=\mbox{\scriptsize\it guitar}\}} + 
\beta_{\mbox{\scriptsize\it piano}} \cdot 1_{\{\mbox{\scriptsize\it Instrument}_i=\mbox{\scriptsize\it piano}\}} + 
\beta_{\mbox{\scriptsize\it string}} \cdot 1_{\{\mbox{\scriptsize\it Instrument}_i=\mbox{\scriptsize\it string}\}} + \\ & &  
\beta_{\mbox{\scriptsize\it contrary}} \cdot 1_{\{\mbox{\scriptsize\it Voice}_i=\mbox{\scriptsize\it contracy}\}} + 
\beta_{\mbox{\scriptsize\it par3rd}} \cdot 1_{\{\mbox{\scriptsize\it Voice}_i=\mbox{\scriptsize\it par3rd}\}} + 
\beta_{\mbox{\scriptsize\it par5th}} \cdot 1_{\{\mbox{\scriptsize\it Voice}_i=\mbox{\scriptsize\it par5th}\}} + \\ & & 
\beta_{\mbox{\scriptsize\it I-IV-V}} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it I-IV-V}\}} + 
\beta_{\mbox{\scriptsize\it I-V-IV}} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it I-V-IV}\}} + \\ & & 
\beta_{\mbox{\scriptsize\it I-V-VI}} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it I-V-VI}\}} + 
\beta_{\mbox{\scriptsize\it IV-I-V}} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it IV-I-V}\}} + \epsilon_i ~,~~ \epsilon_i \stackrel{iid}{\sim} N(0,\sigma^2)\\
\alpha_{0j} & = & \beta_0 + \eta_{0j} ~,~~ \eta_j \stackrel{iid}{\sim} N(0,\tau_0^2)
\end{eqnarray*} where \(1_{\{c{condition}\}}\) are dummy variables that
equal 1 if \(\{\mbox{\it condition}\}\) is true, and equal 0 if
\(\{\mbox{\it condition}\}\) is false. Note that because of collinearity
with the intercept, the first level of every factor will be dropped by
lmer().

\bf Note:\it{} All the fixed-effect terms for the experimental factors have to be in the level-1 model since they are all level-1 covariates (that is, they vary with $i$ rather than $j$, and the level-2 model can only deal with terms that vary with $j$).

\rm

\hypertarget{bii}{%
\subsubsection{2(b)(ii)}\label{bii}}

Re-examine the influence of the three main experimental factors
(Instrument, Harmony \& Voice) on Classical ratings (that is, repeat
part 2(a)), using models that always include the random intercept for
participants (Please make sure the models are fitted with REML=FALSE
(i.e.~maximum likelihood) to ensure that the model comparisons make
sense.). \label{p:2b.ii}

\it

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# Again, we build some basic models to check whether we need 3{-}way or 2{-}way interactions.}

\DocumentationTok{\#\# Note that in the data frame, "Classical" and "Popular" are, for some reason, string}
\DocumentationTok{\#\# variables.  lm() was able to deal with this by converting the strings to their }
\DocumentationTok{\#\# obvious numeric equivalents, but lmer() can\textquotesingle{}t deal. So we have to convert them, before}
\DocumentationTok{\#\# moving forward.  We\textquotesingle{}ll go ahead and convert them in both "fulldata" and"nomissdata"}

\NormalTok{fulldata}\SpecialCharTok{$}\NormalTok{Classical }\OtherTok{\textless{}{-}} \FunctionTok{as.numeric}\NormalTok{(fulldata}\SpecialCharTok{$}\NormalTok{Classical)}
\NormalTok{fulldata}\SpecialCharTok{$}\NormalTok{Popular }\OtherTok{\textless{}{-}} \FunctionTok{as.numeric}\NormalTok{(fulldata}\SpecialCharTok{$}\NormalTok{Popular)}
\NormalTok{nomissdata}\SpecialCharTok{$}\NormalTok{Classical }\OtherTok{\textless{}{-}} \FunctionTok{as.numeric}\NormalTok{(nomissdata}\SpecialCharTok{$}\NormalTok{Classical)}
\NormalTok{nomissdata}\SpecialCharTok{$}\NormalTok{Popular }\OtherTok{\textless{}{-}} \FunctionTok{as.numeric}\NormalTok{(nomissdata}\SpecialCharTok{$}\NormalTok{Popular)}

\NormalTok{lmer.all.interactions }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{*}\NormalTok{ Harmony }\SpecialCharTok{*}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject),}
                              \AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.no.threeway }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.all.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ Instrument }\SpecialCharTok{:}\NormalTok{ Harmony }\SpecialCharTok{:}\NormalTok{ Voice)}

\NormalTok{lmer.no.interactions }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject),}
                             \AttributeTok{data=}\NormalTok{fulldata)}

\FunctionTok{display}\NormalTok{(lmer.all.interactions)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Instrument * Harmony * Voice + (1 | 
##     Subject), data = fulldata)
##                                            coef.est coef.se
## (Intercept)                                 4.49     0.27  
## Instrumentpiano                             1.02     0.32  
## Instrumentstring                            2.79     0.32  
## HarmonyI-V-IV                              -0.41     0.32  
## HarmonyI-V-VI                               1.08     0.32  
## HarmonyIV-I-V                              -0.43     0.32  
## Voicepar3rd                                -0.24     0.32  
## Voicepar5th                                -0.64     0.32  
## Instrumentpiano:HarmonyI-V-IV               0.81     0.45  
## Instrumentstring:HarmonyI-V-IV              0.89     0.45  
## Instrumentpiano:HarmonyI-V-VI               0.16     0.45  
## Instrumentstring:HarmonyI-V-VI              0.04     0.45  
## Instrumentpiano:HarmonyIV-I-V               0.46     0.45  
## Instrumentstring:HarmonyIV-I-V              0.30     0.45  
## Instrumentpiano:Voicepar3rd                -0.06     0.45  
## Instrumentstring:Voicepar3rd               -0.05     0.45  
## Instrumentpiano:Voicepar5th                 0.63     0.45  
## Instrumentstring:Voicepar5th                0.59     0.45  
## HarmonyI-V-IV:Voicepar3rd                   0.07     0.45  
## HarmonyI-V-VI:Voicepar3rd                  -1.44     0.45  
## HarmonyIV-I-V:Voicepar3rd                   0.52     0.45  
## HarmonyI-V-IV:Voicepar5th                   0.56     0.45  
## HarmonyI-V-VI:Voicepar5th                  -0.14     0.45  
## HarmonyIV-I-V:Voicepar5th                   0.77     0.45  
## Instrumentpiano:HarmonyI-V-IV:Voicepar3rd  -0.65     0.64  
## Instrumentstring:HarmonyI-V-IV:Voicepar3rd -0.62     0.63  
## Instrumentpiano:HarmonyI-V-VI:Voicepar3rd   0.99     0.64  
## Instrumentstring:HarmonyI-V-VI:Voicepar3rd  1.27     0.63  
## Instrumentpiano:HarmonyIV-I-V:Voicepar3rd  -0.22     0.64  
## Instrumentstring:HarmonyIV-I-V:Voicepar3rd  0.26     0.63  
## Instrumentpiano:HarmonyI-V-IV:Voicepar5th  -0.97     0.64  
## Instrumentstring:HarmonyI-V-IV:Voicepar5th -1.28     0.63  
## Instrumentpiano:HarmonyI-V-VI:Voicepar5th  -0.44     0.64  
## Instrumentstring:HarmonyI-V-VI:Voicepar5th -0.45     0.63  
## Instrumentpiano:HarmonyIV-I-V:Voicepar5th  -0.97     0.64  
## Instrumentstring:HarmonyIV-I-V:Voicepar5th -1.02     0.63  
## 
## Error terms:
##  Groups   Name        Std.Dev.
##  Subject  (Intercept) 1.30    
##  Residual             1.87    
## ---
## number of obs: 2493, groups: Subject, 70
## AIC = 10484.5, DIC = 10331.3
## deviance = 10369.9
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# So here, \textbackslash{}hat\textbackslash{}tau = 1.30 and \textbackslash{}hat\textbackslash{}sigma = 1.87, whereas for lm.all.interactions,}
\DocumentationTok{\#\# \textbackslash{}hat\textbackslash{}sigma = 2.28.  It\textquotesingle{}s interesting (and not surprising!! why??) to note that}
\DocumentationTok{\#\# sqrt(1.30\^{}2 + 1.87\^{}2) = 2.277477: Taking away the square roots, the two variance }
\DocumentationTok{\#\# components from the lmer() model add up to the residual variance in the lm() model.}

\NormalTok{lmer}\FloatTok{.0} \OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}} \DecValTok{1} \SpecialCharTok{+}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject),}\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.iv }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.no.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{:}\NormalTok{Voice)}
\NormalTok{lmer.ih }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.no.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{:}\NormalTok{Harmony)}
\NormalTok{lmer.vh }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.no.interactions, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{+}\NormalTok{ Voice}\SpecialCharTok{:}\NormalTok{Harmony)}

\DocumentationTok{\#\# For AIC and BIC we just need to make sure that the lmer() model is fitted }
\DocumentationTok{\#\# with REML=F}

\NormalTok{IC }\OtherTok{\textless{}{-}} \ControlFlowTok{function}\NormalTok{(M) \{}
\NormalTok{    M.ml }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(M,}\AttributeTok{REML=}\NormalTok{F)  }\DocumentationTok{\#\# ensure the model is fitted with REML=F}
\NormalTok{    x }\OtherTok{\textless{}{-}}\FunctionTok{c}\NormalTok{(}\AttributeTok{AIC=}\FunctionTok{AIC}\NormalTok{(M.ml),}\AttributeTok{BIC=}\FunctionTok{BIC}\NormalTok{(M.ml))}
    \FunctionTok{return}\NormalTok{(x)}
\NormalTok{\}}


\FunctionTok{print}\NormalTok{(x }\OtherTok{\textless{}{-}} \FunctionTok{rbind}\NormalTok{(}\AttributeTok{lmer.0=}\FunctionTok{IC}\NormalTok{(lmer}\FloatTok{.0}\NormalTok{),}
      \AttributeTok{lmer.no.interactions=}\FunctionTok{IC}\NormalTok{(lmer.no.interactions),}
      \AttributeTok{lmer.iv=}\FunctionTok{IC}\NormalTok{(lmer.iv),}
      \AttributeTok{lmer.ih=}\FunctionTok{IC}\NormalTok{(lmer.ih),}
      \AttributeTok{lmer.vh=}\FunctionTok{IC}\NormalTok{(lmer.vh),}
      \AttributeTok{lmer.no.threway=}\FunctionTok{IC}\NormalTok{(lmer.no.threeway),}
      \AttributeTok{lmer.all.interactions=}\FunctionTok{IC}\NormalTok{(lmer.all.interactions))}
\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                            AIC      BIC
## lmer.0                11440.30 11457.76
## lmer.no.interactions  10438.26 10496.47
## lmer.iv               10443.53 10525.03
## lmer.ih               10447.33 10540.47
## lmer.vh               10425.69 10518.83
## lmer.no.threway       10439.99 10591.34
## lmer.all.interactions 10445.91 10667.12
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{x }\OtherTok{\textless{}{-}} \FunctionTok{as.data.frame}\NormalTok{(x)}

\NormalTok{x[x}\SpecialCharTok{$}\NormalTok{AIC}\SpecialCharTok{==}\FunctionTok{min}\NormalTok{(x}\SpecialCharTok{$}\NormalTok{AIC),]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##              AIC      BIC
## lmer.vh 10425.69 10518.83
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{x[x}\SpecialCharTok{$}\NormalTok{BIC}\SpecialCharTok{==}\FunctionTok{min}\NormalTok{(x}\SpecialCharTok{$}\NormalTok{BIC),]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                           AIC      BIC
## lmer.no.interactions 10438.26 10496.47
\end{verbatim}

There are two interesting things to note here;

\begin{itemize}
\item Each lmer() model fits better than the corresponding lm() model, in the sense that the AIC and BIC are always lower for the lmer() model than the lm() model.  This already suggests that the random effect is useful (though we should check this with BIC also).
\item Once again, AIC picks the model with Voice:Harmony interaction, and BIC picks the no-interaction model.
\end{itemize}

\rm

\hypertarget{biii}{%
\subsubsection{2(b)(iii)}\label{biii}}

Is the random effect needed? Justify your answer with evidence. You may
(or may not) find valid evidence in fitted models, in statistical tests,
and/or in fit indices. \label{p:2b.iii}

\it

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# We begin by defining AIC, BIC, and DIC so that they apply to both }
\DocumentationTok{\#\# lmer() models with mle\textquotesingle{}s and to lm() models.}

\DocumentationTok{\#\# From now on I\textquotesingle{}ll assume that AIC, BIC and DIC are evaluated on lmer() }
\DocumentationTok{\#\# models with REML=F}

\DocumentationTok{\#\# AIC {-}{-} no change needed}

\DocumentationTok{\#\# BIC {-}{-} no change needed}

\DocumentationTok{\#\# DIc {-}{-} need to check to see if the model is "lm" or "lmer"...}


\NormalTok{DIC }\OtherTok{\textless{}{-}} \ControlFlowTok{function}\NormalTok{(M) \{}
         \ControlFlowTok{if}\NormalTok{(}\FunctionTok{class}\NormalTok{(M)}\SpecialCharTok{==}\StringTok{"lm"}\NormalTok{) \{ x }\OtherTok{\textless{}{-}} \FunctionTok{AIC}\NormalTok{(M) \}}
         \ControlFlowTok{else}\NormalTok{ \{ x }\OtherTok{\textless{}{-}} \FunctionTok{unname}\NormalTok{(}\FunctionTok{extractDIC}\NormalTok{(M)) \}   }
         \FunctionTok{return}\NormalTok{(x)}
\NormalTok{       \}}


\NormalTok{IC }\OtherTok{\textless{}{-}} \ControlFlowTok{function}\NormalTok{(M) \{}
    \ControlFlowTok{if}\NormalTok{ (}\FunctionTok{class}\NormalTok{(M)}\SpecialCharTok{==}\StringTok{"lm"}\NormalTok{) \{ M.ml }\OtherTok{\textless{}{-}}\NormalTok{ M \}  }\DocumentationTok{\#\# don\textquotesingle{}t have to refit lm() models.}
    \ControlFlowTok{else}\NormalTok{ \{ M.ml }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(M,}\AttributeTok{REML=}\NormalTok{F) \}  }\DocumentationTok{\#\# ensure that lmer() is fitted with REML=F}
\NormalTok{    x }\OtherTok{\textless{}{-}}\FunctionTok{c}\NormalTok{(}\AttributeTok{AIC=}\FunctionTok{AIC}\NormalTok{(M.ml),}\AttributeTok{BIC=}\FunctionTok{BIC}\NormalTok{(M.ml),}\AttributeTok{DIC=}\FunctionTok{DIC}\NormalTok{(M.ml))}
    \FunctionTok{return}\NormalTok{(x)}
\NormalTok{\}}

\DocumentationTok{\#\# I\textquotesingle{}m going to go ahead and calculate all the IC\textquotesingle{}s for all the models, }
\DocumentationTok{\#\# in case it is interesting to look at them later}

\NormalTok{x1 }\OtherTok{\textless{}{-}} \FunctionTok{rbind}\NormalTok{(}\AttributeTok{lm.0=}\FunctionTok{IC}\NormalTok{(lm}\FloatTok{.0}\NormalTok{),}
      \AttributeTok{lm.no.interactions=}\FunctionTok{IC}\NormalTok{(lm.no.interactions),}
      \AttributeTok{lm.iv=}\FunctionTok{IC}\NormalTok{(lm.iv),}
      \AttributeTok{lm.ih=}\FunctionTok{IC}\NormalTok{(lm.ih),}
      \AttributeTok{lm.vh=}\FunctionTok{IC}\NormalTok{(lm.vh),}
      \AttributeTok{lm.no.threway=}\FunctionTok{IC}\NormalTok{(lm.no.threeway),}
      \AttributeTok{lm.all.interactions=}\FunctionTok{IC}\NormalTok{(lm.all.interactions))}

\NormalTok{x2 }\OtherTok{\textless{}{-}} \FunctionTok{rbind}\NormalTok{(}\AttributeTok{lmer.0=}\FunctionTok{IC}\NormalTok{(lmer}\FloatTok{.0}\NormalTok{),}
      \AttributeTok{lmer.no.interactions=}\FunctionTok{IC}\NormalTok{(lmer.no.interactions),}
      \AttributeTok{lmer.iv=}\FunctionTok{IC}\NormalTok{(lmer.iv),}
      \AttributeTok{lmer.ih=}\FunctionTok{IC}\NormalTok{(lmer.ih),}
      \AttributeTok{lmer.vh=}\FunctionTok{IC}\NormalTok{(lmer.vh),}
      \AttributeTok{lmer.no.threway=}\FunctionTok{IC}\NormalTok{(lmer.no.threeway),}
      \AttributeTok{lmer.all.interactions=}\FunctionTok{IC}\NormalTok{(lmer.all.interactions))}

\DocumentationTok{\#\# All I am really interested in is the difference in DIC values between }
\DocumentationTok{\#\# the lm() models (which do not have a random effect) and the lmer() models}
\DocumentationTok{\#\# (which have a random intercept for each lister).}

\FunctionTok{cbind}\NormalTok{(}\StringTok{"DIC(lm){-}DIC(lmer)"} \OtherTok{=}\NormalTok{ x1[,}\DecValTok{3}\NormalTok{] }\SpecialCharTok{{-}}\NormalTok{ x2[,}\DecValTok{3}\NormalTok{])}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                     DIC(lm)-DIC(lmer)
## lm.0                         493.7825
## lm.no.interactions           788.1399
## lm.iv                        797.0081
## lm.ih                        800.9709
## lm.vh                        808.0558
## lm.no.threway                829.7836
## lm.all.interactions          859.8721
\end{verbatim}

The differences in DIC values between models without a random intercept
(the lm() models) and models with a random intercept (the lmer() models)
is never less than 400, and always favors the lmer() model with the
random intercept. For the two models that seem to be working best, the
no-interactions model and the model that adds Voice:Harmony to that
model, the difference is around 800.

Since we know a difference of 3 is interesting and a difference of 10 is
really something to pay attention to, clearly the random intercept for
each person is important to have in the model, regardless of which fixed
effects we use. \rm

\hypertarget{problem-2c.}{%
\subsection{Problem 2(c).}\label{problem-2c.}}

The random intercept in a repeated measures model can account for
``personal biases'' in ratings: for example, perhaps person A is more
inclined to rate everything as classical, and person B is more inclined
to rate everything as popular. This can be accounted for by the random
intercept. Alternatively, perhaps personal biases vary with the type of
instrument, type of harmony, and/or type of voice leading. For example,
perhaps people vary in the degree to which they are inclined to call
music played by a string quartet ``classical''. This suggests, e.g., a
random effect of the form \texttt{(Instrument|Subject)} or
\texttt{(0 + Instrument|Subject)} (and similarly for \verb|Harmony| and
\verb|Voice|). One could argue for a similar random effect for each
person/harmony combination, and for each person/voice leading
combination.

\hypertarget{ci}{%
\subsubsection{2(c)(i)}\label{ci}}

Determine whether the model in 2(b)(ii) is improved by adding one or
more of these three new random effect terms. Find the best combination
of these terms (A good rule of thumb is to only include a term like
(Voice\(|\)Subject) in the model if Voice is already a fixed effect.)
and provide suitable evidence for your answer. \label{p:2c.i}

\it

So far the two models

\begin{itemize}
\item \verb|lmer.no.interactions|, with formula \verb/Classical ~ Instrument + Harmony + Voice + (1 | Subject)/
\item \verb|lm.vh|, with formula \verb/Classical ~ Instrument + Harmony + Voice + (1 | Subject) + Harmony:Voice/
\end{itemize}

seem to be the best from 2(b)(ii). I will show what we get for each of
these two models.

There are two things to note before I begin:

\begin{itemize}
\item When I am considering more than one random effect, I can model without correlations, for example like this:
\begin{verbatim}
Classical ~ Inst + Voic + Harm + (1|Subject) + (0+Inst|Subject) + (0+Voic|Subject) 
\end{verbatim}
or with correlations, for example like this:
\begin{verbatim}
Classical ~ Inst + Voic + Harm + (1 + Inst + Voic | Subject)
\end{verbatim}
I am going to first try to model \bf with\it{} correlations.  But it turns out that the models are a little more interpretable if we force some correlations to be zero, so I will show final models like that also.
\item In addition to the original models, there are seven modified models to consider (3 models with one additional random effect, 3 models with two additional random effects, and 1 model with all three random effects).
\end{itemize}

\hypertarget{lets-first-look-at-lmer.no.interactions}{%
\paragraph{\texorpdfstring{Let's first look at
\texttt{lmer.no.interactions}:}{Let's first look at lmer.no.interactions:}}\label{lets-first-look-at-lmer.no.interactions}}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{formula}\NormalTok{(lmer.no.interactions)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Classical ~ Instrument + Harmony + Voice + (1 | Subject)
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.ni }\OtherTok{\textless{}{-}}\NormalTok{ lmer.no.interactions }\DocumentationTok{\#\# just to have a shorter name to type!!}

\NormalTok{lmer.ni.i }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.ni, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.ni.v }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.ni, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Voice}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.ni.h }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.ni, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Harmony}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.ni.iv }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.ni, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.ni.ih }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.ni, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Harmony}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.ni.vh }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.ni, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.ni.ivh }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.ni, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{DIC.ml }\OtherTok{\textless{}{-}} \ControlFlowTok{function}\NormalTok{(M) }\FunctionTok{DIC}\NormalTok{(}\FunctionTok{update}\NormalTok{(M,}\AttributeTok{REML=}\NormalTok{F))}

\FunctionTok{print}\NormalTok{(x }\OtherTok{\textless{}{-}} \FunctionTok{rbind}\NormalTok{(}\AttributeTok{lmer.ni=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.ni)),}
      \AttributeTok{lmer.ni.i=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.ni.i)),}
      \AttributeTok{lmer.ni.v=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.ni.v)),}
      \AttributeTok{lmer.ni.h=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.ni.h)),}
      \AttributeTok{lmer.ni.iv=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.ni.iv)),}
      \AttributeTok{lmer.ni.ih=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.ni.ih)),}
      \AttributeTok{lmer.ni.vh=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.ni.vh)),}
      \AttributeTok{lmer.ni.ivh=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.ni.ivh)))}
\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                   DIC
## lmer.ni     10418.260
## lmer.ni.i   10038.681
## lmer.ni.v   10418.231
## lmer.ni.h   10316.495
## lmer.ni.iv  10032.766
## lmer.ni.ih   9859.542
## lmer.ni.vh  10315.787
## lmer.ni.ivh  9842.594
\end{verbatim}

Three observations are important to make here.

First, when we specify a term like (Harmony\textbar Subject), we get a
random coefficient for
\bf each level\it{} of the discrete Harmony variable, as can be seen in this summary of the model \verb|lmer.ni.h|:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{display}\NormalTok{(lmer.ni.h)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Instrument + Harmony + Voice + (1 + 
##     Harmony | Subject), data = fulldata)
##                  coef.est coef.se
## (Intercept)       4.34     0.19  
## Instrumentpiano   1.37     0.09  
## Instrumentstring  3.12     0.09  
## HarmonyI-V-IV    -0.03     0.10  
## HarmonyI-V-VI     0.77     0.17  
## HarmonyIV-I-V     0.04     0.10  
## Voicepar3rd      -0.40     0.09  
## Voicepar5th      -0.36     0.09  
## 
## Error terms:
##  Groups   Name          Std.Dev. Corr              
##  Subject  (Intercept)   1.34                       
##           HarmonyI-V-IV 0.09      1.00             
##           HarmonyI-V-VI 1.18     -0.31 -0.31       
##           HarmonyIV-I-V 0.03      0.33  0.33 -1.00 
##  Residual               1.81                       
## ---
## number of obs: 2493, groups: Subject, 70
## AIC = 10376.8, DIC = 10294.9
## deviance = 10316.9
\end{verbatim}

Second, six of the models complain about ``boundary (singular) fit''. As
we discussed in class, this can mean either (a) a \(\tau^2\) is
essentially 0; or (b) a \(\rho\) is essentially \(-1\)or +1 (more
generally this complaint happens when the variance covariance matrix of
the random vector of \(\eta\) is not of full rank). For example, in the
summary of \verb|lmer.ni.h| above, we can see correlations of +1 and
\$-\$1.

Correlations of +1 and \(-1\)mean that there are really fewer random
effects than we have tried to specify. In the case of \verb|lmer.ni.h|
we see that the random effect for \verb|Harmony = I-V-IV| is identical
to the random intercept (since their correlation is \(-1\)), and the
random effect for \verb|Harmony = I-V-VI| is exactly \(-1\) times the
random effect for \verb|Harmony = IV-I-V| (since their correlation is
\(-1\)).

These correlations aren't necessarily bad (especially because there were
no convergence warnings); they are just telling us we could have a
simpler model if some of the coefficients shared the same random
component \(\eta\). It is not so easy to implement a constraint like
``sharing \(\eta\)'s'\,' in lmer(), so we won't try. We'll just observe
that the model seems to be a bit over-specified.

(It is conceptually easier to implement a ``sharing \(\eta\)'s'\,'
constraint when building a Bayesian version of the model, and maybe we
will take a look at this later in the semester).

Finally, there is a clear winner in terms of DIC. Remarkably enough, it
is the most complicated model. Here is a summary of the model:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{display}\NormalTok{(lmer.ni.ivh)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Instrument + Harmony + Voice + (1 + 
##     Instrument + Voice + Harmony | Subject), data = fulldata)
##                  coef.est coef.se
## (Intercept)       4.34     0.22  
## Instrumentpiano   1.37     0.17  
## Instrumentstring  3.12     0.24  
## HarmonyI-V-IV    -0.03     0.10  
## HarmonyI-V-VI     0.77     0.18  
## HarmonyIV-I-V     0.04     0.09  
## Voicepar3rd      -0.40     0.09  
## Voicepar5th      -0.36     0.08  
## 
## Error terms:
##  Groups   Name             Std.Dev. Corr                                      
##  Subject  (Intercept)      1.65                                               
##           Instrumentpiano  1.30     -0.42                                     
##           Instrumentstring 1.88     -0.62  0.66                               
##           Voicepar3rd      0.35     -0.24  0.41  0.59                         
##           Voicepar5th      0.31     -0.19  0.23  0.31  0.83                   
##           HarmonyI-V-IV    0.34      0.47 -0.39 -0.28 -0.66 -0.88             
##           HarmonyI-V-VI    1.28      0.02 -0.27 -0.42 -0.75 -0.32  0.18       
##           HarmonyIV-I-V    0.11      0.09 -0.32  0.01  0.36  0.43 -0.12  0.04 
##  Residual                  1.54
\end{verbatim}

\begin{verbatim}
## Warning in commonArgs(par, fn, control, environment()): maxfun < 10 *
## length(par)^2 is not recommended.
\end{verbatim}

\begin{verbatim}
## ---
## number of obs: 2493, groups: Subject, 70
## AIC = 9952.5, DIC = 9822.7
## deviance = 9842.6
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# There is a warning here about "maxfun" being less than the recommended}
\DocumentationTok{\#\# number.  "maxfun" is the number of iterations needed for the non{-}linear}
\DocumentationTok{\#\# maximization (that we talked about briefly in class).  If we were also}
\DocumentationTok{\#\# getting a convergence warning, I would want to increase maxfun (you can}
\DocumentationTok{\#\# figure out how with a google search). But since there doesn\textquotesingle{}t seem to be }
\DocumentationTok{\#\# a convergence problem, I\textquotesingle{}m not going to worry about maxfun.}
\end{Highlighting}
\end{Shaded}

This model also has a singular variance-covariance matrix for the
\(\eta\)'s, but it is not because any correlation is +1 or \$-\$1.
Instead, the particular combination of correlations makes the
variance-covariance matrix less than full rank. That makes the model a
little harder to interpret, and I will show an alternative below that is
a little easier to interpret.

\hypertarget{now-lets-look-at-lmer.ni.vh}{%
\paragraph{\texorpdfstring{Now let's look at
\texttt{lmer.ni.vh}:}{Now let's look at lmer.ni.vh:}}\label{now-lets-look-at-lmer.ni.vh}}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{formula}\NormalTok{(lmer.vh)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Classical ~ Instrument + Harmony + Voice + (1 | Subject) + Harmony:Voice
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.vh.i }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.vh, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.vh.v }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.vh, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Voice}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.vh.h }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.vh, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Harmony}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.vh.iv }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.vh, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.vh.ih }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.vh, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Harmony}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.vh.vh }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.vh, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony}\SpecialCharTok{|}\NormalTok{Subject))}

\NormalTok{lmer.vh.ivh }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.vh, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1} \SpecialCharTok{+}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony}\SpecialCharTok{|}\NormalTok{Subject))}

\FunctionTok{print}\NormalTok{(x }\OtherTok{\textless{}{-}} \FunctionTok{rbind}\NormalTok{(}\AttributeTok{lmer.vh=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.vh)),}
      \AttributeTok{lmer.vh.i=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.vh.i)),}
      \AttributeTok{lmer.vh.v=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.vh.v)),}
      \AttributeTok{lmer.vh.h=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.vh.h)),}
      \AttributeTok{lmer.vh.iv=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.vh.iv)),}
      \AttributeTok{lmer.vh.ih=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.vh.ih)),}
      \AttributeTok{lmer.vh.vh=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.vh.vh)),}
      \AttributeTok{lmer.vh.ivh=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.vh.ivh)))}
\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                   DIC
## lmer.vh     10393.687
## lmer.vh.i   10007.133
## lmer.vh.v   10393.659
## lmer.vh.h   10289.893
## lmer.vh.iv  10001.103
## lmer.vh.ih   9823.783
## lmer.vh.vh  10289.173
## lmer.vh.ivh  9806.266
\end{verbatim}

All of the comments that applied to the models we built above that
elaborated on the \texttt{lmer.ni} model also apply to these models as
well.

And once again, the most complex model, \verb|lmer.vh.ivh|, is preferred
by DIC. Let's take a quick look at it:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{display}\NormalTok{(lmer.vh.ivh)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Instrument + Harmony + Voice + (1 + 
##     Instrument + Voice + Harmony | Subject) + Harmony:Voice, 
##     data = fulldata)
##                           coef.est coef.se
## (Intercept)                4.26     0.23  
## Instrumentpiano            1.37     0.17  
## Instrumentstring           3.12     0.24  
## HarmonyI-V-IV              0.16     0.16  
## HarmonyI-V-VI              1.14     0.21  
## HarmonyIV-I-V             -0.18     0.15  
## Voicepar3rd               -0.27     0.16  
## Voicepar5th               -0.24     0.15  
## HarmonyI-V-IV:Voicepar3rd -0.37     0.21  
## HarmonyI-V-VI:Voicepar3rd -0.68     0.21  
## HarmonyIV-I-V:Voicepar3rd  0.53     0.21  
## HarmonyI-V-IV:Voicepar5th -0.19     0.21  
## HarmonyI-V-VI:Voicepar5th -0.42     0.21  
## HarmonyIV-I-V:Voicepar5th  0.12     0.21  
## 
## Error terms:
##  Groups   Name             Std.Dev. Corr                                      
##  Subject  (Intercept)      1.65                                               
##           Instrumentpiano  1.30     -0.42                                     
##           Instrumentstring 1.88     -0.62  0.66                               
##           Voicepar3rd      0.35     -0.24  0.40  0.59                         
##           Voicepar5th      0.31     -0.19  0.23  0.31  0.84                   
##           HarmonyI-V-IV    0.34      0.48 -0.39 -0.28 -0.68 -0.88             
##           HarmonyI-V-VI    1.28      0.03 -0.26 -0.42 -0.75 -0.32  0.17       
##           HarmonyIV-I-V    0.08      0.18 -0.44 -0.01  0.40  0.61 -0.30  0.00 
##  Residual                  1.53
\end{verbatim}

\begin{verbatim}
## Warning in commonArgs(par, fn, control, environment()): maxfun < 10 *
## length(par)^2 is not recommended.
\end{verbatim}

\begin{verbatim}
## ---
## number of obs: 2493, groups: Subject, 70
## AIC = 9938.1, DIC = 9776.4
## deviance = 9806.2
\end{verbatim}

As before, the singularity in this model is not due to a specific
correlation of +1 or \(-1\), but rather due to the fact that the
estimated variance-covariance matrix for the vector of \(\eta\)'s is
less than full rank.

\hypertarget{alternative-models-with-correlations-constrained-to-be-zero.}{%
\paragraph{Alternative models with correlations constrained to be
zero.}\label{alternative-models-with-correlations-constrained-to-be-zero.}}

The last thing I want to show is what these models look like if we force
correlations between (groups of) random effects to be zero.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.ni.ivh.c0 }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{1}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+} 
\NormalTok{    (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Instrument}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Voice}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{+}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Harmony}\SpecialCharTok{|}\NormalTok{Subject),}
    \AttributeTok{data=}\NormalTok{fulldata)}

\DocumentationTok{\#\# or equivalently lmer.ni.ivh.c0 \textless{}{-} lmer(Classical \textasciitilde{} Instrument + Harmony + Voice + }
\DocumentationTok{\#\#                                (Instrument+Harmony+Voice||Subject),data=fulldata)}
\FunctionTok{display}\NormalTok{(lmer.ni.ivh.c0)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Instrument + Harmony + Voice + (1 | 
##     Subject) + (0 + Instrument | Subject) + (0 + Voice | Subject) + 
##     (0 + Harmony | Subject), data = fulldata)
##                  coef.est coef.se
## (Intercept)       4.34     0.23  
## Instrumentpiano   1.37     0.17  
## Instrumentstring  3.12     0.24  
## HarmonyI-V-IV    -0.03     0.09  
## HarmonyI-V-VI     0.77     0.18  
## HarmonyIV-I-V     0.04     0.09  
## Voicepar3rd      -0.40     0.08  
## Voicepar5th      -0.36     0.08  
## 
## Error terms:
##  Groups    Name             Std.Dev. Corr             
##  Subject   (Intercept)      0.00                      
##  Subject.1 Instrumentguitar 1.16                      
##            Instrumentpiano  1.01      0.29            
##            Instrumentstring 0.74     -0.97 -0.28      
##  Subject.2 Voicecontrary    0.72                      
##            Voicepar3rd      0.78     0.92             
##            Voicepar5th      0.78     0.97  0.99       
##  Subject.3 HarmonyI-IV-V    1.09                      
##            HarmonyI-V-IV    1.23     0.99             
##            HarmonyI-V-VI    1.21     0.39  0.48       
##            HarmonyIV-I-V    1.11     1.00  0.98  0.40 
##  Residual                   1.55                      
## ---
## number of obs: 2493, groups: Subject, 70
## AIC = 9952.5, DIC = 9848.8
## deviance = 9868.7
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.vh.ivh.c0 }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Harmony}\SpecialCharTok{*}\NormalTok{Voice }\SpecialCharTok{+} 
\NormalTok{                         (Instrument}\SpecialCharTok{+}\NormalTok{Harmony}\SpecialCharTok{+}\NormalTok{Voice}\SpecialCharTok{||}\NormalTok{Subject),}\AttributeTok{data=}\NormalTok{fulldata)}

\FunctionTok{display}\NormalTok{(lmer.vh.ivh.c0)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Instrument + Harmony * Voice + ((1 | 
##     Subject) + (0 + Instrument | Subject) + (0 + Harmony | Subject) + 
##     (0 + Voice | Subject)), data = fulldata)
##                           coef.est coef.se
## (Intercept)                4.26     0.24  
## Instrumentpiano            1.37     0.17  
## Instrumentstring           3.12     0.24  
## HarmonyI-V-IV              0.16     0.15  
## HarmonyI-V-VI              1.14     0.21  
## HarmonyIV-I-V             -0.18     0.15  
## Voicepar3rd               -0.27     0.16  
## Voicepar5th               -0.24     0.15  
## HarmonyI-V-IV:Voicepar3rd -0.37     0.21  
## HarmonyI-V-VI:Voicepar3rd -0.68     0.21  
## HarmonyIV-I-V:Voicepar3rd  0.53     0.21  
## HarmonyI-V-IV:Voicepar5th -0.19     0.21  
## HarmonyI-V-VI:Voicepar5th -0.42     0.21  
## HarmonyIV-I-V:Voicepar5th  0.12     0.21  
## 
## Error terms:
##  Groups    Name             Std.Dev. Corr             
##  Subject   (Intercept)      0.00                      
##  Subject.1 Instrumentguitar 1.15                      
##            Instrumentpiano  1.00      0.29            
##            Instrumentstring 0.73     -1.00 -0.31      
##  Subject.2 HarmonyI-IV-V    1.08                      
##            HarmonyI-V-IV    1.22     0.99             
##            HarmonyI-V-VI    1.20     0.38  0.47       
##            HarmonyIV-I-V    1.10     1.00  0.97  0.39 
##  Subject.3 Voicecontrary    0.75                      
##            Voicepar3rd      0.80     0.92             
##            Voicepar5th      0.81     0.97  0.99       
##  Residual                   1.54                      
## ---
## number of obs: 2493, groups: Subject, 70
## AIC = 9938.2, DIC = 9802.9
## deviance = 9832.6
\end{verbatim}

These models do not fit as well as the models where the correlations are
free (and the differences are pretty dramatic):

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{x }\OtherTok{\textless{}{-}} \FunctionTok{t}\NormalTok{(}\FunctionTok{sapply}\NormalTok{(}\FunctionTok{list}\NormalTok{(lmer.ni.ivh,lmer.ni.ivh.c0,lmer.vh.ivh,lmer.vh.ivh.c0),IC))}
\FunctionTok{dimnames}\NormalTok{(x)[[}\DecValTok{1}\NormalTok{]] }\OtherTok{\textless{}{-}}\FunctionTok{c}\NormalTok{(}\StringTok{"lmer.ni.ivh"}\NormalTok{,}\StringTok{"lmer.ni.ivh.c0"}\NormalTok{,}\StringTok{"lmer.vh.ivh"}\NormalTok{,}\StringTok{"lmer.vh.ivh.c0"}\NormalTok{)}
\NormalTok{x}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                     AIC      BIC      DIC
## lmer.ni.ivh    9932.594 10194.55 9842.594
## lmer.ni.ivh.c0 9932.689 10118.97 9868.689
## lmer.vh.ivh    9908.266 10205.15 9806.266
## lmer.vh.ivh.c0 9908.575 10129.78 9832.575
\end{verbatim}

However, the random effects seem to make a bit more sense. For example,
comparing the \(\hat\tau\)'s and \(\hat\rho\)'s for the
\verb|lmer.ni.ivh| model vs the \verb|lmer.ni.ivh.c0| model,

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{VarCorr}\NormalTok{(lmer.ni.ivh)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##  Groups   Name             Std.Dev. Corr                                     
##  Subject  (Intercept)      1.64621                                           
##           Instrumentpiano  1.29744  -0.423                                   
##           Instrumentstring 1.88032  -0.621  0.662                            
##           Voicepar3rd      0.35061  -0.240  0.408  0.588                     
##           Voicepar5th      0.31161  -0.186  0.230  0.314  0.831              
##           HarmonyI-V-IV    0.33714   0.474 -0.389 -0.278 -0.659 -0.876       
##           HarmonyI-V-VI    1.27767   0.025 -0.267 -0.423 -0.746 -0.324  0.181
##           HarmonyIV-I-V    0.10752   0.092 -0.322  0.006  0.361  0.432 -0.118
##  Residual                  1.53959                                           
##        
##        
##        
##        
##        
##        
##        
##        
##   0.037
## 
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{VarCorr}\NormalTok{(lmer.ni.ivh.c0)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##  Groups    Name             Std.Dev.   Corr               
##  Subject   (Intercept)      0.00012282                    
##  Subject.1 Instrumentguitar 1.15640451                    
##            Instrumentpiano  1.00862464  0.291             
##            Instrumentstring 0.73756023 -0.968 -0.282      
##  Subject.2 Voicecontrary    0.72008759                    
##            Voicepar3rd      0.78002145 0.919              
##            Voicepar5th      0.78408719 0.967  0.989       
##  Subject.3 HarmonyI-IV-V    1.09022145                    
##            HarmonyI-V-IV    1.22883993 0.988              
##            HarmonyI-V-VI    1.21096104 0.391  0.477       
##            HarmonyIV-I-V    1.11156133 0.997  0.977  0.402
##  Residual                   1.54778331
\end{verbatim}

we see that \(\hat\sigma\) is almost the same for the two models, but
instead of an overall intercept \(\eta_{0j}\) (whose \(\hat\tau_0\) is
so small we could consider omitting it), we are getting a distinct
\(\eta\) for every level of every design variable, and the correlations
between the \(\eta\)'s make some sense. For example, referring to
\texttt{lmer.ni.ivh.c0},

\begin{itemize}

\item Among the $\eta$'s for Instrument, $\hat\rho(\eta_{Instrumentguitar},\eta_{Instrumentstring})=-0.968$ .  This suggests that the listeners' personal tendency to call a piece classical when the instrument is strings is equal to and opposite their personal tendency to call it classical when the instrument is guitar.  On the other hand  $\hat\rho(\eta_{Instrumentguitar},\eta_{Instrumentpiano})=0.291$ and $\hat\rho(\eta_{Instrumentpiano},\eta_{Instrumentstring})=-0.282$, suggesting that any personal tendency to call something classical if it is piano is much more ambiguous than for the other two instrments.

\item Looking at the $\hat\rho$'s among the $\eta$'s for Voice, it seems like the tendency to call something classical doesn't change that much with the type of voice leading; all three $\eta$'s here are pretty highly correlated (estimated correlations of 0.919, 0.967 and 0.989) 

\item Looking at the $\hat\rho$'s among the $\eta$'s for Harmony, the personal tendency to call something classical is pretty much the same for the I-IV-V, I-V-IV and IV-I-V harmonies, but how listeners hear the I-V-VI harmony is somewhat different.

\end{itemize}

Very similar interpretations can be made for the \(\hat\rho\)'s and
\(\hat\tau\)'s for the \texttt{lmer.vh.ivh.c0} model. Further analysis
using the ranef() function to extract estimated \(\eta\) values would be
useful to confirm these ideas.

As modelers we would now have to decide (in consultation with
Dr.~Jimenez, of course) whether the greater interpretability is worth
the substantially worse fits of these models.

In the remainder of this document I will focus on the model
\texttt{lmer.ni.ivh.c0} (no fixed-effect interactions, and zero
correlations between groups of \(\eta\)'s) because

\begin{itemize}
\item The results for fixed effects are similar to what we would get if we allowed all the $\rho$'s to be estimated, and the random effect correlations are more interpretable. 
\item lmer() runs a little faster on them, and we have a lot of models to fit below.
\end{itemize}

There is an argument for keeping the Voice:Harmony in the model, but it
just makes the calculations, and especially writing out multilevel and
hierarchical models. In addition, as we will see below, the model with
the Voice:Harmony interaction is consistently selected by by AIC, and
the model without this interaction is consistently selected by ABF I may
return to this interaction later.

\rm

\hypertarget{cii}{%
\subsubsection{2(c)(ii)}\label{cii}}

Re-examine the influence of the three main experimental factors
(Instrument, Harmony \& Voice) on Classical ratings, using the random
effects you found in part (i). Comment briefly on your findings,
providing suitable brief evidence for each result. In addition, comment
on the sizes of the variances of whichever random effects you added to
the model, with respect to each other and with respect to the estimated
residual variance. \label{p:2c.ii}

\it

This is basically going back to the kind of analysis we did in 2(a), but
now with the random effects we have found in 2(c)(i).

First we re-consider the fixed effects with fully-correlated \(\eta\)'s,
\texttt{(Instrument\ +\ Voice\ +\ Harmony\ \textbar{}\ Subject)}:

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# We do not consider}
\DocumentationTok{\#\#  lmer.0 \textless{}{-} lmer(1 + (Instrument + Voice + Harmony | Subject), data=fulldata) }
\DocumentationTok{\#\# because it does not have fixed effects corresponding to the \textbackslash{}eta\textquotesingle{}s for }
\DocumentationTok{\#\# Instrument, Voice and Harmony.}
\DocumentationTok{\#\#}

\NormalTok{lmer.ni.ivh }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+} 
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{|}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.iv.ivh }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{:}\NormalTok{Voice }\SpecialCharTok{+}  
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{|}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.ih.ivh }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{:}\NormalTok{Harmony }\SpecialCharTok{+} 
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{|}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.vh.ivh }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+}\NormalTok{ Voice}\SpecialCharTok{:}\NormalTok{Harmony }\SpecialCharTok{+} 
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{|}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
## Model failed to converge with max|grad| = 0.00882431 (tol = 0.002, component 1)
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# There is a convergence warning here, but the convergence measure 0.0088 is reasonably }
\DocumentationTok{\#\# close to the tolerance 0.002, so I won\textquotesingle{}t worry too much about it. I could try other}
\DocumentationTok{\#\# optimizers, or just increase maxfun, to see if I can get to convergence but I will }
\DocumentationTok{\#\# let this be for now.}

\NormalTok{lmer.no.threeway.ivh }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{*}\NormalTok{ Voice }\SpecialCharTok{*}\NormalTok{ Harmony }\SpecialCharTok{{-}} 
\NormalTok{                           Instrument}\SpecialCharTok{:}\NormalTok{Voice}\SpecialCharTok{:}\NormalTok{Harmony }\SpecialCharTok{+}  
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{|}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.all.interactions.ivh }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{*}\NormalTok{ Voice }\SpecialCharTok{*}\NormalTok{ Harmony }\SpecialCharTok{+} 
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{|}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}


\FunctionTok{print}\NormalTok{(x }\OtherTok{\textless{}{-}} \FunctionTok{rbind}\NormalTok{(}
      \AttributeTok{lmer.ni.ivh=}\FunctionTok{IC}\NormalTok{(lmer.ni.ivh),}
      \AttributeTok{lmer.iv.ivh=}\FunctionTok{IC}\NormalTok{(lmer.iv.ivh),}
      \AttributeTok{lmer.ih.ivh=}\FunctionTok{IC}\NormalTok{(lmer.ih.ivh),}
      \AttributeTok{lmer.vh.ivh=}\FunctionTok{IC}\NormalTok{(lmer.vh.ivh),}
      \AttributeTok{lmer.no.threeway.ivh=}\FunctionTok{IC}\NormalTok{(lmer.no.threeway.ivh),}
      \AttributeTok{lmer.all.interactions.ivh=}\FunctionTok{IC}\NormalTok{(lmer.all.interactions.ivh))}
\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                                AIC      BIC      DIC
## lmer.ni.ivh               9932.589 10194.54 9842.589
## lmer.iv.ivh               9936.604 10221.84 9838.604
## lmer.ih.ivh               9939.570 10236.45 9837.570
## lmer.vh.ivh               9908.228 10205.11 9806.228
## lmer.no.threeway.ivh      9919.129 10274.22 9797.129
## lmer.all.interactions.ivh 9922.881 10347.83 9776.881
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{x }\OtherTok{\textless{}{-}} \FunctionTok{as.data.frame}\NormalTok{(x)}
\NormalTok{x[x}\SpecialCharTok{$}\NormalTok{AIC}\SpecialCharTok{==}\FunctionTok{min}\NormalTok{(x}\SpecialCharTok{$}\NormalTok{AIC),]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                  AIC      BIC      DIC
## lmer.vh.ivh 9908.228 10205.11 9806.228
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{x[x}\SpecialCharTok{$}\NormalTok{BIC}\SpecialCharTok{==}\FunctionTok{min}\NormalTok{(x}\SpecialCharTok{$}\NormalTok{BIC),]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                  AIC      BIC      DIC
## lmer.ni.ivh 9932.589 10194.54 9842.589
\end{verbatim}

AIC likes the model \texttt{lmer.vh.ivh} with only the interaction
Voice:Harmony, while BIC likes the simpler no-interactions model
\texttt{lmer.ni.ivh}.

Now, let's try the same models forcing correlations beween groups of
\(\eta\)'s to be zero,
\texttt{(Instrument\ +\ Voice\ +\ Harmony\ \textbar{}\textbar{}\ Subject)}:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.ni.ivh.c0 }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+} 
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{||}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.iv.ivh.c0 }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{:}\NormalTok{Voice }\SpecialCharTok{+}  
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{||}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.ih.ivh.c0 }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+}\NormalTok{ Instrument}\SpecialCharTok{:}\NormalTok{Harmony }\SpecialCharTok{+} 
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{||}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.vh.ivh.c0 }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+}\NormalTok{ Voice}\SpecialCharTok{:}\NormalTok{Harmony }\SpecialCharTok{+} 
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{||}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.no.threeway.ivh.c0 }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{*}\NormalTok{ Voice }\SpecialCharTok{*}\NormalTok{ Harmony }\SpecialCharTok{{-}} 
\NormalTok{                              Instrument}\SpecialCharTok{:}\NormalTok{Voice}\SpecialCharTok{:}\NormalTok{Harmony }\SpecialCharTok{+}  
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{||}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}

\NormalTok{lmer.all.interactions.ivh.c0 }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{*}\NormalTok{ Voice }\SpecialCharTok{*}\NormalTok{ Harmony }\SpecialCharTok{+} 
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{||}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}

\FunctionTok{print}\NormalTok{(x }\OtherTok{\textless{}{-}} \FunctionTok{rbind}\NormalTok{(}
      \AttributeTok{lmer.ni.ivh.c0=}\FunctionTok{IC}\NormalTok{(lmer.ni.ivh.c0),}
      \AttributeTok{lmer.iv.ivh.c0=}\FunctionTok{IC}\NormalTok{(lmer.iv.ivh.c0),}
      \AttributeTok{lmer.ih.ivh.c0=}\FunctionTok{IC}\NormalTok{(lmer.ih.ivh.c0),}
      \AttributeTok{lmer.vh.ivh.c0=}\FunctionTok{IC}\NormalTok{(lmer.vh.ivh.c0),}
      \AttributeTok{lmer.no.threeway.ivh.c0=}\FunctionTok{IC}\NormalTok{(lmer.no.threeway.ivh.c0),}
      \AttributeTok{lmer.all.interactions.ivh.c0=}\FunctionTok{IC}\NormalTok{(lmer.all.interactions.ivh.c0))}
\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                                   AIC      BIC      DIC
## lmer.ni.ivh.c0               9932.799 10119.08 9868.799
## lmer.iv.ivh.c0               9936.752 10146.32 9864.752
## lmer.ih.ivh.c0               9939.833 10161.04 9863.833
## lmer.vh.ivh.c0               9908.575 10129.78 9832.575
## lmer.no.threeway.ivh.c0      9919.656 10199.08 9823.656
## lmer.all.interactions.ivh.c0 9917.334 10266.61 9797.334
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{x }\OtherTok{\textless{}{-}} \FunctionTok{as.data.frame}\NormalTok{(x)}
\NormalTok{x[x}\SpecialCharTok{$}\NormalTok{AIC}\SpecialCharTok{==}\FunctionTok{min}\NormalTok{(x}\SpecialCharTok{$}\NormalTok{AIC),]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                     AIC      BIC      DIC
## lmer.vh.ivh.c0 9908.575 10129.78 9832.575
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{x[x}\SpecialCharTok{$}\NormalTok{BIC}\SpecialCharTok{==}\FunctionTok{min}\NormalTok{(x}\SpecialCharTok{$}\NormalTok{BIC),]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                     AIC      BIC      DIC
## lmer.ni.ivh.c0 9932.799 10119.08 9868.799
\end{verbatim}

For these models, AIC prefers \texttt{lmer.vh.ivh.c0}, while BIC prefers
\texttt{lmer.ni.ivh.c0}. These are the same fixed effects as found with
the \texttt{(Instrument\ +\ Voice\ +\ Harmony\ \textbar{}\ Subject)}
random effects.

For simplicity I will prefer the \texttt{lmer.ni.ivh.c0} model, but the
Voice:Harmony interaction keeps coming up; it may be worth returning to
the model(s) with the Voice:Harmony interaction later.

\rm

\hypertarget{ciii}{%
\subsubsection{2(c)(iii)}\label{ciii}}

Carefully write this model in mathematical terms as a hierarchical
linear model, using notation from class. Indicate values of estimated
parameters (and their SE's, if available). \label{p:2c.iii}

\it

I am taking

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.ni.ivh.c0 }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+} 
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{||}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}
\end{Highlighting}
\end{Shaded}

to be the my preferred model, for now.

First, I will write this as a multilevel model (it will look very
similar to the model for 2(b)(i)), and then I will re-write it as a
hierarchical model.

I will omit the correlations between \(\eta\)'s below when I write the
multi-level model, but I will include them for clariry when I write the
hierarchical model. We have seen in 2(b)(ii) what the correlation
structure will look like, depending on whether we are using
\texttt{(Instrument\ +\ Voice\ +\ Harmony\ \textbar{}\ Subject)} or
\texttt{(Instrument\ +\ Voice\ +\ Harmony\ \textbar{}\textbar{}\ Subject)}.

\hypertarget{multilevel-model}{%
\paragraph{Multilevel model:}\label{multilevel-model}}

\def\c#1{\mbox{\scriptsize\it #1}}

\begin{eqnarray*}
\mbox{Classical}_i & = & \alpha_{0j[i]} + 
\alpha_{\mbox{\scriptsize\it guitar},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Instrument}_i=\mbox{\scriptsize\it guitar}\}} + 
\alpha_{\mbox{\scriptsize\it piano},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Instrument}_i=\mbox{\scriptsize\it piano}\}} + 
\alpha_{\mbox{\scriptsize\it string},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Instrument}_i=\mbox{\scriptsize\it string}\}} + \\ & &  
\alpha_{\mbox{\scriptsize\it contrary},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Voice}_i=\mbox{\scriptsize\it contracy}\}} + 
\alpha_{\mbox{\scriptsize\it par3rd},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Voice}_i=\mbox{\scriptsize\it par3rd}\}} + 
\alpha_{\mbox{\scriptsize\it par5th},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Voice}_i=\mbox{\scriptsize\it par5th}\}} + \\ & & 
\alpha_{\mbox{\scriptsize\it I-IV-V},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it I-IV-V}\}} + 
\alpha_{\mbox{\scriptsize\it I-V-IV},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it I-V-IV}\}} + \\ & & 
\alpha_{\mbox{\scriptsize\it I-V-VI},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it I-V-VI}\}} + 
\alpha_{\mbox{\scriptsize\it IV-I-V},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it IV-I-V}\}} + \epsilon_i ~,~~ \epsilon_i \stackrel{iid}{\sim} N(0,\sigma^2)\\
\alpha_{0j} & = & \beta_0 + \eta_{0j} ~,~~ \eta_{0j} \stackrel{iid}{\sim} N(0,\tau_0^2) \\
\alpha_{\mbox{\scriptsize\it guitar},j} & = & \beta_{\mbox{\scriptsize\it guitar}} + \eta_{\mbox{\scriptsize\it guitar},j}~,~~ 
   \eta_{\mbox{\scriptsize\it guitar},j} \stackrel{iid}{\sim} N(0,\tau_{\mbox{\scriptsize\it guitar}}^2) \\
\alpha_{\mbox{\scriptsize\it piano},j} & = &  \beta_{\mbox{\scriptsize\it piano}} + \eta_{\mbox{\scriptsize\it piano},j}~,~~ 
   \eta_{\mbox{\scriptsize\it piano},j} \stackrel{iid}{\sim} N(0,\tau_{\mbox{\scriptsize\it piano}}^2) \\
\alpha_{\mbox{\scriptsize\it string},j} & = & \beta_{\mbox{\scriptsize\it string}} + \eta_{\mbox{\scriptsize\it string},j}~,~~ 
   \eta_{\mbox{\scriptsize\it string},j} \stackrel{iid}{\sim} N(0,\tau_{\mbox{\scriptsize\it string}}^2) \\
\alpha_{\mbox{\scriptsize\it contrary},j} & = & \beta_{\mbox{\scriptsize\it contrary}} + \eta_{\mbox{\scriptsize\it contrary},j}~,~~ 
   \eta_{\mbox{\scriptsize\it contrary},j} \stackrel{iid}{\sim} N(0,\tau_{\mbox{\scriptsize\it contrary}}^2) \\
\alpha_{\mbox{\scriptsize\it par3rd},j} & = & \beta_{\mbox{\scriptsize\it par3rd}} + \eta_{\mbox{\scriptsize\it par3rd},j}~,~~ 
   \eta_{\mbox{\scriptsize\it par3rd},j}  \stackrel{iid}{\sim} N(0,\tau_{\mbox{\scriptsize\it par3rd}}^2) \\
\alpha_{\mbox{\scriptsize\it par5th},j} & = & \beta_{\mbox{\scriptsize\it par5th}} + \eta_{\mbox{\scriptsize\it par5th},j}~,~~ 
   \eta_{\mbox{\scriptsize\it par5th},j}  \stackrel{iid}{\sim} N(0,\tau_{\mbox{\scriptsize\it par5th}}^2) \\
\alpha_{\mbox{\scriptsize\it I-IV-V},j} & = & \beta_{\mbox{\scriptsize\it I-IV-V}} + \eta_{\mbox{\scriptsize\it I-IV-V},j}~,~~ 
   \eta_{\mbox{\scriptsize\it I-IV-V},j} \stackrel{iid}{\sim} N(0,\tau_{\mbox{\scriptsize\it I-IV-V}}^2) \\
\alpha_{\mbox{\scriptsize\it I-V-IV},j} & = & \beta_{\mbox{\scriptsize\it I-V-IV}} + \eta_{\mbox{\scriptsize\it I-V-IV},j}~,~~ 
   \eta_{\mbox{\scriptsize\it I-V-IV},j} \stackrel{iid}{\sim} N(0,\tau_{\mbox{\scriptsize\it I-V-IV}}^2) \\
\alpha_{\mbox{\scriptsize\it I-V-VI},j} & = & \beta_{\mbox{\scriptsize\it I-V-VI}} + \eta_{\mbox{\scriptsize\it I-V-VI},j}~,~~ 
   \eta_{\mbox{\scriptsize\it I-V-VI},j} \stackrel{iid}{\sim} N(0,\tau_{\mbox{\scriptsize\it I-V-VI}}^2) \\
\alpha_{\mbox{\scriptsize\it IV-I-V},j} & = & \beta_{\mbox{\scriptsize\it IV-I-V}} + \eta_{\mbox{\scriptsize\it IV-I-V},j}~,~~ 
   \eta_{\mbox{\scriptsize\it IV-I-V},j} \stackrel{iid}{\sim} N(0,\tau_{\mbox{\scriptsize\it IV-I-V}}^2) 
\end{eqnarray*}

\hypertarget{hierarchical-model}{%
\paragraph{Hierarchical model:}\label{hierarchical-model}}

It's actually somewhat difficult (or at least tedious) to write the
Hierarchical model efficiently. One way would be to define
\begin{eqnarray*}
m_i & = & \alpha_{0j[i]} + 
\alpha_{\mbox{\scriptsize\it guitar},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Instrument}_i=\mbox{\scriptsize\it guitar}\}} + 
\alpha_{\mbox{\scriptsize\it piano},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Instrument}_i=\mbox{\scriptsize\it piano}\}} + 
\alpha_{\mbox{\scriptsize\it string},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Instrument}_i=\mbox{\scriptsize\it string}\}} + \\ & &  
\alpha_{\mbox{\scriptsize\it contrary},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Voice}_i=\mbox{\scriptsize\it contracy}\}} + 
\alpha_{\mbox{\scriptsize\it par3rd},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Voice}_i=\mbox{\scriptsize\it par3rd}\}} + 
\alpha_{\mbox{\scriptsize\it par5th},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Voice}_i=\mbox{\scriptsize\it par5th}\}} + \\ & & 
\alpha_{\mbox{\scriptsize\it I-IV-V},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it I-IV-V}\}} + 
\alpha_{\mbox{\scriptsize\it I-V-IV},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it I-V-IV}\}} + \\ & & 
\alpha_{\mbox{\scriptsize\it I-V-VI},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it I-V-VI}\}} + 
\alpha_{\mbox{\scriptsize\it IV-I-V},j[i]} \cdot 1_{\{\mbox{\scriptsize\it Harmony}_i=\mbox{\scriptsize\it IV-I-V}\}}
\end{eqnarray*} and then write, ignoring the correlations again,
\begin{eqnarray*}
Classical_i & \sim & N(m_i,\sigma^2)\\
\alpha_{0j} & \sim & \beta_0 + \eta_{0j} ~,~~ \eta_{0j} \stackrel{iid}{\sim} N(0,\tau_0^2) \\
\alpha_{\mbox{\scriptsize\it guitar},j} & \sim & N(\beta_{\mbox{\scriptsize\it guitar}},\tau_{\mbox{\scriptsize\it guitar}}^2) \\
\alpha_{\mbox{\scriptsize\it piano},j} & \sim &  N(\beta_{\mbox{\scriptsize\it piano}},\tau_{\mbox{\scriptsize\it piano}}^2) \\
\alpha_{\mbox{\scriptsize\it string},j} & \sim & N(\beta_{\mbox{\scriptsize\it string}},\tau_{\mbox{\scriptsize\it string}}^2) \\
\alpha_{\mbox{\scriptsize\it contrary},j} & \sim & N(\beta_{\mbox{\scriptsize\it contrary}},\tau_{\mbox{\scriptsize\it contrary}}^2) \\
\alpha_{\mbox{\scriptsize\it par3rd},j} & \sim & N(\beta_{\mbox{\scriptsize\it par3rd}},\tau_{\mbox{\scriptsize\it par3rd}}^2) \\
\alpha_{\mbox{\scriptsize\it par5th},j} & \sim & N(\beta_{\mbox{\scriptsize\it par5th}},\tau_{\mbox{\scriptsize\it par5th}}^2) \\
\alpha_{\mbox{\scriptsize\it I-IV-V},j} & \sim & N(\beta_{\mbox{\scriptsize\it I-IV-V}},\tau_{\mbox{\scriptsize\it I-IV-V}}^2) \\
\alpha_{\mbox{\scriptsize\it I-V-IV},j} & \sim & N(\beta_{\mbox{\scriptsize\it I-V-IV}},\tau_{\mbox{\scriptsize\it I-V-IV}}^2) \\
\alpha_{\mbox{\scriptsize\it I-V-VI},j} & \sim & N(\beta_{\mbox{\scriptsize\it I-V-VI}},\tau_{\mbox{\scriptsize\it I-V-VI}}^2) \\
\alpha_{\mbox{\scriptsize\it IV-I-V},j} & \sim & N(\beta_{\mbox{\scriptsize\it IV-I-V}},\tau_{\mbox{\scriptsize\it IV-I-V}}^2) 
\end{eqnarray*} A fully correct specification would express the random
vector of \(\alpha\)'s as a multivariate normal, like this for the model
with random effects
\texttt{(Instrument\ +\ Voice\ +\ Harmony\ \textbar{}\textbar{}\ Subject)},
for example:

\small

\[
\hspace*{-0.75in}\left[\begin{array}{c}
\alpha_{0j} \\
\alpha_{\mbox{\scriptsize\it guitar},j} \\
\alpha_{\mbox{\scriptsize\it piano},j}  \\
\alpha_{\mbox{\scriptsize\it string},j} \\
\alpha_{\mbox{\scriptsize\it contrary},j} \\
\alpha_{\mbox{\scriptsize\it par3rd},j} \\
\alpha_{\mbox{\scriptsize\it par5th},j} \\
\alpha_{\mbox{\scriptsize\it I-IV-V},j} \\
\alpha_{\mbox{\scriptsize\it I-V-IV},j} \\
\alpha_{\mbox{\scriptsize\it I-V-VI},j} \\
\alpha_{\mbox{\scriptsize\it IV-I-V},j} 
\end{array}\right] 
\stackrel{iid}{\sim}
N\left(
\left[\begin{array}{c}
\beta_{0j} \\
\beta_{\mbox{\scriptsize\it guitar}} \\
\beta_{\mbox{\scriptsize\it piano}} \\
\beta_{\mbox{\scriptsize\it string}} \\
\beta_{\mbox{\scriptsize\it contrary}} \\
\beta_{\mbox{\scriptsize\it par3rd}} \\
\beta_{\mbox{\scriptsize\it par5th}} \\
\beta_{\mbox{\scriptsize\it I-IV-V}} \\
\beta_{\mbox{\scriptsize\it I-V-IV}} \\
\beta_{\mbox{\scriptsize\it I-V-VI}} \\
\beta_{\mbox{\scriptsize\it IV-I-V}} 
\end{array}\right] ,
\left[\begin{array}{ccccccccccc}
\tau_{0j}  \\
0 & \tau_{\mbox{\scriptsize\it guitar}}^2  \\
0 & \rho_{gp} & \tau_{\mbox{\scriptsize\it piano}}^2  \\
0 & \rho_{gs} &\rho_{ps} & \tau_{\mbox{\scriptsize\it string}}^2 & \multicolumn{7}{c}{\mbox{\em(symmetric)}} \\
0 & 0 & 0 & 0 & \tau_{\mbox{\scriptsize\it contrary}}^2 \\
0 & 0 & 0 & 0 & \rho_{c3} & \tau_{\mbox{\scriptsize\it par3rd}}^2 \\
0 & 0 & 0 & 0 & \rho_{c5} & \rho_{35} & \tau_{\mbox{\scriptsize\it par5th}}^2 \\
0 & 0 & 0 & 0 & 0 & 0 & 0 & \tau_{\mbox{\scriptsize\it I-IV-V}}^2 \\
0 & 0 & 0 & 0 & 0 & 0 & 0 & \rho_{145,154} & \tau_{\mbox{\scriptsize\it I-V-IV}}^2 \\
0 & 0 & 0 & 0 & 0 & 0 & 0 & \rho_{145,156} & \rho_{154,156} & \tau_{\mbox{\scriptsize\it I-V-VI}}^2 \\
0 & 0 & 0 & 0 & 0 & 0 & 0 & \rho_{145,415} & \rho_{154,415} & \rho_{156,415} & \tau_{\mbox{\scriptsize\it IV-I-V}}^2 
\end{array}\right] 
\right)
\] \normalsize

\noindent and the model with random effects
\texttt{(Instrument\ +\ Voice\ +\ Harmony\ \textbar{}\ Subject)} would
look the same, except the 0's in the variance-covariance matrix above
would be replaced with additional correlations.

The estimated \(\beta\)'s, \(\tau\)'s and \(\rho\)'s can be read off the
display() output below.

\begin{itemize}
\item The $\hat\beta$'s and their SE's can be read from the ``coef.est'' and ``coef.se'' columns in the first table in the output
\item The $\hat\tau$'s and $\hat\rho$'s can be read from the second table, labelled ``Error terms:'', with the $\hat\tau$'s in the ``Std.Dev'' column, and the nonzero $\hat\rho$'s in the columns labelled "corr".
\end{itemize}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{display}\NormalTok{(lmer.ni.ivh.c0)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Instrument + Voice + Harmony + ((1 | 
##     Subject) + (0 + Instrument | Subject) + (0 + Voice | Subject) + 
##     (0 + Harmony | Subject)), data = fulldata)
##                  coef.est coef.se
## (Intercept)       4.34     0.23  
## Instrumentpiano   1.37     0.17  
## Instrumentstring  3.12     0.24  
## Voicepar3rd      -0.40     0.08  
## Voicepar5th      -0.36     0.08  
## HarmonyI-V-IV    -0.03     0.09  
## HarmonyI-V-VI     0.77     0.18  
## HarmonyIV-I-V     0.04     0.09  
## 
## Error terms:
##  Groups    Name             Std.Dev. Corr             
##  Subject   (Intercept)      0.00                      
##  Subject.1 Instrumentguitar 1.15                      
##            Instrumentpiano  1.01      0.29            
##            Instrumentstring 0.73     -0.99 -0.29      
##  Subject.2 Voicecontrary    0.72                      
##            Voicepar3rd      0.78     0.92             
##            Voicepar5th      0.79     0.97  0.99       
##  Subject.3 HarmonyI-IV-V    1.09                      
##            HarmonyI-V-IV    1.23     0.99             
##            HarmonyI-V-VI    1.21     0.39  0.48       
##            HarmonyIV-I-V    1.11     1.00  0.98  0.40 
##  Residual                   1.55                      
## ---
## number of obs: 2493, groups: Subject, 70
## AIC = 9952.5, DIC = 9848.8
## deviance = 9868.7
\end{verbatim}

\rm

Because they are design variables in the experiment, the main effects
for the three experimental factors, Instrument, Harmony, and Voice,
should be included in all models for the remainder of this homework,
regardless of what you found about their influence or lack of influence
on ratings. (Interactions among the design variables need not be
included if you do not find them important, but the main effects for the
design variables should \textbf{always} be in your models.)

\hypertarget{problem-3.}{%
\section{Problem 3.}\label{problem-3.}}

\underline{\textit{Person covariates.}} For this problem, begin with
your best model from problem 2.

\hypertarget{problem-3a.}{%
\subsection{Problem 3(a).}\label{problem-3a.}}

Determine which person covariates should be added to the model as fixed
effects.\\
Show a suitable summary of your work, and list the final set of
variables that you would include in the model.
\textit{Hint: Some covariates that are actually
factor variables are coded as numeric.  Be careful to treat them as factors!}
\label{p:3a}

\it

So, again, I'm going to start with the model

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.ni.ivh.c0 }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(Classical }\SpecialCharTok{\textasciitilde{}}\NormalTok{ Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{+} 
\NormalTok{                               (Instrument }\SpecialCharTok{+}\NormalTok{ Voice }\SpecialCharTok{+}\NormalTok{ Harmony }\SpecialCharTok{||}\NormalTok{ Subject), }\AttributeTok{data=}\NormalTok{fulldata)}
\end{Highlighting}
\end{Shaded}

and I note that the ``person covariates'' are level-2 covariates
(persons are the ``groups'' in this analysis) so they will only enter as
fixed effects, not random effects in the \texttt{(...\textbar{}Subject)}
notation.

Here is my strategy:

\begin{itemize}
\item I will work with the smaller data set `nomissdata` since I need a data set that will be the same no matter which person covariates I put in the model, and in the larger data set `fulldata` many of the person covariates have missing values (so the actual data set would change, depending on what covariates are in the model and where the missing values for those covariates were, as I am comparing models).


\item I will start with `fitLMER.fnc` from `library(LMERConvenienceFunctions)`.  This does backwards selection of fixed effects, followed by forward selection of random effects, and then one more pass of backwards selection of fixed effects.  I will have it skip the forward selection of random effects (by not suggesting any new random effects for it to try).  I will also need to specify that `fitLMER.fnc` should use AIC or BIC for variable selection.

\item If `fitLMER.fnc` doesn't produce useful results, I will try some sort of forward selection of fixed effects by hand.  (But it turns out that `fitLMER.fnc` does just fine with my data set `nomissdata`, so I didn't try forward selection by hand...)

\end{itemize}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{library}\NormalTok{(LMERConvenienceFunctions)}

\NormalTok{nomissdata}\SpecialCharTok{$}\NormalTok{Classical }\OtherTok{\textless{}{-}} \FunctionTok{as.numeric}\NormalTok{(nomissdata}\SpecialCharTok{$}\NormalTok{Classical)}
\NormalTok{nomissdata}\SpecialCharTok{$}\NormalTok{Popular }\OtherTok{\textless{}{-}} \FunctionTok{as.numeric}\NormalTok{(nomissdata}\SpecialCharTok{$}\NormalTok{Popular)}
\end{Highlighting}
\end{Shaded}

Here's what happens if I treat all the predictors (except for the three
design factors Instrument, Voice, and Harmony) as continuous. This is
not the right way to handle this problem (the categorical variables
should be converted to factors!), but it is good ``practice'' before
trying to convert predictors to factors..

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{big.predictors }\OtherTok{\textless{}{-}} \FunctionTok{names}\NormalTok{(nomissdata)[}\SpecialCharTok{{-}}\FunctionTok{c}\NormalTok{(}\DecValTok{1}\NormalTok{,}\DecValTok{2}\NormalTok{,}\DecValTok{24}\NormalTok{,}\DecValTok{25}\NormalTok{,}\DecValTok{26}\NormalTok{)]}

\DocumentationTok{\#\# I am omitting the following variables from the set of predictors }
\DocumentationTok{\#\# that I want to start with, for backward selection:}

\FunctionTok{names}\NormalTok{(nomissdata)[}\FunctionTok{c}\NormalTok{(}\DecValTok{1}\NormalTok{,}\DecValTok{2}\NormalTok{,}\DecValTok{24}\NormalTok{,}\DecValTok{25}\NormalTok{,}\DecValTok{26}\NormalTok{)]}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## [1] "X"         "Subject"   "first12"   "Classical" "Popular"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# Now build the model formula...}

\NormalTok{big.fla }\OtherTok{\textless{}{-}} \FunctionTok{paste}\NormalTok{(}\StringTok{"Classical \textasciitilde{}"}\NormalTok{, }\FunctionTok{paste}\NormalTok{(big.predictors,}\AttributeTok{collapse=}\StringTok{" + "}\NormalTok{),}
                 \StringTok{"+ (Instrument + Voice + Harmony || Subject)"}\NormalTok{)}

\NormalTok{lmer.big.cts }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(big.fla, }\AttributeTok{data=}\NormalTok{nomissdata, }\AttributeTok{REML=}\NormalTok{F)}

\DocumentationTok{\#\# We can already get a pretty good idea of what\textquotesingle{}s going on by looking at the }
\DocumentationTok{\#\# display() function and the fixed effects coefficient table from summary()}

\FunctionTok{display}\NormalTok{(lmer.big.cts)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Harmony + Instrument + Voice + Selfdeclare + 
##     OMSI + X16.minus.17 + ConsInstr + ConsNotes + Instr.minus.Notes + 
##     PachListen + ClsListen + KnowRob + KnowAxis + X1990s2000s + 
##     X1990s2000s.minus.1960s1970s + CollegeMusic + NoClass + APTheory + 
##     Composing + PianoPlay + GuitarPlay + ((1 | Subject) + (0 + 
##     Instrument | Subject) + (0 + Voice | Subject) + (0 + Harmony | 
##     Subject)), data = nomissdata, REML = F)
##                              coef.est coef.se
## (Intercept)                   3.68     1.09  
## HarmonyI-V-IV                 0.00     0.12  
## HarmonyI-V-VI                 0.85     0.23  
## HarmonyIV-I-V                 0.06     0.12  
## Instrumentpiano               1.65     0.23  
## Instrumentstring              3.59     0.31  
## Voicepar3rd                  -0.41     0.12  
## Voicepar5th                  -0.30     0.11  
## Selfdeclare                  -0.68     0.23  
## OMSI                          0.00     0.00  
## X16.minus.17                 -0.13     0.05  
## ConsInstr                    -0.05     0.11  
## ConsNotes                    -0.13     0.10  
## PachListen                    0.14     0.16  
## ClsListen                     0.22     0.11  
## KnowRob                      -0.01     0.08  
## KnowAxis                      0.08     0.07  
## X1990s2000s                   0.08     0.13  
## X1990s2000s.minus.1960s1970s  0.05     0.10  
## CollegeMusic                  0.00     0.37  
## NoClass                      -0.15     0.13  
## APTheory                      0.53     0.38  
## Composing                     0.17     0.12  
## PianoPlay                     0.32     0.09  
## GuitarPlay                   -0.04     0.14  
## 
## Error terms:
##  Groups    Name             Std.Dev. Corr             
##  Subject   (Intercept)      0.00                      
##  Subject.1 Instrumentguitar 0.54                      
##            Instrumentpiano  1.04     -0.51            
##            Instrumentstring 1.40     -0.96  0.25      
##  Subject.2 Voicecontrary    0.22                      
##            Voicepar3rd      0.39     -0.01            
##            Voicepar5th      0.23      0.03  1.00      
##  Subject.3 HarmonyI-IV-V    0.62                      
##            HarmonyI-V-IV    0.96     0.99             
##            HarmonyI-V-VI    1.52     0.50  0.60       
##            HarmonyIV-I-V    0.55     0.88  0.91  0.67 
##  Residual                   1.57                      
## ---
## number of obs: 1541, groups: Subject, 43
## AIC = 6216.4, DIC = 6118.4
## deviance = 6118.4
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lmer.big.cts)}\SpecialCharTok{$}\NormalTok{coef}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                                   Estimate  Std. Error      t value
## (Intercept)                   3.6813739923 1.085988927  3.389881702
## HarmonyI-V-IV                -0.0031738739 0.124861735 -0.025419108
## HarmonyI-V-VI                 0.8532781981 0.230920855  3.695111032
## HarmonyIV-I-V                 0.0582520439 0.121740780  0.478492450
## Instrumentpiano               1.6494238199 0.233951478  7.050281695
## Instrumentstring              3.5881401037 0.308614177 11.626621112
## Voicepar3rd                  -0.4056778691 0.119501771 -3.394743569
## Voicepar5th                  -0.3018293244 0.108745156 -2.775565702
## Selfdeclare                  -0.6766228577 0.228199869 -2.965044905
## OMSI                          0.0022369805 0.001055517  2.119321242
## X16.minus.17                 -0.1251491628 0.052235978 -2.395842242
## ConsInstr                    -0.0547756467 0.107163024 -0.511143158
## ConsNotes                    -0.1256052767 0.096036916 -1.307885369
## PachListen                    0.1364695462 0.159052370  0.858016427
## ClsListen                     0.2248604332 0.109752191  2.048801317
## KnowRob                      -0.0050796118 0.082631982 -0.061472709
## KnowAxis                      0.0821291703 0.065345085  1.256853064
## X1990s2000s                   0.0796342115 0.126942851  0.627323328
## X1990s2000s.minus.1960s1970s  0.0491254477 0.104612685  0.469593601
## CollegeMusic                  0.0008036376 0.366443030  0.002193076
## NoClass                      -0.1482552060 0.125022307 -1.185830033
## APTheory                      0.5256146986 0.378816713  1.387517184
## Composing                     0.1747574077 0.123939930  1.410016995
## PianoPlay                     0.3161581749 0.085331519  3.705057366
## GuitarPlay                   -0.0364482595 0.135695743 -0.268602821
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# now let\textquotesingle{}s see what fitLMER.fnc does with it...}

\NormalTok{lmer.AIC.cts }\OtherTok{\textless{}{-}} \FunctionTok{fitLMER.fnc}\NormalTok{(lmer.big.cts, }\AttributeTok{backfit.on=}\StringTok{"t"}\NormalTok{, }\AttributeTok{method=}\StringTok{"AIC"}\NormalTok{, }\AttributeTok{prune.ranefs=}\NormalTok{F,}
                        \AttributeTok{log.file.name=}\StringTok{"lmerAICcts.log"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Warning in fitLMER.fnc(lmer.big.cts, backfit.on = "t", method = "AIC", prune.ranefs = F, : Argument "ran.effects" is empty, which means you will not be forward-fitting the random effect structure of your model. You could just as well run function "bfFixefLMER_F.fnc" or "bfFixefLMER_t.fnc".
## TRUE
\end{verbatim}

\begin{verbatim}
## ======================================================
## ===              backfitting fixed effects         ===
## ======================================================
## setting REML to FALSE
## processing model terms of interaction level 1 
##   iteration 1 
##      t-value for term "CollegeMusic" = 0.002193076 < 2 
##      not part of higher-order interaction
##      AIC simple = 6215; AIC complex = 6216; decrease = -1 < 5
##      removing term
##   iteration 2 
##      t-value for term "KnowRob" = 0.1803665 < 2 
##      not part of higher-order interaction
##      AIC simple = 6213; AIC complex = 6215; decrease = -2 < 5
##      removing term
##   iteration 3 
##      t-value for term "GuitarPlay" = 0.2851146 < 2 
##      not part of higher-order interaction
##      AIC simple = 6210; AIC complex = 6213; decrease = -2 < 5
##      removing term
##   iteration 4 
##      t-value for term "ConsInstr" = 0.4599885 < 2 
##      not part of higher-order interaction
##      AIC simple = 6210; AIC complex = 6210; decrease = 0 < 5
##      removing term
##   iteration 5 
##      t-value for term "Instr.minus.Notes" = 0.4599279 < 2 
##      not part of higher-order interaction
##      AIC simple = 6209; AIC complex = 6210; decrease = -1 < 5
##      removing term
##   iteration 6 
##      t-value for term "X1990s2000s" = 0.6474848 < 2 
##      not part of higher-order interaction
##      AIC simple = 6207; AIC complex = 6209; decrease = -2 < 5
##      removing term
##   iteration 7 
##      t-value for term "PachListen" = 0.8448703 < 2 
##      not part of higher-order interaction
##      AIC simple = 6205; AIC complex = 6207; decrease = -1 < 5
##      removing term
##   iteration 8 
##      t-value for term "X1990s2000s.minus.1960s1970s" = 0.8392038 < 2 
##      not part of higher-order interaction
##      AIC simple = 6204; AIC complex = 6205; decrease = -2 < 5
##      removing term
##   iteration 9 
##      t-value for term "KnowAxis" = 1.418387 < 2 
##      not part of higher-order interaction
##      AIC simple = 6204; AIC complex = 6204; decrease = 0 < 5
##      removing term
##   iteration 10 
##      t-value for term "Composing" = 1.681283 < 2 
##      not part of higher-order interaction
##      AIC simple = 6205; AIC complex = 6204; decrease = 1 < 5
##      removing term
##   iteration 11 
##      t-value for term "NoClass" = 1.093392 < 2 
##      not part of higher-order interaction
##      AIC simple = 6203; AIC complex = 6205; decrease = -2 < 5
##      removing term
##   iteration 12 
##      t-value for term "APTheory" = 1.786381 < 2 
##      not part of higher-order interaction
##      AIC simple = 6204; AIC complex = 6203; decrease = 1 < 5
##      removing term
## ======================================================
## ===            forwardfitting random effects       ===
## ======================================================
##  ===       random slopes       ===
## ======================================================
## ===            re-backfitting fixed effects        ===
## ======================================================
## setting REML to FALSE
## processing model terms of interaction level 1 
##   all terms of interaction level 1 significant
## resetting REML to TRUE
## log file is lmerAICcts.log
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{display}\NormalTok{(lmer.AIC.cts)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Harmony + Instrument + Voice + Selfdeclare + 
##     OMSI + X16.minus.17 + ConsNotes + ClsListen + PianoPlay + 
##     (1 | Subject) + (0 + Instrument | Subject) + (0 + Voice | 
##     Subject) + (0 + Harmony | Subject), data = nomissdata, REML = TRUE)
##                  coef.est coef.se
## (Intercept)       4.85     0.43  
## HarmonyI-V-IV     0.00     0.12  
## HarmonyI-V-VI     0.85     0.23  
## HarmonyIV-I-V     0.06     0.12  
## Instrumentpiano   1.65     0.24  
## Instrumentstring  3.59     0.31  
## Voicepar3rd      -0.41     0.12  
## Voicepar5th      -0.30     0.11  
## Selfdeclare      -0.63     0.23  
## OMSI              0.00     0.00  
## X16.minus.17     -0.17     0.05  
## ConsNotes        -0.20     0.08  
## ClsListen         0.23     0.09  
## PianoPlay         0.31     0.09  
## 
## Error terms:
##  Groups    Name             Std.Dev. Corr             
##  Subject   (Intercept)      0.00                      
##  Subject.1 Instrumentguitar 0.65                      
##            Instrumentpiano  1.13     -0.21            
##            Instrumentstring 1.30     -1.00  0.21      
##  Subject.2 Voicecontrary    0.58                      
##            Voicepar3rd      0.55     0.67             
##            Voicepar5th      0.39     0.82  0.97       
##  Subject.3 HarmonyI-IV-V    0.65                      
##            HarmonyI-V-IV    0.96     0.99             
##            HarmonyI-V-VI    1.57     0.53  0.62       
##            HarmonyIV-I-V    0.63     0.87  0.86  0.67 
##  Residual                   1.57                      
## ---
## number of obs: 1541, groups: Subject, 43
## AIC = 6248.7, DIC = 6083.5
## deviance = 6128.1
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lmer.AIC.cts)}\SpecialCharTok{$}\NormalTok{coef}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                      Estimate   Std. Error     t value
## (Intercept)       4.850128815 0.4306915327 11.26125880
## HarmonyI-V-IV    -0.003130549 0.1232592276 -0.02539809
## HarmonyI-V-VI     0.853254887 0.2332438982  3.65820883
## HarmonyIV-I-V     0.058383810 0.1235626617  0.47250366
## Instrumentpiano   1.649414704 0.2366808516  6.96894021
## Instrumentstring  3.588159942 0.3122461530 11.49144643
## Voicepar3rd      -0.405581399 0.1205039746 -3.36570972
## Voicepar5th      -0.301801014 0.1110063662 -2.71877212
## Selfdeclare      -0.626112117 0.2340433074 -2.67519770
## OMSI              0.002632546 0.0009494972  2.77256884
## X16.minus.17     -0.169918687 0.0494846154 -3.43376796
## ConsNotes        -0.201800805 0.0842397583 -2.39555299
## ClsListen         0.230143925 0.0935048438  2.46130484
## PianoPlay         0.305140544 0.0931992356  3.27406702
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.BIC.cts }\OtherTok{\textless{}{-}} \FunctionTok{fitLMER.fnc}\NormalTok{(lmer.big.cts, }\AttributeTok{backfit.on=}\StringTok{"t"}\NormalTok{, }\AttributeTok{method=}\StringTok{"BIC"}\NormalTok{, }\AttributeTok{prune.ranefs=}\NormalTok{F,}
                        \AttributeTok{log.file.name=}\StringTok{"lmerBICcts.log"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Warning in fitLMER.fnc(lmer.big.cts, backfit.on = "t", method = "BIC", prune.ranefs = F, : Argument "ran.effects" is empty, which means you will not be forward-fitting the random effect structure of your model. You could just as well run function "bfFixefLMER_F.fnc" or "bfFixefLMER_t.fnc".
## TRUE
\end{verbatim}

\begin{verbatim}
## ======================================================
## ===              backfitting fixed effects         ===
## ======================================================
## setting REML to FALSE
## processing model terms of interaction level 1 
##   iteration 1 
##      t-value for term "CollegeMusic" = 0.002193076 < 2 
##      not part of higher-order interaction
##      BIC simple = 6471; BIC complex = 6478; decrease = -7 < 5
##      removing term
##   iteration 2 
##      t-value for term "KnowRob" = 0.1803665 < 2 
##      not part of higher-order interaction
##      BIC simple = 6464; BIC complex = 6471; decrease = -8 < 5
##      removing term
##   iteration 3 
##      t-value for term "GuitarPlay" = 0.2851146 < 2 
##      not part of higher-order interaction
##      BIC simple = 6456; BIC complex = 6464; decrease = -7 < 5
##      removing term
##   iteration 4 
##      t-value for term "ConsInstr" = 0.4599885 < 2 
##      not part of higher-order interaction
##      BIC simple = 6456; BIC complex = 6456; decrease = 0 < 5
##      removing term
##   iteration 5 
##      t-value for term "Instr.minus.Notes" = 0.4599279 < 2 
##      not part of higher-order interaction
##      BIC simple = 6449; BIC complex = 6456; decrease = -7 < 5
##      removing term
##   iteration 6 
##      t-value for term "X1990s2000s" = 0.6474848 < 2 
##      not part of higher-order interaction
##      BIC simple = 6442; BIC complex = 6449; decrease = -8 < 5
##      removing term
##   iteration 7 
##      t-value for term "PachListen" = 0.8448703 < 2 
##      not part of higher-order interaction
##      BIC simple = 6435; BIC complex = 6442; decrease = -7 < 5
##      removing term
##   iteration 8 
##      t-value for term "X1990s2000s.minus.1960s1970s" = 0.8392038 < 2 
##      not part of higher-order interaction
##      BIC simple = 6428; BIC complex = 6435; decrease = -7 < 5
##      removing term
##   iteration 9 
##      t-value for term "KnowAxis" = 1.418387 < 2 
##      not part of higher-order interaction
##      BIC simple = 6423; BIC complex = 6428; decrease = -6 < 5
##      removing term
##   iteration 10 
##      t-value for term "Composing" = 1.681283 < 2 
##      not part of higher-order interaction
##      BIC simple = 6418; BIC complex = 6423; decrease = -4 < 5
##      removing term
##   iteration 11 
##      t-value for term "NoClass" = 1.093392 < 2 
##      not part of higher-order interaction
##      BIC simple = 6411; BIC complex = 6418; decrease = -7 < 5
##      removing term
##   iteration 12 
##      t-value for term "APTheory" = 1.786381 < 2 
##      not part of higher-order interaction
##      BIC simple = 6407; BIC complex = 6411; decrease = -4 < 5
##      removing term
## ======================================================
## ===            forwardfitting random effects       ===
## ======================================================
##  ===       random slopes       ===
## ======================================================
## ===            re-backfitting fixed effects        ===
## ======================================================
## setting REML to FALSE
## processing model terms of interaction level 1 
##   all terms of interaction level 1 significant
## resetting REML to TRUE
## log file is lmerBICcts.log
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{display}\NormalTok{(lmer.BIC.cts)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Harmony + Instrument + Voice + Selfdeclare + 
##     OMSI + X16.minus.17 + ConsNotes + ClsListen + PianoPlay + 
##     (1 | Subject) + (0 + Instrument | Subject) + (0 + Voice | 
##     Subject) + (0 + Harmony | Subject), data = nomissdata, REML = TRUE)
##                  coef.est coef.se
## (Intercept)       4.85     0.43  
## HarmonyI-V-IV     0.00     0.12  
## HarmonyI-V-VI     0.85     0.23  
## HarmonyIV-I-V     0.06     0.12  
## Instrumentpiano   1.65     0.24  
## Instrumentstring  3.59     0.31  
## Voicepar3rd      -0.41     0.12  
## Voicepar5th      -0.30     0.11  
## Selfdeclare      -0.63     0.23  
## OMSI              0.00     0.00  
## X16.minus.17     -0.17     0.05  
## ConsNotes        -0.20     0.08  
## ClsListen         0.23     0.09  
## PianoPlay         0.31     0.09  
## 
## Error terms:
##  Groups    Name             Std.Dev. Corr             
##  Subject   (Intercept)      0.00                      
##  Subject.1 Instrumentguitar 0.65                      
##            Instrumentpiano  1.13     -0.21            
##            Instrumentstring 1.30     -1.00  0.21      
##  Subject.2 Voicecontrary    0.58                      
##            Voicepar3rd      0.55     0.67             
##            Voicepar5th      0.39     0.82  0.97       
##  Subject.3 HarmonyI-IV-V    0.65                      
##            HarmonyI-V-IV    0.96     0.99             
##            HarmonyI-V-VI    1.57     0.53  0.62       
##            HarmonyIV-I-V    0.63     0.87  0.86  0.67 
##  Residual                   1.57                      
## ---
## number of obs: 1541, groups: Subject, 43
## AIC = 6248.7, DIC = 6083.5
## deviance = 6128.1
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lmer.BIC.cts)}\SpecialCharTok{$}\NormalTok{coef}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                      Estimate   Std. Error     t value
## (Intercept)       4.850128815 0.4306915327 11.26125880
## HarmonyI-V-IV    -0.003130549 0.1232592276 -0.02539809
## HarmonyI-V-VI     0.853254887 0.2332438982  3.65820883
## HarmonyIV-I-V     0.058383810 0.1235626617  0.47250366
## Instrumentpiano   1.649414704 0.2366808516  6.96894021
## Instrumentstring  3.588159942 0.3122461530 11.49144643
## Voicepar3rd      -0.405581399 0.1205039746 -3.36570972
## Voicepar5th      -0.301801014 0.1110063662 -2.71877212
## Selfdeclare      -0.626112117 0.2340433074 -2.67519770
## OMSI              0.002632546 0.0009494972  2.77256884
## X16.minus.17     -0.169918687 0.0494846154 -3.43376796
## ConsNotes        -0.201800805 0.0842397583 -2.39555299
## ClsListen         0.230143925 0.0935048438  2.46130484
## PianoPlay         0.305140544 0.0931992356  3.27406702
\end{verbatim}

Both AIC and BIC kept all three design factors (I would have added them
back in if they hadn't!), and in fact they also kept the same person
covariates in this case: ``Selfdeclare'', ``OMSI'', ``X16.minus.17'',
``ConsNotes'', ``ClsListen'' and ``PianoPlay''. Also the correlations
among the random effects look very similar to the correlations from our
work from problem 2.

Now let's try this after converting the predictors that should be
categorical to categorical variables:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{nomissdata.cat }\OtherTok{\textless{}{-}}\NormalTok{ nomissdata}

\FunctionTok{names}\NormalTok{(nomissdata.cat)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##  [1] "X"                            "Subject"                     
##  [3] "Harmony"                      "Instrument"                  
##  [5] "Voice"                        "Selfdeclare"                 
##  [7] "OMSI"                         "X16.minus.17"                
##  [9] "ConsInstr"                    "ConsNotes"                   
## [11] "Instr.minus.Notes"            "PachListen"                  
## [13] "ClsListen"                    "KnowRob"                     
## [15] "KnowAxis"                     "X1990s2000s"                 
## [17] "X1990s2000s.minus.1960s1970s" "CollegeMusic"                
## [19] "NoClass"                      "APTheory"                    
## [21] "Composing"                    "PianoPlay"                   
## [23] "GuitarPlay"                   "first12"                     
## [25] "Classical"                    "Popular"
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# Let\textquotesingle{}s see which variables "look like" they should be categorical:}

\FunctionTok{apply}\NormalTok{(nomissdata.cat[,}\DecValTok{8}\SpecialCharTok{:}\DecValTok{24}\NormalTok{],}\DecValTok{2}\NormalTok{,table)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## $X16.minus.17
## 
## -0.5 -1.0 -2.0 -4.0  0.0  1.0  2.0  3.0  4.0  5.0  6.0  7.0  9.0 
##   36  252   36   36  431  144  180  108   30   72   36   72  108 
## 
## $ConsInstr
## 
## 0.00 0.67 1.00 1.67 2.33 2.67 3.00 3.33 3.67 4.00 4.33 5.00 
##   36   36  173  108  108   36  324   36  108   36  252  288 
## 
## $ConsNotes
## 
##   0   1   3   4   5 
## 360 215 462  36 468 
## 
## $Instr.minus.Notes
## 
## -0.67 -1.00 -1.33 -2.00 -4.00  0.00  0.67  1.00  1.33  2.00  2.33  2.67  3.00 
##   144    36   108   102    36   395   144    36   108   180    72    36    36 
##  3.33  4.00  4.33 
##    36    36    36 
## 
## $PachListen
## 
##    2    3    4    5 
##   72  108   36 1325 
## 
## $ClsListen
## 
##   0   1   3   4   5 
## 216 576 498  36 215 
## 
## $KnowRob
## 
##    0    1    5 
## 1151  144  246 
## 
## $KnowAxis
## 
##    0    1    5 
## 1109   36  396 
## 
## $X1990s2000s
## 
##    0    2    3    4    5 
##  108  108  252   72 1001 
## 
## $X1990s2000s.minus.1960s1970s
## 
##  -2  -3   0   1   2   3   4   5 
##  36  36 389 108 288 504  36 144 
## 
## $CollegeMusic
## 
##    0    1 
##  324 1217 
## 
## $NoClass
## 
##   0   1   2   3   4   8 
## 611 750  72  36  36  36 
## 
## $APTheory
## 
##    0    1 
## 1187  354 
## 
## $Composing
## 
##   0   1   2   3   4   5 
## 828 216 173 144 144  36 
## 
## $PianoPlay
## 
##   0   1   4   5 
## 828 425 108 180 
## 
## $GuitarPlay
## 
##    0    1    2    4    5 
## 1152  107   36   72  174 
## 
## $first12
## 
## guitar  piano string 
##    468    467    606
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# It\textquotesingle{}s a little hard to judge exactly which variables should be categorical for sure.}
\DocumentationTok{\#\# I think the ones that definitely should be are: "PachListen", "ClsListen",}
\DocumentationTok{\#\# "KnowRob", "KnowAxis", "CollegeMusic", "APTheory", "Composing",}
\DocumentationTok{\#\# "PianoPlay" and "GuitarPlay". The variable "NoClass" (number of music classes }
\DocumentationTok{\#\# taken) should definitely be quantitative.   The others are kind of ambiguous to }
\DocumentationTok{\#\# me (either because Jimemez applied arithmetic to them, or they take on values}
\DocumentationTok{\#\# other than non{-}negative integers), and I am going to leave them as numeric/quantitative.}

\NormalTok{cat.vars }\OtherTok{\textless{}{-}} \FunctionTok{c}\NormalTok{(}\StringTok{"PachListen"}\NormalTok{, }\StringTok{"ClsListen"}\NormalTok{, }\StringTok{"KnowRob"}\NormalTok{, }\StringTok{"KnowAxis"}\NormalTok{, }\StringTok{"CollegeMusic"}\NormalTok{,}
              \StringTok{"APTheory"}\NormalTok{, }\StringTok{"Composing"}\NormalTok{, }\StringTok{"PianoPlay"}\NormalTok{, }\StringTok{"GuitarPlay"}\NormalTok{)}

\NormalTok{nomissdata.cat[,cat.vars] }\OtherTok{\textless{}{-}} \FunctionTok{apply}\NormalTok{(nomissdata.cat[,cat.vars],}\DecValTok{2}\NormalTok{,as.factor)}

\FunctionTok{str}\NormalTok{(nomissdata.cat)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 'data.frame':    1541 obs. of  26 variables:
##  $ X                           : int  1 2 3 4 5 6 7 8 9 10 ...
##  $ Subject                     : chr  "15" "15" "15" "15" ...
##  $ Harmony                     : chr  "I-IV-V" "I-IV-V" "I-IV-V" "I-IV-V" ...
##  $ Instrument                  : chr  "guitar" "guitar" "guitar" "piano" ...
##  $ Voice                       : chr  "contrary" "par3rd" "par5th" "contrary" ...
##  $ Selfdeclare                 : int  5 5 5 5 5 5 5 5 5 5 ...
##  $ OMSI                        : int  734 734 734 734 734 734 734 734 734 734 ...
##  $ X16.minus.17                : num  5 5 5 5 5 5 5 5 5 5 ...
##  $ ConsInstr                   : num  4.33 4.33 4.33 4.33 4.33 4.33 4.33 4.33 4.33 4.33 ...
##  $ ConsNotes                   : int  5 5 5 5 5 5 5 5 5 5 ...
##  $ Instr.minus.Notes           : num  -0.67 -0.67 -0.67 -0.67 -0.67 -0.67 -0.67 -0.67 -0.67 -0.67 ...
##  $ PachListen                  : chr  "5" "5" "5" "5" ...
##  $ ClsListen                   : chr  "4" "4" "4" "4" ...
##  $ KnowRob                     : chr  "0" "0" "0" "0" ...
##  $ KnowAxis                    : chr  "0" "0" "0" "0" ...
##  $ X1990s2000s                 : int  5 5 5 5 5 5 5 5 5 5 ...
##  $ X1990s2000s.minus.1960s1970s: int  2 2 2 2 2 2 2 2 2 2 ...
##  $ CollegeMusic                : chr  "0" "0" "0" "0" ...
##  $ NoClass                     : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ APTheory                    : chr  "0" "0" "0" "0" ...
##  $ Composing                   : chr  "4" "4" "4" "4" ...
##  $ PianoPlay                   : chr  "1" "1" "1" "1" ...
##  $ GuitarPlay                  : chr  "5" "5" "5" "5" ...
##  $ first12                     : chr  "string" "string" "string" "string" ...
##  $ Classical                   : num  3 3 1 3 2 8 10 6 5 1 ...
##  $ Popular                     : num  9 7 8 7 8 3 1 4 5 8 ...
\end{verbatim}

Now I am ready to try \texttt{fitLMER.fnc} on the new data set with
categorical variables\ldots{}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.big.cat }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(big.fla, }\AttributeTok{data=}\NormalTok{nomissdata.cat, }\AttributeTok{REML=}\NormalTok{F)}

\DocumentationTok{\#\# It\textquotesingle{}s harder to see what\textquotesingle{}s going on by just listing the fitted parameters, }
\DocumentationTok{\#\# because of all the levels of the categorical variables...}

\FunctionTok{display}\NormalTok{(lmer.big.cat)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Harmony + Instrument + Voice + Selfdeclare + 
##     OMSI + X16.minus.17 + ConsInstr + ConsNotes + Instr.minus.Notes + 
##     PachListen + ClsListen + KnowRob + KnowAxis + X1990s2000s + 
##     X1990s2000s.minus.1960s1970s + CollegeMusic + NoClass + APTheory + 
##     Composing + PianoPlay + GuitarPlay + ((1 | Subject) + (0 + 
##     Instrument | Subject) + (0 + Voice | Subject) + (0 + Harmony | 
##     Subject)), data = nomissdata.cat, REML = F)
##                              coef.est coef.se
## (Intercept)                   3.95     0.88  
## HarmonyI-V-IV                 0.00     0.12  
## HarmonyI-V-VI                 0.85     0.23  
## HarmonyIV-I-V                 0.06     0.12  
## Instrumentpiano               1.65     0.24  
## Instrumentstring              3.59     0.31  
## Voicepar3rd                  -0.41     0.12  
## Voicepar5th                  -0.30     0.11  
## Selfdeclare                   0.03     0.30  
## OMSI                          0.00     0.00  
## X16.minus.17                 -0.16     0.03  
## ConsInstr                     0.10     0.09  
## ConsNotes                    -0.36     0.06  
## PachListen3                  -0.77     0.50  
## PachListen4                   1.76     0.73  
## PachListen5                   0.10     0.52  
## ClsListen1                   -0.43     0.27  
## ClsListen3                    0.28     0.35  
## ClsListen4                    4.48     1.32  
## ClsListen5                   -0.71     0.50  
## KnowRob1                      0.72     0.59  
## KnowRob5                      1.45     0.50  
## KnowAxis1                    -8.30     1.35  
## KnowAxis5                    -0.71     0.29  
## X1990s2000s                  -0.12     0.14  
## X1990s2000s.minus.1960s1970s  0.18     0.09  
## CollegeMusic1                -0.40     0.25  
## NoClass                       0.70     0.20  
## APTheory1                     2.69     0.34  
## Composing1                   -0.76     0.26  
## Composing2                    0.10     0.61  
## Composing3                   -1.04     0.59  
## Composing4                    0.86     0.42  
## Composing5                   -2.43     1.22  
## PianoPlay1                    0.42     0.39  
## PianoPlay4                    1.44     0.58  
## PianoPlay5                   -0.65     0.46  
## GuitarPlay1                   2.13     0.76  
## GuitarPlay2                   1.70     0.88  
## GuitarPlay4                   3.63     1.14  
## GuitarPlay5                  -4.02     0.76  
## 
## Error terms:
##  Groups    Name             Std.Dev. Corr              
##  Subject   (Intercept)      0.00                       
##  Subject.1 Instrumentguitar 0.92                       
##            Instrumentpiano  1.19      0.11             
##            Instrumentstring 1.01     -0.99  0.03       
##  Subject.2 Voicecontrary    0.29                       
##            Voicepar3rd      0.17     -1.00             
##            Voicepar5th      0.03     -1.00  1.00       
##  Subject.3 HarmonyI-IV-V    0.33                       
##            HarmonyI-V-IV    0.34      0.75             
##            HarmonyI-V-VI    1.00     -0.95 -0.50       
##            HarmonyIV-I-V    0.30      0.45 -0.25 -0.71 
##  Residual                   1.56                       
## ---
## number of obs: 1541, groups: Subject, 43
## AIC = 6164.1, DIC = 6034.1
## deviance = 6034.1
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lmer.big.cat)}\SpecialCharTok{$}\NormalTok{coef}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                                   Estimate  Std. Error     t value
## (Intercept)                   3.9531580972 0.878152635  4.50167538
## HarmonyI-V-IV                -0.0028357045 0.117893394 -0.02405312
## HarmonyI-V-VI                 0.8532210152 0.229921549  3.71092235
## HarmonyIV-I-V                 0.0586270806 0.122915676  0.47696992
## Instrumentpiano               1.6496148985 0.236975428  6.96112215
## Instrumentstring              3.5883033249 0.308656349 11.62556135
## Voicepar3rd                  -0.4054215732 0.119452160 -3.39400790
## Voicepar5th                  -0.3018048466 0.108672545 -2.77719498
## Selfdeclare                   0.0302706736 0.299334912  0.10112644
## OMSI                          0.0004636591 0.001194244  0.38824482
## X16.minus.17                 -0.1626115439 0.033070560 -4.91710884
## ConsInstr                     0.1004254755 0.086418322  1.16208546
## ConsNotes                    -0.3604394380 0.058701062 -6.14025409
## PachListen3                  -0.7658539821 0.495453301 -1.54576421
## PachListen4                   1.7611290015 0.734606180  2.39737842
## PachListen5                   0.0986276615 0.516798655  0.19084349
## ClsListen1                   -0.4289116110 0.273876885 -1.56607452
## ClsListen3                    0.2823756938 0.347065046  0.81361029
## ClsListen4                    4.4808564514 1.324553790  3.38291769
## ClsListen5                   -0.7135216369 0.502462993 -1.42004814
## KnowRob1                      0.7200038045 0.593964819  1.21219941
## KnowRob5                      1.4549009200 0.504369317  2.88459442
## KnowAxis1                    -8.3018600221 1.350202188 -6.14860507
## KnowAxis5                    -0.7114059669 0.291256008 -2.44254521
## X1990s2000s                  -0.1226034209 0.144256740 -0.84989735
## X1990s2000s.minus.1960s1970s  0.1832978776 0.090033396  2.03588763
## CollegeMusic1                -0.3985025149 0.254006985 -1.56886439
## NoClass                       0.7042037001 0.202450395  3.47840122
## APTheory1                     2.6909002738 0.336951357  7.98602000
## Composing1                   -0.7580432073 0.257247906 -2.94674199
## Composing2                    0.1042501140 0.613230931  0.17000140
## Composing3                   -1.0446691822 0.589801023 -1.77122308
## Composing4                    0.8608696357 0.423941284  2.03063412
## Composing5                   -2.4291041166 1.216977436 -1.99601410
## PianoPlay1                    0.4233711297 0.393097314  1.07701354
## PianoPlay4                    1.4372965685 0.579951505  2.47830475
## PianoPlay5                   -0.6497473871 0.456819846 -1.42232741
## GuitarPlay1                   2.1346898665 0.762377223  2.80004413
## GuitarPlay2                   1.7016151294 0.876012296  1.94245576
## GuitarPlay4                   3.6339724496 1.139673435  3.18860854
## GuitarPlay5                  -4.0170598820 0.759369488 -5.28999380
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# so let\textquotesingle{}s see what fitLMER.fnc does with it...}

\NormalTok{lmer.AIC.cat }\OtherTok{\textless{}{-}} \FunctionTok{fitLMER.fnc}\NormalTok{(lmer.big.cat, }\AttributeTok{backfit.on=}\StringTok{"t"}\NormalTok{, }\AttributeTok{method=}\StringTok{"AIC"}\NormalTok{, }\AttributeTok{prune.ranefs=}\NormalTok{F,}
                        \AttributeTok{log.file.name=}\StringTok{"lmerAICcat.log"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Warning in fitLMER.fnc(lmer.big.cat, backfit.on = "t", method = "AIC", prune.ranefs = F, : Argument "ran.effects" is empty, which means you will not be forward-fitting the random effect structure of your model. You could just as well run function "bfFixefLMER_F.fnc" or "bfFixefLMER_t.fnc".
## TRUE
\end{verbatim}

\begin{verbatim}
## ======================================================
## ===              backfitting fixed effects         ===
## ======================================================
## setting REML to FALSE
## processing model terms of interaction level 1 
##   iteration 1 
##      t-value for term "Selfdeclare" = 0.1011264 < 2 
##      not part of higher-order interaction
##      AIC simple = 6162; AIC complex = 6164; decrease = -2 < 5
##      removing term
##   iteration 2 
##      t-value for term "OMSI" = 1.105238 < 2 
##      not part of higher-order interaction
##      AIC simple = 6161; AIC complex = 6162; decrease = -1 < 5
##      removing term
##   iteration 3 
##      t-value for term "ConsInstr" = 1.149557 < 2 
##      not part of higher-order interaction
##      AIC simple = 6161; AIC complex = 6161; decrease = 0 < 5
##      removing term
##   iteration 4 
##      t-value for term "Instr.minus.Notes" = 1.148654 < 2 
##      not part of higher-order interaction
##      AIC simple = 6160; AIC complex = 6161; decrease = -1 < 5
##      removing term
##   iteration 5 
##      t-value for term "CollegeMusic" = 1.55688 < 2 
##      not part of higher-order interaction
##      AIC simple = 6160; AIC complex = 6160; decrease = 0 < 5
##      removing term
##   iteration 6 
##      t-value for term "PianoPlay" = 1.565929 < 2 
##      not part of higher-order interaction
##      AIC simple = 6157; AIC complex = 6160; decrease = -3 < 5
##      removing term
## ======================================================
## ===            forwardfitting random effects       ===
## ======================================================
##  ===       random slopes       ===
## ======================================================
## ===            re-backfitting fixed effects        ===
## ======================================================
## setting REML to FALSE
## processing model terms of interaction level 1 
##   all terms of interaction level 1 significant
## resetting REML to TRUE
## log file is lmerAICcat.log
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{display}\NormalTok{(lmer.AIC.cat)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Harmony + Instrument + Voice + X16.minus.17 + 
##     ConsNotes + PachListen + ClsListen + KnowRob + KnowAxis + 
##     X1990s2000s + X1990s2000s.minus.1960s1970s + NoClass + APTheory + 
##     Composing + GuitarPlay + (1 | Subject) + (0 + Instrument | 
##     Subject) + (0 + Voice | Subject) + (0 + Harmony | Subject), 
##     data = nomissdata.cat, REML = TRUE)
##                              coef.est coef.se
## (Intercept)                   4.56     0.82  
## HarmonyI-V-IV                 0.00     0.12  
## HarmonyI-V-VI                 0.85     0.23  
## HarmonyIV-I-V                 0.06     0.12  
## Instrumentpiano               1.65     0.24  
## Instrumentstring              3.59     0.31  
## Voicepar3rd                  -0.41     0.12  
## Voicepar5th                  -0.30     0.11  
## X16.minus.17                 -0.18     0.04  
## ConsNotes                    -0.30     0.07  
## PachListen3                  -1.30     0.58  
## PachListen4                   1.97     0.82  
## PachListen5                  -0.51     0.46  
## ClsListen1                   -0.09     0.31  
## ClsListen3                    0.69     0.37  
## ClsListen4                    6.43     0.97  
## ClsListen5                   -0.05     0.48  
## KnowRob1                     -0.36     0.43  
## KnowRob5                      1.54     0.38  
## KnowAxis1                    -6.30     1.21  
## KnowAxis5                    -0.49     0.29  
## X1990s2000s                  -0.23     0.10  
## X1990s2000s.minus.1960s1970s  0.23     0.08  
## NoClass                       0.74     0.16  
## APTheory1                     2.64     0.33  
## Composing1                   -0.43     0.27  
## Composing2                    1.06     0.42  
## Composing3                    0.04     0.38  
## Composing4                    0.71     0.48  
## Composing5                   -2.76     1.10  
## GuitarPlay1                   0.87     0.53  
## GuitarPlay2                   3.36     0.50  
## GuitarPlay4                   2.06     0.88  
## GuitarPlay5                  -4.19     0.67  
## 
## Error terms:
##  Groups    Name             Std.Dev. Corr              
##  Subject   (Intercept)      0.00                       
##  Subject.1 Instrumentguitar 0.82                       
##            Instrumentpiano  1.33      0.21             
##            Instrumentstring 1.17     -0.89  0.25       
##  Subject.2 Voicecontrary    0.41                       
##            Voicepar3rd      0.05     -1.00             
##            Voicepar5th      0.09      1.00 -1.00       
##  Subject.3 HarmonyI-IV-V    0.37                       
##            HarmonyI-V-IV    0.47      0.88             
##            HarmonyI-V-VI    1.10     -0.53 -0.06       
##            HarmonyIV-I-V    0.15      0.36 -0.13 -0.98 
##  Residual                   1.57                       
## ---
## number of obs: 1541, groups: Subject, 43
## AIC = 6210.7, DIC = 5987.2
## deviance = 6041.0
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lmer.AIC.cat)}\SpecialCharTok{$}\NormalTok{coef}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                                  Estimate Std. Error     t value
## (Intercept)                   4.560668282 0.82224735  5.54658922
## HarmonyI-V-IV                -0.002601496 0.11837613 -0.02197652
## HarmonyI-V-VI                 0.853784731 0.23260416  3.67054803
## HarmonyIV-I-V                 0.058616838 0.12479073  0.46972111
## Instrumentpiano               1.649677029 0.23691795  6.96307313
## Instrumentstring              3.588086291 0.31225598 11.49084896
## Voicepar3rd                  -0.405828862 0.12092681 -3.35598748
## Voicepar5th                  -0.302210111 0.10977693 -2.75294746
## X16.minus.17                 -0.181975506 0.04099517 -4.43894954
## ConsNotes                    -0.299036371 0.06531837 -4.57813605
## PachListen3                  -1.297044535 0.58436752 -2.21956986
## PachListen4                   1.969786884 0.81809301  2.40777866
## PachListen5                  -0.508387402 0.45818847 -1.10955957
## ClsListen1                   -0.087688253 0.31317911 -0.27999394
## ClsListen3                    0.685627826 0.37296141  1.83833452
## ClsListen4                    6.429004044 0.97295021  6.60774211
## ClsListen5                   -0.054662554 0.48071453 -0.11371105
## KnowRob1                     -0.364503939 0.42843815 -0.85077377
## KnowRob5                      1.539721539 0.38408177  4.00883784
## KnowAxis1                    -6.303276982 1.20748995 -5.22014860
## KnowAxis5                    -0.485715461 0.29209998 -1.66283979
## X1990s2000s                  -0.233736836 0.10284320 -2.27274962
## X1990s2000s.minus.1960s1970s  0.231998764 0.08326177  2.78637792
## NoClass                       0.743205121 0.15872351  4.68238852
## APTheory1                     2.643103000 0.33425232  7.90750822
## Composing1                   -0.428123441 0.27363922 -1.56455439
## Composing2                    1.060007404 0.42495963  2.49437198
## Composing3                    0.042378529 0.38237540  0.11082964
## Composing4                    0.707093227 0.48486194  1.45833932
## Composing5                   -2.755735737 1.09542436 -2.51567871
## GuitarPlay1                   0.866779171 0.52774744  1.64241285
## GuitarPlay2                   3.361163778 0.49749842  6.75612951
## GuitarPlay4                   2.063650150 0.87931803  2.34687575
## GuitarPlay5                  -4.186721050 0.67478654 -6.20451178
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.BIC.cat }\OtherTok{\textless{}{-}} \FunctionTok{fitLMER.fnc}\NormalTok{(lmer.big.cat, }\AttributeTok{backfit.on=}\StringTok{"t"}\NormalTok{, }\AttributeTok{method=}\StringTok{"BIC"}\NormalTok{, }\AttributeTok{prune.ranefs=}\NormalTok{F,}
                        \AttributeTok{log.file.name=}\StringTok{"lmerBICcat.log"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Warning in fitLMER.fnc(lmer.big.cat, backfit.on = "t", method = "BIC", prune.ranefs = F, : Argument "ran.effects" is empty, which means you will not be forward-fitting the random effect structure of your model. You could just as well run function "bfFixefLMER_F.fnc" or "bfFixefLMER_t.fnc".
## TRUE
\end{verbatim}

\begin{verbatim}
## ======================================================
## ===              backfitting fixed effects         ===
## ======================================================
## setting REML to FALSE
## processing model terms of interaction level 1 
##   iteration 1 
##      t-value for term "Selfdeclare" = 0.1011264 < 2 
##      not part of higher-order interaction
##      BIC simple = 6504; BIC complex = 6511; decrease = -7 < 5
##      removing term
##   iteration 2 
##      t-value for term "OMSI" = 1.105238 < 2 
##      not part of higher-order interaction
##      BIC simple = 6498; BIC complex = 6504; decrease = -6 < 5
##      removing term
##   iteration 3 
##      t-value for term "ConsInstr" = 1.149557 < 2 
##      not part of higher-order interaction
##      BIC simple = 6498; BIC complex = 6498; decrease = 0 < 5
##      removing term
##   iteration 4 
##      t-value for term "Instr.minus.Notes" = 1.148654 < 2 
##      not part of higher-order interaction
##      BIC simple = 6491; BIC complex = 6498; decrease = -7 < 5
##      removing term
##   iteration 5 
##      t-value for term "CollegeMusic" = 1.55688 < 2 
##      not part of higher-order interaction
##      BIC simple = 6486; BIC complex = 6491; decrease = -5 < 5
##      removing term
##   iteration 6 
##      t-value for term "PianoPlay" = 1.565929 < 2 
##      not part of higher-order interaction
##      BIC simple = 6467; BIC complex = 6486; decrease = -19 < 5
##      removing term
## ======================================================
## ===            forwardfitting random effects       ===
## ======================================================
##  ===       random slopes       ===
## ======================================================
## ===            re-backfitting fixed effects        ===
## ======================================================
## setting REML to FALSE
## processing model terms of interaction level 1 
##   all terms of interaction level 1 significant
## resetting REML to TRUE
## log file is lmerBICcat.log
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{display}\NormalTok{(lmer.BIC.cat)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ Harmony + Instrument + Voice + X16.minus.17 + 
##     ConsNotes + PachListen + ClsListen + KnowRob + KnowAxis + 
##     X1990s2000s + X1990s2000s.minus.1960s1970s + NoClass + APTheory + 
##     Composing + GuitarPlay + (1 | Subject) + (0 + Instrument | 
##     Subject) + (0 + Voice | Subject) + (0 + Harmony | Subject), 
##     data = nomissdata.cat, REML = TRUE)
##                              coef.est coef.se
## (Intercept)                   4.56     0.82  
## HarmonyI-V-IV                 0.00     0.12  
## HarmonyI-V-VI                 0.85     0.23  
## HarmonyIV-I-V                 0.06     0.12  
## Instrumentpiano               1.65     0.24  
## Instrumentstring              3.59     0.31  
## Voicepar3rd                  -0.41     0.12  
## Voicepar5th                  -0.30     0.11  
## X16.minus.17                 -0.18     0.04  
## ConsNotes                    -0.30     0.07  
## PachListen3                  -1.30     0.58  
## PachListen4                   1.97     0.82  
## PachListen5                  -0.51     0.46  
## ClsListen1                   -0.09     0.31  
## ClsListen3                    0.69     0.37  
## ClsListen4                    6.43     0.97  
## ClsListen5                   -0.05     0.48  
## KnowRob1                     -0.36     0.43  
## KnowRob5                      1.54     0.38  
## KnowAxis1                    -6.30     1.21  
## KnowAxis5                    -0.49     0.29  
## X1990s2000s                  -0.23     0.10  
## X1990s2000s.minus.1960s1970s  0.23     0.08  
## NoClass                       0.74     0.16  
## APTheory1                     2.64     0.33  
## Composing1                   -0.43     0.27  
## Composing2                    1.06     0.42  
## Composing3                    0.04     0.38  
## Composing4                    0.71     0.48  
## Composing5                   -2.76     1.10  
## GuitarPlay1                   0.87     0.53  
## GuitarPlay2                   3.36     0.50  
## GuitarPlay4                   2.06     0.88  
## GuitarPlay5                  -4.19     0.67  
## 
## Error terms:
##  Groups    Name             Std.Dev. Corr              
##  Subject   (Intercept)      0.00                       
##  Subject.1 Instrumentguitar 0.82                       
##            Instrumentpiano  1.33      0.21             
##            Instrumentstring 1.17     -0.89  0.25       
##  Subject.2 Voicecontrary    0.41                       
##            Voicepar3rd      0.05     -1.00             
##            Voicepar5th      0.09      1.00 -1.00       
##  Subject.3 HarmonyI-IV-V    0.37                       
##            HarmonyI-V-IV    0.47      0.88             
##            HarmonyI-V-VI    1.10     -0.53 -0.06       
##            HarmonyIV-I-V    0.15      0.36 -0.13 -0.98 
##  Residual                   1.57                       
## ---
## number of obs: 1541, groups: Subject, 43
## AIC = 6210.7, DIC = 5987.2
## deviance = 6041.0
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{summary}\NormalTok{(lmer.BIC.cat)}\SpecialCharTok{$}\NormalTok{coef}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                                  Estimate Std. Error     t value
## (Intercept)                   4.560668282 0.82224735  5.54658922
## HarmonyI-V-IV                -0.002601496 0.11837613 -0.02197652
## HarmonyI-V-VI                 0.853784731 0.23260416  3.67054803
## HarmonyIV-I-V                 0.058616838 0.12479073  0.46972111
## Instrumentpiano               1.649677029 0.23691795  6.96307313
## Instrumentstring              3.588086291 0.31225598 11.49084896
## Voicepar3rd                  -0.405828862 0.12092681 -3.35598748
## Voicepar5th                  -0.302210111 0.10977693 -2.75294746
## X16.minus.17                 -0.181975506 0.04099517 -4.43894954
## ConsNotes                    -0.299036371 0.06531837 -4.57813605
## PachListen3                  -1.297044535 0.58436752 -2.21956986
## PachListen4                   1.969786884 0.81809301  2.40777866
## PachListen5                  -0.508387402 0.45818847 -1.10955957
## ClsListen1                   -0.087688253 0.31317911 -0.27999394
## ClsListen3                    0.685627826 0.37296141  1.83833452
## ClsListen4                    6.429004044 0.97295021  6.60774211
## ClsListen5                   -0.054662554 0.48071453 -0.11371105
## KnowRob1                     -0.364503939 0.42843815 -0.85077377
## KnowRob5                      1.539721539 0.38408177  4.00883784
## KnowAxis1                    -6.303276982 1.20748995 -5.22014860
## KnowAxis5                    -0.485715461 0.29209998 -1.66283979
## X1990s2000s                  -0.233736836 0.10284320 -2.27274962
## X1990s2000s.minus.1960s1970s  0.231998764 0.08326177  2.78637792
## NoClass                       0.743205121 0.15872351  4.68238852
## APTheory1                     2.643103000 0.33425232  7.90750822
## Composing1                   -0.428123441 0.27363922 -1.56455439
## Composing2                    1.060007404 0.42495963  2.49437198
## Composing3                    0.042378529 0.38237540  0.11082964
## Composing4                    0.707093227 0.48486194  1.45833932
## Composing5                   -2.755735737 1.09542436 -2.51567871
## GuitarPlay1                   0.866779171 0.52774744  1.64241285
## GuitarPlay2                   3.361163778 0.49749842  6.75612951
## GuitarPlay4                   2.063650150 0.87931803  2.34687575
## GuitarPlay5                  -4.186721050 0.67478654 -6.20451178
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{formula}\NormalTok{(lmer.AIC.cat)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Classical ~ Harmony + Instrument + Voice + X16.minus.17 + ConsNotes + 
##     PachListen + ClsListen + KnowRob + KnowAxis + X1990s2000s + 
##     X1990s2000s.minus.1960s1970s + NoClass + APTheory + Composing + 
##     GuitarPlay + (1 | Subject) + (0 + Instrument | Subject) + 
##     (0 + Voice | Subject) + (0 + Harmony | Subject)
## <environment: 0x0000000029301450>
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{formula}\NormalTok{(lmer.BIC.cat)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## Classical ~ Harmony + Instrument + Voice + X16.minus.17 + ConsNotes + 
##     PachListen + ClsListen + KnowRob + KnowAxis + X1990s2000s + 
##     X1990s2000s.minus.1960s1970s + NoClass + APTheory + Composing + 
##     GuitarPlay + (1 | Subject) + (0 + Instrument | Subject) + 
##     (0 + Voice | Subject) + (0 + Harmony | Subject)
## <environment: 0x0000000029301450>
\end{verbatim}

Once again, AIC and BIC picked the same models, but they are somewhat
different from the models chosen when we treated all the predictors as
continuous. The person-level covariates chosen were:

\begin{itemize}
\item X16.minus.17
\item ConsNotes
\item PachListen
\item ClsListen
\item KnowRob
\item KnowAxis
\item X1990s2000s
\item X1990s2000s.minus.1960s1970s
\item NoClass
\item APTheory
\item Composing
\item GuitarPlay 
\end{itemize}

Based on the \texttt{fitLMER.fnc} work, these are the person level
covariates that I would add to the model.

\hypertarget{aside-1}{%
\paragraph{Aside \#1:}\label{aside-1}}

When we look at display(lmer.AIC.cat) or display(lmer.BIC.cat), we that
there is really only one \(\eta\) shared among the levels of Voice
(because of the correlations of +1 and \(-1\)), and so it would be nice
to implement this restriction in the model. Unfortunately it is not
straightforward to use lmer() to restrict \(\eta\)'s in this way\ldots{}

\hypertarget{aside-2}{%
\paragraph{Aside \#2:}\label{aside-2}}

We haven't looked at any residual plots. Let's at least look at residual
plots for \texttt{lmer.AIC.cat}, that we are reasonably happy with

Note: For some reason, the hlm\_resid function in library(HLMdiag)
wasn't producing the 2nd level residuals (\(\eta\)'s), and so I just
extracted raw residuals directly:

\begin{itemize}
\item Conditional residuals are in `residuals(fitted.lmer.model)`
\item $\eta$'s are in `ranef(fitted.lmer.model)`
\end{itemize}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{res.cond }\OtherTok{\textless{}{-}} \FunctionTok{residuals}\NormalTok{(lmer.AIC.cat)}
\NormalTok{fit.cond }\OtherTok{\textless{}{-}} \FunctionTok{fitted}\NormalTok{(lmer.AIC.cat)}

\NormalTok{eta }\OtherTok{\textless{}{-}} \FunctionTok{ranef}\NormalTok{(lmer.AIC.cat)}\SpecialCharTok{$}\NormalTok{Subject}
\NormalTok{fit.grp }\OtherTok{\textless{}{-}} \FunctionTok{sapply}\NormalTok{(}\FunctionTok{split}\NormalTok{(fit.cond,nomissdata.cat}\SpecialCharTok{$}\NormalTok{Subject),mean)}

\FunctionTok{par}\NormalTok{(}\AttributeTok{mfcol=}\FunctionTok{c}\NormalTok{(}\DecValTok{4}\NormalTok{,}\DecValTok{3}\NormalTok{))}

\DocumentationTok{\#\# 1}

\FunctionTok{plot}\NormalTok{(res.cond }\SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.cond,}\AttributeTok{xlab=}\StringTok{"Conditional Fitted Values Per Listener"}\NormalTok{, }\AttributeTok{ylab=}\StringTok{"Conditional Residuals"}\NormalTok{,}
     \AttributeTok{main=}\StringTok{"Level 1 Residuals"}\NormalTok{)}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(res.cond,}\AttributeTok{ylab=}\StringTok{"Conditional Residual QUantiles"}\NormalTok{, }\AttributeTok{main=}\StringTok{"Level 1 Residuals"}\NormalTok{)}
\FunctionTok{qqline}\NormalTok{(res.cond)}

\DocumentationTok{\#\# 2}

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"(Intercept)"} \SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Intercept "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"(Intercept)"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Intercept "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"(Intercept)"}\NormalTok{)}

\DocumentationTok{\#\# 3}

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Instrumentguitar }\SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Instrument=guitar "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Instrumentguitar,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Instrument=guitar "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Instrumentguitar)}

\DocumentationTok{\#\# 4 }

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Instrumentpiano }\SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Instrument=piano "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Instrumentpiano,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Instrument=piano "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Instrumentpiano)}

\DocumentationTok{\#\# 5 }

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Instrumentstring }\SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Instrument=string "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Instrumentstring,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Instrument=string "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Instrumentstring)}

\DocumentationTok{\#\# 6}

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Voicecontrary }\SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Voice=contrary "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Voicecontrary,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Voice=contrary "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Voicecontrary)}
\end{Highlighting}
\end{Shaded}

\includegraphics{hw09-solutions_files/figure-latex/unnamed-chunk-32-1.pdf}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{par}\NormalTok{(}\AttributeTok{mfcol=}\FunctionTok{c}\NormalTok{(}\DecValTok{4}\NormalTok{,}\DecValTok{3}\NormalTok{))}


\DocumentationTok{\#\# 7}

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Voicepar3rd }\SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Voice=par3rd "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Voicepar3rd,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Voice=par3rd "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Voicepar3rd)}

\DocumentationTok{\#\# 8}

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Voicepar5th}\SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Voice=par5th "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Voicepar5th,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Voice=par5th "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\NormalTok{Voicepar5th)}

\DocumentationTok{\#\# 9}

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyI{-}IV{-}V"}  \SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Harmony=I{-}IV{-}V "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyI{-}IV{-}V"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Harmony=I{-}IV{-}V "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyI{-}IV{-}V"}\NormalTok{)}

\DocumentationTok{\#\# 10}

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyI{-}V{-}IV"} \SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Harmony=I{-}V{-}IV "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyI{-}V{-}IV"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Harmony=I{-}V{-}IV "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyI{-}V{-}IV"}\NormalTok{)}

\DocumentationTok{\#\# 11}

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyI{-}V{-}VI"} \SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Harmony=I{-}V{-}VI "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyI{-}V{-}VI"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Harmony=I{-}V{-}VI "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyI{-}V{-}VI"}\NormalTok{)}

\DocumentationTok{\#\# 12}

\FunctionTok{plot}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyIV{-}I{-}V"} \SpecialCharTok{\textasciitilde{}}\NormalTok{ fit.grp,}\AttributeTok{xlab=}\StringTok{"Mean Fitted Values Per Listener"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(eta),}
     \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Harmony=IV{-}I{-}V "}\NormalTok{,eta)))}
\FunctionTok{abline}\NormalTok{(}\AttributeTok{h=}\DecValTok{0}\NormalTok{)}

\FunctionTok{qqnorm}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyIV{-}I{-}V"}\NormalTok{,}\AttributeTok{ylab=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(eta,}\StringTok{" QUantiles"}\NormalTok{)), }
       \AttributeTok{main=}\FunctionTok{expression}\NormalTok{(}\FunctionTok{paste}\NormalTok{(}\StringTok{"Harmony=IV{-}I{-}V "}\NormalTok{,eta)))}
\FunctionTok{qqline}\NormalTok{(eta}\SpecialCharTok{$}\StringTok{"HarmonyIV{-}I{-}V"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\includegraphics{hw09-solutions_files/figure-latex/unnamed-chunk-33-1.pdf}
All these residual plots adn qq plots look pretty good. The only one
that looks a bit strange is the first plot, i.e., the plot of the
level-1 conditional residuals. It is seriated (points fall along lines
within the plot), but that is essentially because all of the fixed
effect predictors are discrete.

\rm

\hypertarget{problem-3b.}{%
\subsection{Problem 3(b).}\label{problem-3b.}}

Once the fixed effects are settled, go back and check to see whether
there should be any change in the random effects; focus on random slopes
for design variables and their interactions. Provide suitable evidence
to justify your results. \label{p:3b}

\it

Let's go ahead and just do the same model selection exercise we did
before, for the random effects\ldots{}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.AIC.i }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.AIC.cat, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Voice}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Harmony}\SpecialCharTok{|}\NormalTok{Subject)) }
\NormalTok{lmer.AIC.v }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.AIC.cat, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Instrument}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Harmony}\SpecialCharTok{|}\NormalTok{Subject))}
\NormalTok{lmer.AIC.h }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.AIC.cat, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Instrument}\SpecialCharTok{|}\NormalTok{Subject) }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Voice}\SpecialCharTok{|}\NormalTok{Subject))}
\NormalTok{lmer.AIC.iv }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.AIC.cat, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Harmony}\SpecialCharTok{|}\NormalTok{Subject))}
\NormalTok{lmer.AIC.ih }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.AIC.cat, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Voice}\SpecialCharTok{|}\NormalTok{Subject))}
\NormalTok{lmer.AIC.vh }\OtherTok{\textless{}{-}} \FunctionTok{update}\NormalTok{(lmer.AIC.cat, . }\SpecialCharTok{\textasciitilde{}}\NormalTok{ . }\SpecialCharTok{{-}}\NormalTok{ (}\DecValTok{0}\SpecialCharTok{+}\NormalTok{Instrument}\SpecialCharTok{|}\NormalTok{Subject))}
\NormalTok{lmer.AIC.ivh }\OtherTok{\textless{}{-}}\NormalTok{ lmer.AIC.cat}

\FunctionTok{print}\NormalTok{(x }\OtherTok{\textless{}{-}} \FunctionTok{rbind}\NormalTok{(}
  \AttributeTok{lmer.AIC.i=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.AIC.i)),}
  \AttributeTok{lmer.AIC.v=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.AIC.v)),}
  \AttributeTok{lmer.AIC.h=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.AIC.h)),}
  \AttributeTok{lmer.AIC.iv=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.AIC.iv)),}
  \AttributeTok{lmer.AIC.ih=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.AIC.ih)),}
  \AttributeTok{lmer.AIC.vh=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.AIC.vh)),}
  \AttributeTok{lmer.AIC.ivh=}\FunctionTok{c}\NormalTok{(}\AttributeTok{DIC=}\FunctionTok{DIC.ml}\NormalTok{(lmer.AIC.ivh)))}
\NormalTok{  )}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                   DIC
## lmer.AIC.i   6143.984
## lmer.AIC.v   6417.701
## lmer.AIC.h   6364.563
## lmer.AIC.iv  6140.944
## lmer.AIC.ih  6046.190
## lmer.AIC.vh  6363.701
## lmer.AIC.ivh 6040.996
\end{verbatim}

We just end up with \texttt{lmer.AIC.ivh}, which is the same model that
we got from backwards elimination in part (a).

\rm

\hypertarget{problem-3c.}{%
\subsection{Problem 3(c).}\label{problem-3c.}}

Briefly interpret the effect of each variable kept in the final model,
on Classical ratings. \label{p:3c}

\it

From the summary of the fixed effect estimates (the \(\hat\beta\)'s),

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{round}\NormalTok{(}\FunctionTok{summary}\NormalTok{(lmer.AIC.ivh)}\SpecialCharTok{$}\NormalTok{coef,}\DecValTok{2}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                              Estimate Std. Error t value
## (Intercept)                      4.56       0.82    5.55
## HarmonyI-V-IV                    0.00       0.12   -0.02
## HarmonyI-V-VI                    0.85       0.23    3.67
## HarmonyIV-I-V                    0.06       0.12    0.47
## Instrumentpiano                  1.65       0.24    6.96
## Instrumentstring                 3.59       0.31   11.49
## Voicepar3rd                     -0.41       0.12   -3.36
## Voicepar5th                     -0.30       0.11   -2.75
## X16.minus.17                    -0.18       0.04   -4.44
## ConsNotes                       -0.30       0.07   -4.58
## PachListen3                     -1.30       0.58   -2.22
## PachListen4                      1.97       0.82    2.41
## PachListen5                     -0.51       0.46   -1.11
## ClsListen1                      -0.09       0.31   -0.28
## ClsListen3                       0.69       0.37    1.84
## ClsListen4                       6.43       0.97    6.61
## ClsListen5                      -0.05       0.48   -0.11
## KnowRob1                        -0.36       0.43   -0.85
## KnowRob5                         1.54       0.38    4.01
## KnowAxis1                       -6.30       1.21   -5.22
## KnowAxis5                       -0.49       0.29   -1.66
## X1990s2000s                     -0.23       0.10   -2.27
## X1990s2000s.minus.1960s1970s     0.23       0.08    2.79
## NoClass                          0.74       0.16    4.68
## APTheory1                        2.64       0.33    7.91
## Composing1                      -0.43       0.27   -1.56
## Composing2                       1.06       0.42    2.49
## Composing3                       0.04       0.38    0.11
## Composing4                       0.71       0.48    1.46
## Composing5                      -2.76       1.10   -2.52
## GuitarPlay1                      0.87       0.53    1.64
## GuitarPlay2                      3.36       0.50    6.76
## GuitarPlay4                      2.06       0.88    2.35
## GuitarPlay5                     -4.19       0.67   -6.20
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{library}\NormalTok{(car)}

\FunctionTok{vif}\NormalTok{(lmer.AIC.ivh)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                                    GVIF Df GVIF^(1/(2*Df))
## Harmony                        1.000076  3        1.000013
## Instrument                     1.000044  2        1.000011
## Voice                          1.000051  2        1.000013
## X16.minus.17                   3.530400  1        1.878936
## ConsNotes                      3.575293  1        1.890845
## PachListen                     5.851646  3        1.342393
## ClsListen                     49.197065  4        1.627393
## KnowRob                       14.217264  2        1.941798
## KnowAxis                      19.040883  2        2.088920
## X1990s2000s                    5.192462  1        2.278697
## X1990s2000s.minus.1960s1970s   5.063978  1        2.250328
## NoClass                       10.545040  1        3.247313
## APTheory                       4.428740  1        2.104457
## Composing                    241.595012  5        1.731047
## GuitarPlay                   218.942296  4        1.961287
\end{verbatim}

There don't appear to be serious collinearities, so we can infer from
the fixed effects coeffient table above that

\begin{itemize}
\item The harmony I-V-IV adds nearly a point on average to the Classical rating, vs. the "baseline" harmony I-IV-V.  The other harmonies do not really change the Classical rating much. \bf Listeners generally thing I-V-IV is a more classical sounding harmony structure.\it
\item The instrument "piano" adds over 1.5 points to the Classical rating, vs. the baseline instrument "guitar".  The instrument "string" adds over 3.5 points to the Classical rating, vs. "guitar".  \bf Listeners think piano, and especially strings, sound more classical than guitar.\it
\item The voice-leading patterns "parallel 3rds" and "parallel 5ths" reduce the Classical rating by roughly 0.4 and 0.3 points, respectively, relative to the baseline "contrary motion". \bf Listeners think contrary motion sounds mildly more classical than the other two voice-leading patterns.\it
\item We don't know what `X16.minus17` is, but it has a small negative effect on Classical ratings.
\item People who concentrate on the notes when listening have classical ratings which are on average about 0.3 lower ($\hat\beta_{ConsNotes}=-0.3$), than people who do not concentrate on the notes.
\item The pattern with PachListen (familiarity with Pachelbel's Canon) and ClsListen (how much do you listen to classical music) is not straightforward.  For example, listeners who scored 4 out of 5 on these two measures added nearly 2, and nearly 7 points, respectively, to their Classical ratings, on average, relative to the baseline category of 0 ( = `` not at all'') on each variable.  But listeners who scored 5 out of 5 on these measures actually reduced their Classical ratings somewhat, on average, relative to category 0.  However, these may be due in part to a small-sample bias issue, since only one listener scored 4 on either PachListen or ClsListen. \bf(See calculations below.)\it
\item The patterns with KnowRob (have you heard Rob Paravonian’s Pachelbel Rant) and KnowAxis (have you heard Axis of Evil’s Comedy bit on the 4 Pachelbel chords in popular music?) is more straightforward.  Relative to baseline category 0 (= ``not at all''),  listeners who express modest familiarity with the Rob Paravonian bit have lower Classical ratings by about 0.36 points, whereas those with modest familiarity with the Axis of Awesome bit, reduce their classical ratings by more than six points. The giant jump ($\hat\beta_{KNowAxis=1}-6.30$) may be mainly a small-sample bias issue, since only one listener scored in category 1 on this variable.   \bf(See calculations below.)\it
\item Each additional point of familiarity with music from the 1990's conveys a small reduction in Classical ratings on average, but each additional point in the contrast between familiarity with music from the 1990's vs familiarity withmusic from the 1960's conveys a similar-sized increase in Classical ratings; it seems likely that these two effects basically cancel each other out.
\item Each additional Music class that a listener has taken increases their Classical scores by about 0.74; on the other hand, having taken AP Theory conveys a 2.64 increase in classical ratings. 
\item The pattern of estimated coefficients for Composing is again strange, with a score of 5 on the Composing variable associated with a 2.76 point drop in Classical ratings, breaking the pattern established by the other levels of Composing.  Once again, this may be mostly a small-sample bias issue since only one listener is in category 5 on the Composing variable.  \bf(See calculations below.)\it
\item The pattern of coefficients for GuitarPlay is also hard to understand, but the coefficients for categories 2 and 4 may be in part a consequence of small sample bias.  \bf(See calculations below.)\it
\end{itemize}

Here are the calculations that show that categories of some of the
predictors with anomalous coefficients also involve only one or two
examinees, and so the anomalies may be simply small-sample bias issues:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata.cat}\SpecialCharTok{$}\NormalTok{PachListen)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##    2    3    4    5 
##   72  108   36 1325
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{table}\NormalTok{(Subject[PachListen}\SpecialCharTok{==}\StringTok{"4"}\NormalTok{]))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## 53 
## 36
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata.cat}\SpecialCharTok{$}\NormalTok{ClsListen)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##   0   1   3   4   5 
## 216 576 498  36 215
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{table}\NormalTok{(Subject[ClsListen}\SpecialCharTok{==}\StringTok{"4"}\NormalTok{]))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## 15 
## 36
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata.cat}\SpecialCharTok{$}\NormalTok{KnowAxis)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##    0    1    5 
## 1109   36  396
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{table}\NormalTok{(Subject[KnowAxis}\SpecialCharTok{==}\StringTok{"1"}\NormalTok{]))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## 47 
## 36
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata.cat}\SpecialCharTok{$}\NormalTok{Composing)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##   0   1   2   3   4   5 
## 828 216 173 144 144  36
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{table}\NormalTok{(Subject[Composing}\SpecialCharTok{==}\StringTok{"5"}\NormalTok{]))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## 52 
## 36
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata.cat}\SpecialCharTok{$}\NormalTok{GuitarPlay)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##    0    1    2    4    5 
## 1152  107   36   72  174
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{table}\NormalTok{(Subject[GuitarPlay}\SpecialCharTok{==}\StringTok{"2"}\NormalTok{]))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## 59 
## 36
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{table}\NormalTok{(Subject[GuitarPlay}\SpecialCharTok{==}\StringTok{"4"}\NormalTok{]))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
## 47 56 
## 36 36
\end{verbatim}

\rm

\hypertarget{problem-4.}{%
\section{Problem 4.}\label{problem-4.}}

\underline{\textit{Musicians vs. Non-musicians.}} One of the secondary
hypotheses of the researchers is that people who self-identify as
musicians may be influenced by things that do not influence
non-musicians. Dichotomize ``Selfdeclare'' (``are you a musician?'') so
that about half the participants are categorized as self-declared
musicians, and half not. Examine and report on any interactions between
the dichotomized musician variable and other predictors in the model.
Check to see if the results are sensitive to where you dichotomize.
Provide suitable evidence for, and comment on, your results. \label{p:4}

\it

Interstingly, \texttt{Selfdeclare} was not one of the variables in the
final model selected by \texttt{fitLMER.fnc} above. Here is the result
of dichotomizing is and trying to include it in the model.

Since there are 1541 observations in the \texttt{nomissdata.cat} data
frame and, as shown in the table below, about half are accounted for in
the first two categories of \texttt{Selfdeclare}, we'll start by
dichotomizing between 2 and 3.

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{cumsum}\NormalTok{(}\FunctionTok{table}\NormalTok{(nomissdata.cat}\SpecialCharTok{$}\NormalTok{Selfdeclare))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##    1    2    3    4    5    6 
##  144  827 1115 1433 1505 1541
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(Musician }\OtherTok{\textless{}{-}} \FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{ifelse}\NormalTok{(Selfdeclare}\SpecialCharTok{\textless{}=}\DecValTok{2}\NormalTok{,}\DecValTok{0}\NormalTok{,}\DecValTok{1}\NormalTok{)))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##   0   1 
## 827 714
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{big.fla.with.Musician }\OtherTok{\textless{}{-}} \FunctionTok{paste}\NormalTok{(}\StringTok{"Classical \textasciitilde{} ("}\NormalTok{, }\FunctionTok{paste}\NormalTok{(big.predictors,}\AttributeTok{collapse=}\StringTok{" + "}\NormalTok{),}
                 \StringTok{"{-} Selfdeclare)*Musician + (Instrument + Voice + Harmony || Subject)"}\NormalTok{)}

\NormalTok{lmer.big.with.Musician }\OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(big.fla.with.Musician,}\AttributeTok{data=}\NormalTok{nomissdata.cat)}

\FunctionTok{display}\NormalTok{(lmer.big.with.Musician)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## lmer(formula = Classical ~ (Harmony + Instrument + Voice + Selfdeclare + 
##     OMSI + X16.minus.17 + ConsInstr + ConsNotes + Instr.minus.Notes + 
##     PachListen + ClsListen + KnowRob + KnowAxis + X1990s2000s + 
##     X1990s2000s.minus.1960s1970s + CollegeMusic + NoClass + APTheory + 
##     Composing + PianoPlay + GuitarPlay - Selfdeclare) * Musician + 
##     ((1 | Subject) + (0 + Instrument | Subject) + (0 + Voice | 
##         Subject) + (0 + Harmony | Subject)), data = nomissdata.cat)
##                              coef.est coef.se 
## (Intercept)                    176.12   331.81
## HarmonyI-V-IV                    0.01     0.16
## HarmonyI-V-VI                    0.27     0.29
## HarmonyIV-I-V                    0.00     0.17
## Instrumentpiano                  1.96     0.32
## Instrumentstring                 4.14     0.41
## Voicepar3rd                     -0.45     0.17
## Voicepar5th                     -0.25     0.15
## OMSI                            -0.11     0.21
## X16.minus.17                    -8.32    16.65
## ConsInstr                       -6.55    13.10
## ConsNotes                      -31.55    62.81
## PachListen3                     18.35    62.52
## PachListen4                   -224.10   453.96
## PachListen5                    -83.68   155.46
## ClsListen1                    -121.25   242.91
## ClsListen3                    -140.28   279.90
## ClsListen4                    -872.23  1641.54
## ClsListen5                      53.72   117.75
## KnowRob1                      -168.48   306.69
## KnowRob5                        58.05    99.04
## KnowAxis1                    -1375.90  2715.17
## KnowAxis5                        1.85     3.36
## X1990s2000s                     28.30    55.81
## X1990s2000s.minus.1960s1970s    -8.21    14.70
## CollegeMusic1                  -21.48    46.49
## NoClass                        -23.22    45.40
## APTheory1                      174.41   337.85
## Composing1                     -59.98   112.40
## Composing2                    -154.27   356.80
## Composing3                     163.34   269.49
## Composing4                     828.93  1537.85
## Composing5                     197.64   365.36
## PianoPlay1                      26.17    72.92
## PianoPlay4                    -123.26   258.84
## PianoPlay5                    -369.32   748.23
## GuitarPlay1                    -88.04   158.25
## GuitarPlay2                    218.90   446.78
## GuitarPlay4                   -111.64   152.27
## GuitarPlay5                   -699.92  1347.77
## Musician                      -681.23  1339.61
## HarmonyI-V-IV:Musician          -0.03     0.24
## HarmonyI-V-VI:Musician           1.26     0.43
## HarmonyIV-I-V:Musician           0.12     0.25
## Instrumentpiano:Musician        -0.67     0.47
## Instrumentstring:Musician       -1.19     0.61
## Voicepar3rd:Musician             0.09     0.24
## Voicepar5th:Musician            -0.10     0.22
## OMSI:Musician                    1.52     2.92
## X16.minus.17:Musician           68.30   132.21
## ConsInstr:Musician             -24.45    42.54
## ConsNotes:Musician             160.16   295.49
## PachListen3:Musician          -759.95  1478.20
## PachListen5:Musician          -652.51  1199.08
## ClsListen1:Musician            359.10   708.34
## ClsListen3:Musician            239.54   512.92
## KnowRob1:Musician              607.34  1152.88
## 
## Error terms:
##  Groups    Name             Std.Dev. Corr           
##  Subject   (Intercept)      1.61                    
##  Subject.1 Instrumentguitar 1.31                    
##            Instrumentpiano  1.01     0.30           
##            Instrumentstring 1.48     0.10 0.29      
##  Subject.2 Voicecontrary    0.19                    
##            Voicepar3rd      0.64     0.91           
##            Voicepar5th      0.51     0.93 1.00      
##  Subject.3 HarmonyI-IV-V    0.47                    
##            HarmonyI-V-IV    0.60     0.91           
##            HarmonyI-V-VI    1.11     0.04 0.37      
##            HarmonyIV-I-V    0.32     0.70 0.49 0.14 
##  Residual                   1.57                    
## ---
## number of obs: 1541, groups: Subject, 43
## AIC = 6227.1, DIC = 5930.6
## deviance = 5997.9
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{round}\NormalTok{(}\FunctionTok{summary}\NormalTok{(lmer.big.with.Musician)}\SpecialCharTok{$}\NormalTok{coef,}\DecValTok{2}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                              Estimate Std. Error t value
## (Intercept)                    176.12     331.81    0.53
## HarmonyI-V-IV                    0.01       0.16    0.06
## HarmonyI-V-VI                    0.27       0.29    0.91
## HarmonyIV-I-V                    0.00       0.17    0.03
## Instrumentpiano                  1.96       0.32    6.13
## Instrumentstring                 4.14       0.41   10.03
## Voicepar3rd                     -0.45       0.17   -2.68
## Voicepar5th                     -0.25       0.15   -1.68
## OMSI                            -0.11       0.21   -0.56
## X16.minus.17                    -8.32      16.65   -0.50
## ConsInstr                       -6.55      13.10   -0.50
## ConsNotes                      -31.55      62.81   -0.50
## PachListen3                     18.35      62.52    0.29
## PachListen4                   -224.10     453.96   -0.49
## PachListen5                    -83.68     155.46   -0.54
## ClsListen1                    -121.25     242.91   -0.50
## ClsListen3                    -140.28     279.90   -0.50
## ClsListen4                    -872.23    1641.54   -0.53
## ClsListen5                      53.72     117.75    0.46
## KnowRob1                      -168.48     306.69   -0.55
## KnowRob5                        58.05      99.04    0.59
## KnowAxis1                    -1375.90    2715.17   -0.51
## KnowAxis5                        1.85       3.36    0.55
## X1990s2000s                     28.30      55.81    0.51
## X1990s2000s.minus.1960s1970s    -8.21      14.70   -0.56
## CollegeMusic1                  -21.48      46.49   -0.46
## NoClass                        -23.22      45.40   -0.51
## APTheory1                      174.41     337.85    0.52
## Composing1                     -59.98     112.40   -0.53
## Composing2                    -154.27     356.80   -0.43
## Composing3                     163.34     269.49    0.61
## Composing4                     828.93    1537.85    0.54
## Composing5                     197.64     365.36    0.54
## PianoPlay1                      26.17      72.92    0.36
## PianoPlay4                    -123.26     258.84   -0.48
## PianoPlay5                    -369.32     748.23   -0.49
## GuitarPlay1                    -88.04     158.25   -0.56
## GuitarPlay2                    218.90     446.78    0.49
## GuitarPlay4                   -111.64     152.27   -0.73
## GuitarPlay5                   -699.92    1347.77   -0.52
## Musician                      -681.23    1339.61   -0.51
## HarmonyI-V-IV:Musician          -0.03       0.24   -0.11
## HarmonyI-V-VI:Musician           1.26       0.43    2.94
## HarmonyIV-I-V:Musician           0.12       0.25    0.47
## Instrumentpiano:Musician        -0.67       0.47   -1.43
## Instrumentstring:Musician       -1.19       0.61   -1.97
## Voicepar3rd:Musician             0.09       0.24    0.35
## Voicepar5th:Musician            -0.10       0.22   -0.45
## OMSI:Musician                    1.52       2.92    0.52
## X16.minus.17:Musician           68.30     132.21    0.52
## ConsInstr:Musician             -24.45      42.54   -0.57
## ConsNotes:Musician             160.16     295.49    0.54
## PachListen3:Musician          -759.95    1478.20   -0.51
## PachListen5:Musician          -652.51    1199.08   -0.54
## ClsListen1:Musician            359.10     708.34    0.51
## ClsListen3:Musician            239.54     512.92    0.47
## KnowRob1:Musician              607.34    1152.88    0.53
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{large.fixed.effects }\OtherTok{\textless{}{-}} \ControlFlowTok{function}\NormalTok{(M) \{}
\NormalTok{  x }\OtherTok{\textless{}{-}} \FunctionTok{as.data.frame}\NormalTok{(}\FunctionTok{summary}\NormalTok{(M)}\SpecialCharTok{$}\NormalTok{coef)}
  \FunctionTok{return}\NormalTok{(}\FunctionTok{round}\NormalTok{(x[}\FunctionTok{abs}\NormalTok{(x}\SpecialCharTok{$}\StringTok{"t value"}\NormalTok{)}\SpecialCharTok{\textgreater{}=}\FloatTok{2.00}\NormalTok{,],}\DecValTok{2}\NormalTok{))}
\NormalTok{\}}

\FunctionTok{large.fixed.effects}\NormalTok{(lmer.big.with.Musician)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                        Estimate Std. Error t value
## Instrumentpiano            1.96       0.32    6.13
## Instrumentstring           4.14       0.41   10.03
## Voicepar3rd               -0.45       0.17   -2.68
## HarmonyI-V-VI:Musician     1.26       0.43    2.94
\end{verbatim}

We see that

\begin{itemize}
\item Some coefficients are very poorly estimated (e.g. estimated coefficients in the 100's or 1000's even though the response only ranges from 0 to 10).  This is most often caused by severe collinearity and/or small sample size for estimating those particular coefficients.
\item Many possible interactions with Musician are dropped because of collinearities in the model; and 
\item Musician does seem to have a significant interaction the design variables Instrument, Voice and Harmony.  The interactions really only seem to involve one or two levels of each design variable.  
\end{itemize}

Next, we see what happens if we try to dichotomize Musician at different
places:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(nomissdata.cat}\SpecialCharTok{$}\NormalTok{Selfdeclare)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##   1   2   3   4   5   6 
## 144 683 288 318  72  36
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(Musician }\OtherTok{\textless{}{-}} \FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{ifelse}\NormalTok{(Selfdeclare}\SpecialCharTok{\textless{}=}\DecValTok{1}\NormalTok{,}\DecValTok{0}\NormalTok{,}\DecValTok{1}\NormalTok{)))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##    0    1 
##  144 1397
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.Musician}\FloatTok{.1} \OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(big.fla.with.Musician,}\AttributeTok{data=}\NormalTok{nomissdata.cat)}
\FunctionTok{large.fixed.effects}\NormalTok{(lmer.Musician}\FloatTok{.1}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                  Estimate Std. Error t value
## (Intercept)          8.64       2.25    3.84
## Instrumentstring     2.75       1.03    2.68
## Voicepar3rd         -0.88       0.38   -2.28
## X16.minus.17        -0.27       0.09   -2.95
## ConsInstr            0.36       0.14    2.63
## ConsNotes           -0.55       0.10   -5.46
## PachListen3         -2.41       0.82   -2.94
## ClsListen1          -0.95       0.47   -2.04
## ClsListen4           5.60       2.25    2.49
## KnowAxis1           -8.21       2.17   -3.79
## KnowAxis5           -1.22       0.47   -2.61
## APTheory1            3.28       0.63    5.17
## Composing1          -0.87       0.43   -2.04
## GuitarPlay1          2.01       0.89    2.27
## GuitarPlay4          5.44       1.79    3.04
## GuitarPlay5         -3.61       1.04   -3.48
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\DocumentationTok{\#\# This is our original dichotomization:}
\FunctionTok{table}\NormalTok{(Musician }\OtherTok{\textless{}{-}} \FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{ifelse}\NormalTok{(Selfdeclare}\SpecialCharTok{\textless{}=}\DecValTok{2}\NormalTok{,}\DecValTok{0}\NormalTok{,}\DecValTok{1}\NormalTok{)))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##   0   1 
## 827 714
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.Musician}\FloatTok{.2} \OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(big.fla.with.Musician,}\AttributeTok{data=}\NormalTok{nomissdata.cat)}
\FunctionTok{large.fixed.effects}\NormalTok{(lmer.Musician}\FloatTok{.2}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                        Estimate Std. Error t value
## Instrumentpiano            1.96       0.32    6.13
## Instrumentstring           4.14       0.41   10.03
## Voicepar3rd               -0.45       0.17   -2.68
## HarmonyI-V-VI:Musician     1.26       0.43    2.94
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(Musician }\OtherTok{\textless{}{-}} \FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{ifelse}\NormalTok{(Selfdeclare}\SpecialCharTok{\textless{}=}\DecValTok{3}\NormalTok{,}\DecValTok{0}\NormalTok{,}\DecValTok{1}\NormalTok{)))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##    0    1 
## 1115  426
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.Musician}\FloatTok{.3} \OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(big.fla.with.Musician,}\AttributeTok{data=}\NormalTok{nomissdata.cat)}
\FunctionTok{large.fixed.effects}\NormalTok{(lmer.Musician}\FloatTok{.3}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                        Estimate Std. Error t value
## (Intercept)                7.89       2.48    3.19
## Instrumentpiano            1.83       0.28    6.58
## Instrumentstring           3.95       0.36   11.05
## Voicepar3rd               -0.38       0.14   -2.68
## Voicepar5th               -0.29       0.13   -2.28
## PachListen3               -4.76       1.54   -3.09
## PachListen5               -4.32       1.62   -2.66
## ClsListen4                18.73       8.15    2.30
## KnowRob5                   3.74       1.65    2.27
## APTheory1                  6.99       2.05    3.41
## Composing1                -4.48       1.84   -2.44
## GuitarPlay5              -11.70       5.19   -2.26
## HarmonyI-V-VI:Musician     1.41       0.48    2.95
## OMSI:Musician             -0.03       0.01   -2.05
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(Musician }\OtherTok{\textless{}{-}} \FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{ifelse}\NormalTok{(Selfdeclare}\SpecialCharTok{\textless{}=}\DecValTok{4}\NormalTok{,}\DecValTok{0}\NormalTok{,}\DecValTok{1}\NormalTok{)))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##    0    1 
## 1433  108
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.Musician}\FloatTok{.4} \OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(big.fla.with.Musician,}\AttributeTok{data=}\NormalTok{nomissdata.cat)}
\FunctionTok{large.fixed.effects}\NormalTok{(lmer.Musician}\FloatTok{.4}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                        Estimate Std. Error t value
## (Intercept)                6.26       1.31    4.78
## HarmonyI-V-VI              0.72       0.23    3.14
## Instrumentpiano            1.68       0.25    6.79
## Instrumentstring           3.65       0.33   11.20
## Voicepar3rd               -0.37       0.12   -2.98
## Voicepar5th               -0.29       0.11   -2.57
## X16.minus.17              -0.17       0.05   -3.24
## ConsNotes                 -0.51       0.10   -5.25
## PachListen3               -2.30       0.80   -2.86
## KnowAxis1                 -8.67       2.75   -3.16
## APTheory1                  3.59       1.23    2.93
## GuitarPlay4                4.60       1.79    2.56
## GuitarPlay5               -4.92       2.45   -2.01
## HarmonyI-V-VI:Musician     1.95       0.86    2.27
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{table}\NormalTok{(Musician }\OtherTok{\textless{}{-}} \FunctionTok{with}\NormalTok{(nomissdata.cat,}\FunctionTok{ifelse}\NormalTok{(Selfdeclare}\SpecialCharTok{\textless{}=}\DecValTok{5}\NormalTok{,}\DecValTok{0}\NormalTok{,}\DecValTok{1}\NormalTok{)))}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
## 
##    0    1 
## 1505   36
\end{verbatim}

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{lmer.Musician}\FloatTok{.5} \OtherTok{\textless{}{-}} \FunctionTok{lmer}\NormalTok{(big.fla.with.Musician,}\AttributeTok{data=}\NormalTok{nomissdata.cat)}
\FunctionTok{large.fixed.effects}\NormalTok{(lmer.Musician}\FloatTok{.5}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\begin{verbatim}
##                  Estimate Std. Error t value
## (Intercept)          5.53       1.10    5.01
## HarmonyI-V-VI        0.85       0.23    3.65
## Instrumentpiano      1.68       0.24    7.02
## Instrumentstring     3.61       0.32   11.32
## Voicepar3rd         -0.42       0.12   -3.45
## Voicepar5th         -0.31       0.11   -2.82
## X16.minus.17        -0.16       0.05   -3.31
## ConsInstr            0.28       0.12    2.22
## ConsNotes           -0.50       0.09   -5.48
## PachListen3         -2.04       0.78   -2.63
## ClsListen4           4.34       2.06    2.11
## KnowAxis1           -7.20       2.08   -3.47
## KnowAxis5           -0.89       0.42   -2.15
## APTheory1            2.82       0.52    5.46
## Composing1          -1.05       0.40   -2.59
## GuitarPlay4          4.38       1.67    2.62
## GuitarPlay5         -3.59       1.02   -3.53
\end{verbatim}

It's clear that the interactions with Musician vary greatly, depending
on where we dichotomize Selfdeclare to get the Musician variable. Our
original dichotomization produces the most parsimonious result (only
interactions with the three design variables), but exactly where to
dichotomize would depend on a conversation with Dr.~Jimenez.

\rm

\hypertarget{problem-5.}{%
\section{Problem 5.}\label{problem-5.}}

\underline{\textit{Classical vs. Popular.}} It would be totally
appropriate to repeat problems 2, 3 and 4 using ``Popular'' ratings as
the response, rather than classical, and then compare the results for
``Classical'' ratings, vs.~``Popular'' ratings.
\textit{In the interest of time, \textbf{you do not have to do this} for this hw/project.  It would make good "future work" for the paper; and/or it is something you could pursue on your own, if you are curious about it.}
\label{p:5}

\it

This is already a very long set of solutions, so I won't repeat any of
the work using \texttt{Popular} as a response variable! This can just be
``future work''!

\rm

\hypertarget{problem-6.}{%
\section{Problem 6.}\label{problem-6.}}

\underline{\textit{Brief writeup.}} Write an approximately one or two
page professional-quality
summary\footnote{This should form raw materials for the \textbf{Discussion} section of your final IDMRAD paper, so please review what should be in the \textbf{Discussion} section of an IMDRAD or IMRAD paper.  Also, try to keep this close to one or two pages.}
of your findings, suitable for Dr.~Jimenez.\\
Be sure to address:

\begin{itemize}
\item The influence of the three main experimental 
factors (Instrument, Harmony \& Voice);
\item A brief discussion of variance components---is this a standard repeated
measures model, or did we need to include other variance components?  
\item A discussion of other indivdual covariates in the model.
\end{itemize}

You may refer to your earlier work (e.g.~``As I showed in my answer to
part 2(b), blah-blah-blah\ldots''). Don't be sloppy about the
statistical findings, but try to highlight things that will be of
substantive interest to Dr.~Jimenez. Make your summary very readable and
clear. \label{p:6}

\it

The raw data provided by Dr.~Jimenez consisted of 2520 observations of
28 variable each, fully balanced across 70 listeners/subjects and 36
experimental conditions. Observations are groups of 36 audio music
stimuli, grouped by listener. So, as a multi-level data set, the level-1
observations are ratings of how Classical the music in each stimulus
sounds, and the level-2 groups are the set of stimuli that each listener
rated .

From the raw data, we created two data sets:

\begin{itemize}
\item `fulldata`, a data set consisting of all of the original variables except for `X1stInstr` and `X1stInstr`, because they had so much missing data (60\% and 87\%, respectively) that they were nearly useless.  We also removed 27 rows from `fulldata` that had missing Classical and Popular response values, and recode two responses of 19 to 10, assuming that the 19's were intended to be 10's but were keying errors (typo's) when entering the data.   This data set has 2493 observations of 26 variables, and is nearly balanced across the 70 subjects and 36 experimental conditions.
\item `nomissdata`, the largest subset of `fulldata` with no missing values.  This data set has 1541 complete observations of 36 variables.  It is reasonably well balanced across experimental conditions and subjects, but represents only 43 of the original 70 subjects.
\end{itemize}

The distributions of the variables in \texttt{fulldata} and
\texttt{nomissdata} are similar. Although two or three quantitative
variables exhibit noticable skewing, we decided not to do any
transformations, to make the results as immediately interpretable for
Dr.~Jimenez as possible.

With the \texttt{fulldata} data set, we explored the influence of the
three experimental factors, both fixed and random effects, on Classical
ratings.

With the \texttt{nomissdata} data set, we did variable selection on the
person covariate fixed effects and re-considered random effects in light
of a final person covariates model.

EDA and preparation of the data sets is shown in the work for Problem 1
above, pp.~\pageref{p:1}ff.~

\hypertarget{the-influence-of-the-three-main-experimental-factors-instrument-harmony-voice.}{%
\subsubsection{The influence of the three main experimental factors
(Instrument, Harmony \&
Voice).}\label{the-influence-of-the-three-main-experimental-factors-instrument-harmony-voice.}}

We considered the design factors Instrument, Harmony and Voice using
standard linear models with no random effects (Problem 2(a),
pp.~\pageref{p:2a}ff.), and using random-intercept multi-level models
(Problem 2(b)(ii), pp.~\pageref{p:2b.ii}ff.). Overall, the results were

\begin{itemize}
\item In both the standard linear models and in the multi-level models,
\begin{itemize}
\item The model selection criterion AIC (which approximately minimizes prediction error) strongly prefers a model with main effects for Instrument, Voice and Harmony, and an interaction between Voice and Harmony. 
\item The model selection criterion BIC (which approximately minmimizes mis-specification of the model) strongly prefers the model with just the main effects for Instrument, Voice and Harmony.
\end{itemize}
\item The model selection criterion DIC (which is designed to correctly count degrees of freedom when comparing models with different random effects) very strongly preferred models with a random intercept, rather than standard linear models without a random intercept (see Problem 2(b)(iii), pp. \pageref{p:2b.iii} above).
\end{itemize}

The final model from this exploration has main effects for Instrument,
Voice and Harmony, possibly an interaction between Voice and Harmony,
and a random intercept. This is a standard repeated-measures model,
where the value of the random intercept for each listener represents
that listener's overall tendency, over and above the features of the
stimulus, to rate any stimulus as sounding `Classical'.
Table\textasciitilde{}\ref{t:initial-models} summarizes the two models,
with and without the interaction of Voice and Harmony:

\begin{center}
\begin{longtable}{l D{)}{)}{9)0} D{)}{)}{9)0}}
\toprule
 & \multicolumn{1}{c}{Additive Fixed Effects} & \multicolumn{1}{c}{Additive Fixed Effects + Voice*Harmony} \\
\midrule
\endfirsthead
\toprule
 & \multicolumn{1}{c}{Additive Fixed Effects} & \multicolumn{1}{c}{Additive Fixed Effects + Voice*Harmony} \\
\midrule
\endhead
\bottomrule
\endfoot
\bottomrule
\caption{Initial Random-Intercept Models}
\label{t:initial-models}
\endlastfoot \\
(Intercept)               & 4.34 \; (0.19)  & 4.26 \; (0.21)  \\
Instrumentpiano           & 1.37 \; (0.09)  & 1.37 \; (0.09)  \\
Instrumentstring          & 3.12 \; (0.09)  & 3.12 \; (0.09)  \\
HarmonyI-V-IV             & -0.03 \; (0.11) & 0.15 \; (0.18)  \\
HarmonyI-V-VI             & 0.77 \; (0.11)  & 1.14 \; (0.18)  \\
HarmonyIV-I-V             & 0.04 \; (0.11)  & -0.18 \; (0.18) \\
Voicepar3rd               & -0.40 \; (0.09) & -0.28 \; (0.18) \\
Voicepar5th               & -0.36 \; (0.09) & -0.24 \; (0.18) \\
HarmonyI-V-IV:Voicepar3rd &                 & -0.35 \; (0.26) \\
HarmonyI-V-VI:Voicepar3rd &                 & -0.69 \; (0.26) \\
HarmonyIV-I-V:Voicepar3rd &                 & 0.53 \; (0.26)  \\
HarmonyI-V-IV:Voicepar5th &                 & -0.19 \; (0.26) \\
HarmonyI-V-VI:Voicepar5th &                 & -0.43 \; (0.26) \\
HarmonyIV-I-V:Voicepar5th &                 & 0.11 \; (0.26)  \\
\midrule
AIC                       & 10461.00        & 10455.90        \\
BIC                       & 10519.22        & 10549.04        \\
Log Likelihood            & -5220.50        & -5211.95        \\
Num. obs.                 & 2493            & 2493            \\
Num. groups: Subject      & 70              & 70              \\
Var: Subject (Intercept)  & 1.69            & 1.69            \\
Var: Residual             & 3.54            & 3.51            \\
\end{longtable}
\end{center}

\hypertarget{variance-components}{%
\subsubsection{Variance components}\label{variance-components}}

We then explored other random effects. Instead of a single random
intercept that varies from listener to listener, we considered an
expanded set of random effects allowing the coefficients on each level
of each design variable to vary from listener to listener as random
effects. This would allow us to accommodate variations in personal
tendencies to rate someting as classical, where those personal
tendencies vary from from one instrument to another, from one voice to
another, or from one harmony to another.

Exploration and comparison of models with various combinations of random
slopes on levels of the design variables (problem 2(c)(i),
pp.~\pageref{p:2c.i}ff.) led to the conclusion that random slopes on all
the design variables fit the data better. We continued to work with a
main effects only model, and a model that adds the Voice by Harmony
interaction. Because of a high degree of correlation between the random
effects, these models become somehat difficult to interpret. As an
alternative we also considered models in which the random effects for
different design variables are assumed to be uncorrelated. Models with
correlations forced to be zero, summarized in
Table\textasciitilde{}\ref{t:sel-on-design-REs}, do not fit as well as
models that leave all the correlations free, but they are more
interpretable and simpler to work with, so we will continue to assume
zero correlations between random effects for different desigbn factors.

\begin{center}
\begin{small}
\begin{longtable}{l D{)}{)}{9)0} D{)}{)}{9)0}}
\toprule
 & \multicolumn{1}{c}{Additive Fixed Effects} & \multicolumn{1}{c}{Additive Fixed Effects + Voice*Harmony} \\
\midrule
\endfirsthead
\toprule
 & \multicolumn{1}{c}{Additive Fixed Effects} & \multicolumn{1}{c}{Additive Fixed Effects + Voice*Harmony} \\
\midrule
\endhead
\bottomrule
\endfoot
\bottomrule
\caption{Models with expanded variance components}
\label{t:sel-on-design-REs}
\endlastfoot \\
(Intercept)                                      & 4.34 \; (0.23)  & 4.26 \; (0.24)  \\
Instrumentpiano                                  & 1.37 \; (0.17)  & 1.37 \; (0.17)  \\
Instrumentstring                                 & 3.12 \; (0.24)  & 3.12 \; (0.24)  \\
Voicepar3rd                                      & -0.40 \; (0.08) & -0.27 \; (0.16) \\
Voicepar5th                                      & -0.36 \; (0.08) & -0.24 \; (0.15) \\
HarmonyI-V-IV                                    & -0.03 \; (0.09) & 0.16 \; (0.15)  \\
HarmonyI-V-VI                                    & 0.77 \; (0.18)  & 1.14 \; (0.21)  \\
HarmonyIV-I-V                                    & 0.04 \; (0.09)  & -0.18 \; (0.15) \\
Voicepar3rd:HarmonyI-V-IV                        &                 & -0.37 \; (0.21) \\
Voicepar5th:HarmonyI-V-IV                        &                 & -0.19 \; (0.21) \\
Voicepar3rd:HarmonyI-V-VI                        &                 & -0.68 \; (0.21) \\
Voicepar5th:HarmonyI-V-VI                        &                 & -0.42 \; (0.21) \\
Voicepar3rd:HarmonyIV-I-V                        &                 & 0.53 \; (0.21)  \\
Voicepar5th:HarmonyIV-I-V                        &                 & 0.12 \; (0.21)  \\
\midrule
AIC                                              & 9952.54         & 9938.23         \\
BIC                                              & 10138.82        & 10159.43        \\
Log Likelihood                                   & -4944.27        & -4931.11        \\
Num. obs.                                        & 2493            & 2493            \\
Num. groups: Subject                             & 70              & 70              \\
Var: Subject (Intercept)                         & 0.00            & 0.00            \\
Var: Subject.1 Instrumentguitar                  & 1.33            & 1.32            \\
Var: Subject.1 Instrumentpiano                   & 1.01            & 1.01            \\
Var: Subject.1 Instrumentstring                  & 0.54            & 0.53            \\
Cov: Subject.1 Instrumentguitar Instrumentpiano  & 0.33            & 0.33            \\
Cov: Subject.1 Instrumentguitar Instrumentstring & -0.83           & -0.84           \\
Cov: Subject.1 Instrumentpiano Instrumentstring  & -0.22           & -0.23           \\
Var: Subject.2 Voicecontrary                     & 0.52            & 0.56            \\
Var: Subject.2 Voicepar3rd                       & 0.61            & 0.65            \\
Var: Subject.2 Voicepar5th                       & 0.62            & 0.66            \\
Cov: Subject.2 Voicecontrary Voicepar3rd         & 0.52            & 0.55            \\
Cov: Subject.2 Voicecontrary Voicepar5th         & 0.55            & 0.59            \\
Cov: Subject.2 Voicepar3rd Voicepar5th           & 0.61            & 0.65            \\
Var: Subject.3 HarmonyI-IV-V                     & 1.19            & 1.16            \\
Var: Subject.3 HarmonyI-V-IV                     & 1.51            & 1.49            \\
Var: Subject.3 HarmonyI-V-VI                     & 1.47            & 1.44            \\
Var: Subject.3 HarmonyIV-I-V                     & 1.24            & 1.21            \\
Cov: Subject.3 HarmonyI-IV-V HarmonyI-V-IV       & 1.33            & 1.30            \\
Cov: Subject.3 HarmonyI-IV-V HarmonyI-V-VI       & 0.52            & 0.49            \\
Cov: Subject.3 HarmonyI-IV-V HarmonyIV-I-V       & 1.21            & 1.18            \\
Cov: Subject.3 HarmonyI-V-IV HarmonyI-V-VI       & 0.71            & 0.68            \\
Cov: Subject.3 HarmonyI-V-IV HarmonyIV-I-V       & 1.34            & 1.31            \\
Cov: Subject.3 HarmonyI-V-VI HarmonyIV-I-V       & 0.54            & 0.51            \\
Var: Residual                                    & 2.40            & 2.36            \\
\end{longtable}
\end{small}
\end{center}

These models generalize the simple repeated measures model by allowing
listeners' individual tendencies to rate music as Classical to depend in
part on the features of the simulus music. In the remainder of the work
we used the simpler model on the left as a basis for further
exploration.

\hypertarget{person-level-covariates.}{%
\subsubsection{Person-level
covariates.}\label{person-level-covariates.}}

Backwards variable elimination using either AIC or BIC (Problem 3(a),
pp.~\pageref{p:3a} ff.), followed by a check of the random effects
(Problem 3(b), pp.~\pageref{p:3b}), leads to the same ``final'' model,
displayed in Table\textasciitilde{}\ref{t:sel-on-pers-covs}.

\begin{center}
\begin{longtable}{l D{)}{)}{9)0}}
\toprule
 & \multicolumn{1}{c}{Additive Fixed Effects} \\
\midrule
\endfirsthead
\toprule
 & \multicolumn{1}{c}{Additive Fixed Effects} \\
\midrule
\endhead
\bottomrule
\endfoot
\bottomrule
\caption{Model after selection of person covariates}
\label{t:sel-on-pers-covs}
\endlastfoot \\
(Intercept)                                      & 4.56 \; (0.82)  \\
HarmonyI-V-IV                                    & -0.00 \; (0.12) \\
HarmonyI-V-VI                                    & 0.85 \; (0.23)  \\
HarmonyIV-I-V                                    & 0.06 \; (0.12)  \\
Instrumentpiano                                  & 1.65 \; (0.24)  \\
Instrumentstring                                 & 3.59 \; (0.31)  \\
Voicepar3rd                                      & -0.41 \; (0.12) \\
Voicepar5th                                      & -0.30 \; (0.11) \\
X16.minus.17                                     & -0.18 \; (0.04) \\
ConsNotes                                        & -0.30 \; (0.07) \\
PachListen3                                      & -1.30 \; (0.58) \\
PachListen4                                      & 1.97 \; (0.82)  \\
PachListen5                                      & -0.51 \; (0.46) \\
ClsListen1                                       & -0.09 \; (0.31) \\
ClsListen3                                       & 0.69 \; (0.37)  \\
ClsListen4                                       & 6.43 \; (0.97)  \\
ClsListen5                                       & -0.05 \; (0.48) \\
KnowRob1                                         & -0.36 \; (0.43) \\
KnowRob5                                         & 1.54 \; (0.38)  \\
KnowAxis1                                        & -6.30 \; (1.21) \\
KnowAxis5                                        & -0.49 \; (0.29) \\
X1990s2000s                                      & -0.23 \; (0.10) \\
X1990s2000s.minus.1960s1970s                     & 0.23 \; (0.08)  \\
NoClass                                          & 0.74 \; (0.16)  \\
APTheory1                                        & 2.64 \; (0.33)  \\
Composing1                                       & -0.43 \; (0.27) \\
Composing2                                       & 1.06 \; (0.42)  \\
Composing3                                       & 0.04 \; (0.38)  \\
Composing4                                       & 0.71 \; (0.48)  \\
Composing5                                       & -2.76 \; (1.10) \\
GuitarPlay1                                      & 0.87 \; (0.53)  \\
GuitarPlay2                                      & 3.36 \; (0.50)  \\
GuitarPlay4                                      & 2.06 \; (0.88)  \\
GuitarPlay5                                      & -4.19 \; (0.67) \\
\midrule
AIC                                              & 6210.75         \\
BIC                                              & 6520.48         \\
Log Likelihood                                   & -3047.37        \\
Num. obs.                                        & 1541            \\
Num. groups: Subject                             & 43              \\
Var: Subject (Intercept)                         & 0.00            \\
Var: Subject.1 Instrumentguitar                  & 0.68            \\
Var: Subject.1 Instrumentpiano                   & 1.78            \\
Var: Subject.1 Instrumentstring                  & 1.38            \\
Cov: Subject.1 Instrumentguitar Instrumentpiano  & 0.23            \\
Cov: Subject.1 Instrumentguitar Instrumentstring & -0.86           \\
Cov: Subject.1 Instrumentpiano Instrumentstring  & 0.39            \\
Var: Subject.2 Voicecontrary                     & 0.17            \\
Var: Subject.2 Voicepar3rd                       & 0.00            \\
Var: Subject.2 Voicepar5th                       & 0.01            \\
Cov: Subject.2 Voicecontrary Voicepar3rd         & -0.02           \\
Cov: Subject.2 Voicecontrary Voicepar5th         & 0.04            \\
Cov: Subject.2 Voicepar3rd Voicepar5th           & -0.00           \\
Var: Subject.3 HarmonyI-IV-V                     & 0.14            \\
Var: Subject.3 HarmonyI-V-IV                     & 0.22            \\
Var: Subject.3 HarmonyI-V-VI                     & 1.21            \\
Var: Subject.3 HarmonyIV-I-V                     & 0.02            \\
Cov: Subject.3 HarmonyI-IV-V HarmonyI-V-IV       & 0.15            \\
Cov: Subject.3 HarmonyI-IV-V HarmonyI-V-VI       & -0.22           \\
Cov: Subject.3 HarmonyI-IV-V HarmonyIV-I-V       & 0.02            \\
Cov: Subject.3 HarmonyI-V-IV HarmonyI-V-VI       & -0.03           \\
Cov: Subject.3 HarmonyI-V-IV HarmonyIV-I-V       & -0.01           \\
Cov: Subject.3 HarmonyI-V-VI HarmonyIV-I-V       & -0.16           \\
Var: Residual                                    & 2.46            \\
\end{longtable}
\end{center}

This model fits the data well, relative to others considered, and has
good residual diagnostic plots. Once again, correlations between random
effects on different design variables are set to be zero here. Below, we
interpret the fixed effects in the model (see also Problem 3(c),
pp.~\pageref{p:3c}ff.):

\begin{itemize}
\item The effects with clearest interpretation:
\begin{itemize}
\item The harmony I-V-IV adds nearly a point on average to the Classical rating, vs. the "baseline" harmony I-IV-V.  The other harmonies do not really change the Classical rating much. \bf Listeners generally thing I-V-IV is a more classical sounding harmony structure.\it
\item The instrument "piano" adds over 1.5 points to the Classical rating, vs. the baseline instrument "guitar".  The instrument "string" adds over 3.5 points to the Classical rating, vs. "guitar".  \bf Listeners think piano, and especially strings, sound more classical than guitar.\it
\item The voice-leading patterns "parallel 3rds" and "parallel 5ths" reduce the Classical rating by roughly 0.4 and 0.3 points, respectively, relative to the baseline "contrary motion". \bf Listeners think contrary motion sounds mildly more classical than the other two voice-leading patterns.\it
\item We don't know what `X16.minus17` is, but it has a small negative effect on Classical ratings.
\item People who concentrate on the notes when listening have classical ratings which are on average about 0.3 lower ($\hat\beta_{ConsNotes}=-0.3$), than people who do not concentrate on the notes.
\end{itemize}
\item The effects with more ambiguous interpretation: 
\begin{itemize}
\item The pattern with PachListen (familiarity with Pachelbel's Canon) and ClsListen (how much do you listen to classical music) is not straightforward.  For example, listeners who scored 4 out of 5 on these two measures added nearly 2, and nearly 7 points, respectively, to their Classical ratings, on average, relative to the baseline category of 0 ( = `` not at all'') on each variable.  But listeners who scored 5 out of 5 on these measures actually reduced their Classical ratings somewhat, on average, relative to category 0.  However, these may be due in part to a small-sample bias issue, since only one listener scored 4 on either PachListen or ClsListen. 
\item The patterns with KnowRob (have you heard Rob Paravonian’s Pachelbel Rant) and KnowAxis (have you heard Axis of Evil’s Comedy bit on the 4 Pachelbel chords in popular music?) is more straightforward.  Relative to baseline category 0 (= ``not at all''),  listeners who express modest familiarity with the Rob Paravonian bit have lower Classical ratings by about 0.36 points, whereas those with modest familiarity with the Axis of Awesome bit, reduce their classical ratings by more than six points. The giant jump ($\hat\beta_{KNowAxis=1}-6.30$) may be mainly a small-sample bias issue, since only one listener scored in category 1 on this variable.    
\item Each additional point of familiarity with music from the 1990's conveys a small reduction in Classical ratings on average, but each additional point in the contrast between familiarity with music from the 1990's vs familiarity withmusic from the 1960's conveys a similar-sized increase in Classical ratings; it seems likely that these two effects basically cancel each other out.
\item Each additional Music class that a listener has taken increases their Classical scores by about 0.74; on the other hand, having taken AP Theory conveys a 2.64 increase in classical ratings. 
\item The pattern of estimated coefficients for Composing is again strange, with a score of 5 on the Composing variable associated with a 2.76 point drop in Classical ratings, breaking the pattern established by the other levels of Composing.  Once again, this may be mostly a small-sample bias issue since only one listener is in category 5 on the Composing variable.  
\item The pattern of coefficients for GuitarPlay is also hard to understand, but the coefficients for categories 2 and 4 may be in part a consequence of small sample bias.  
\end{itemize}
\end{itemize}

We also looked at whether a dichotomized version of the
\texttt{Selfdeclare} variable had interesting interactions with other
variables in the data set (Problem 4, pp.~\pageref{p:4}ff.). The
dichotomized variable we used was \[
     Musician_i = \left\{\begin{array}{rl}
                  1, & \mbox{\it if $Selfdeclare\leq c$}\\
                  0, & \mbox{\it otherwise}
                  \end{array}\right.
\] We found that if \(c\) is chosen so that about half the sample in
\texttt{nomissdata} consider themselves to be Musicians, then we get
interesting interactions with the three design variables (only). That
is, Musicians tend to rate certain instruments, voice leading or harmony
structures as classsical differently from how non-musicians rate them.

However, the presence or absence of interactions of the dichotomized
``Musician'' variable with design variables or with other person
covariates depends strongly on the value of \(c\) in the above equation.
Therefore, a final determination of the question of interactions of a
dichotomized version of the \texttt{Selfdeclare} with design factors and
other terms in the model should really wait for a careful conversation
with Dr.~Jimenez.

\hypertarget{future-work.}{%
\subsubsection{Future work.}\label{future-work.}}

Here, briefly, are two items to consider for future work:

\begin{itemize}
\item I did not have time to go back and incoprorate the interaction between Voice and Harmony in the later work above.  It's interesting to note that in the earlier work AIC always wants to include it, and BIC does not, and so ultimately including this interaction probably depends on whether the model is being built to produce good predictions, or to produce and accurately specified model that can help explain listeners' reactions to specific musical features.

\item I also did not try interactions among the person covariates in a model like that of Table~\ref{t:sel-on-pers-covs}, other than the ``Musician'' exploration above.  This would also be interesting to pursue further.

\item All listeners in this study also rated all musical stimuli on how Popular they sounded.  It would be very interesting to repeat all of the above analyses using Popular as the response variable, and compare those analyses with the analyses above.
\end{itemize}

\rm

\end{document}
