\documentclass[10pt]{article}

\usepackage{bj,txfonts,hyperref}

\frenchspacing
\def\vtrim{\vspace*{-0.8em}}

\begin{document}

\begin{center}
\Large
36-617: Applied Linear Models\\
Fall 2022\\
HW09 -- Due Fri Nov 18, 11:59pm\\
Grace until Sun Nov 20, 5:00pm
\end{center}

\section*{Announcements}

\begin{itemize}\itemsep=0pt\parskip=0pt

\item Please submit a single pdf to Gradescope using the link in the Assignments page on Canvas, as usual. 

\item There are six  exercises, beginning on p.~\pageref{exercises} below.  They are  intended to be raw materials for the {\bf Data}, {\bf Technical Appendix} and {\bf Discussion} sections of your final IDMRAD paper.  So save all of your work and results as raw materials for the project paper. 

\item As with all hw in this class, you {\em are} allowed to collaborate on this.  Please remember to list your collaborators and any other resources you used, in the pdf that you turn in.

\end{itemize}

\section*{Background And Data}

In 2012, Ivan Jimenez, a composer and musicologist visiting the University of Pittsburgh\footnote{Now at the Sibelius Academy, University of the Arts, Helsinki Finland.}, and student Vincent Rossi, collected data in a designed 
experiment inteneded to measure the influence of instrument, harmonic motion, 
and voice leading on listeners' identification of music as ``classical'' 
or ``popular''. 

They presented 36 musical stimuli to 70 listeners, recruited from the 
population of undergraduates at the University of Pittsburgh, and asked 
the listeners to rate the music on two different scales:
\begin{itemize}\itemsep=0pt\parskip=0pt
\item How classical does the music sound (1 to 10, 1 = not at all, 
10 = very classical sounding);
\item How popular does the music sound (1 to 10, 1 = not at all, 10 = very
popular sounding).
\end{itemize}
Listeners were told that a piece could be rated as both classical and popular, 
neither classical nor popular, or mostly classical and not popular (or vice
versa), so that the scales should have functioned more or less independently.

The 36 stimuli were chosen by completely crossing these factors:
\begin{description}\itemsep=0pt\parskip=0pt
\item[Instrument:] String Quartet, Piano, Electric Guitar
\item[Harmonic Motion:] I-V-vi, I-VI-V, I-V-IV, IV-I-V
\item[Voice Leading:] Contrary Motion, Parallel 3rds, Parallel 5ths
\end{description}
The researchers' main hypotheses were roughly these:
\begin{itemize}
\item Instrument should have the largest influence on rating;
\item One particular harmonic progression, I-V-vi, might be frequently rated
as classical, because it is the beginning progression for Pachelbel's 
{\em Canon in D}, which many people have heard.  On the other hand, it is 
also a very common chord progression in popular music of the past 20 years 
or so---so common that comedy bits have been written about it, e.g. by by the Axis of Awesome (\url{http://www.youtube.com/watch?v=oOlDewpCfZQ}) and by Rob Paravonian (\url{https://www.youtube.com/watch?v=JdxkVQy7QLM}).
\item Based on previous research, it was expected that contrary motion 
would also be frequently rated as classical.
\end{itemize}
A powerpoint presentation exploring the data (looking at cell means only, 
no model fits) is available in the subdirectory ``presentation'' in the
``hw09'' area of our canvas pages, in which some of these ideas are explored 
more fully\footnote{The data set you will be working with is slightly different 
from the one in the powerpoint presentation, so not all numerical summaries will
be the same.}.  You can also click on and listen to some of the stimuli in 
the powerpoint presentation\footnote{You will have to download the directory, 
and enable access to external content in the dialog box at the beginning
of the presentation, to hear the stimuli during the presentation (even so, I wasn't able to get all of the audio to work; ymmv).}.

A brief description of all variables in the data set follows:
\begin{verbatim}
X                                 Line number in the data set (IGNORE.)
Classical                         How classical does the stimulus sound?
Popular                           How popular does the stimulus sound?
Subject	                          Unique subject ID
Harmony	                          Harmonic Motion (4 levels)
Instrument                        Instrument      (3 levels)
Voice                             Voice Leading   (3 levels)
Selfdeclare                       Are you a musician? (1-6, 1=not at all)
OMSI                              Score on a test of musical knowledge
X16.minus.17                      Auxiliary measure of listener's ability to
                                     distinguish classical vs popular music
ConsInstr                         How much did you concentrate on the 
                                     instrument while listening (0-5, 0=not 
                                     at all)
ConsNotes                         How much did you concentrate on the 
                                     notes while listening? (0-5, 0=not at 
                                     all)
Instr.minus.Notes                 Difference between prev. two variables
PachListen                        How familiar are you with Pachelbel's Canon
                                     in D (0-5, 0=not at all)
ClsListen                         How much do you listen to classical music?
                                     (0-5, 0=not at all)
KnowRob                           Have you heard Rob Paravonian's Pachelbel Rant
                                     (0-5, 0=not at all)
KnowAxis                          Have you heard Axis of Evil's Comedy bit on
                                     the 4 Pachelbel chords in popular music?
                                     (0-5, 0=not at all)
X1990s2000s                       How much do you listen to pop and rock from 
                                      the 90's and 2000's? (0-5, 0=not at all)
X1990s2000s.minus.1960s1970s      Difference between prev variable and a 
                                      similar variable referring to 60's
                                      and 70's pop and rock.
CollegeMusic                      Have you taken music classes in college 
                                      (0=no, 1=yes)
NoClass                           How many music classes have you taken?
APTheory                          Did you take AP Music Theory class in
                                      High School (0=no, 1=yes)	
Composing                         Have you done any music composing (0-5,
                                      0=not at all)
PianoPlay                         Do you play piano (0-5, 0=not at all)	
GuitarPlay                        Do you play guitar (0-5, 0=not at all)
X1stInstr                         How proficient are you at your first 
                                      musical instrument (0-5, 0=not at all)
X2ndInstr                         Same, for second musical instrument
first12                           In the experiment, which instrument was
                                  presented to the subject in the first 
                                  12 stimuli? (IGNORE.)
\end{verbatim}
The data is in the file ``ratings.csv''in the hw09 area of the class website. 
While the data is fairly clean and well-organized, there are a few anomalies
that you will have to decide what to do with as you work through data analyses.
If you encounter any of these anomalies, please say how you dealt with each
of them (including, possibly, not changing the data at all).
Notes:
\begin{itemize}
\item You do not need to consider {\tt X} or {\tt first12} in any of your analyses.  
\item  {\tt Classical} and {\tt Popular} are both discrete, but you should treat them as continuous response variables in all of your analyses.
\end{itemize}

\section*{Exercises}\label{exercises}

\begin{enumerate}

\item[] {\em Your answer to question \#1 will be raw materials for the {\bf Data} section in your final IDMRAD paper.  Your answers to the other questions below will be most or all of the technical appendix for your final IDMRAD paper, and will be summarized in a possibly reorganized form in the {\bf Results} section of your paper.}

\item\label{p:0}  Explore the data, looking for usual or unusual relationships, usual or unusual data values, etc.  Perform any data cleaning, transformations, etc. that may be needed; justify your actions.  Submit a brief but interesting summary, with justifications.  \\[-1em]

In your work below, {\tt Classical} should not appear in any models for {\tt Popular}, nor vice-versa.  When evaluating models, please remember to look at appropriate plots as well as test statistics.

\item\label{p:1} \ul{\em The three main experimental factors.}
\begin{enumerate}

\item\label{p:lm} Examine the influence of the three main experimental 
factors (Instrument, Harmony \& Voice) on Classical ratings, using 
conventional linear models and/or analysis of variance models.  
Comment briefly on your findings, providing suitable brief evidence for each
result. {\em Hint: Since there are only three factors here, it is worth considering interactions of all orders.}

\item\label{p:lmer.1} Since we have approximately 36 ratings from each 
participant, we can expand the model in part~\ref{p:lm} by including a random intercept for each participant.  Such a model is sometimes called a ``repeated measures'' model. 
\begin{enumerate}
\item Write this expanded model in mathematical terms as a multi-level
model, using notation we have used in class.  Be sure to carefully identify what are the groups, and what are the individual observations.
\item\label{p:lmer.1.iii} Re-examine the influence of the three main experimental factors 
(Instrument, Harmony \& Voice) on Classical ratings (that is, repeat part~\ref{p:lm}), using models that always include the random intercept for participants\footnote{Please make sure the models are fitted with REML=FALSE (i.e. maximum likelihood) to ensure that the model comparisons make sense.}. 
\item Is the random effect needed?  Justify 
your answer with evidence.  You may (or may not) find valid evidence in fitted models, in statistical tests, and/or in fit indices.
\end{enumerate}

\item\label{p:lmer.2} The random intercept in a repeated measures model 
can account for ``personal biases'' in ratings: for example, perhaps person A is more
inclined to rate everything as classical, and person B is more inclined 
to rate everything as popular.  This can be accounted for by the random 
intercept.  Alternatively, perhaps personal biases vary with the type 
of instrument, type of harmony, and/or type of voice leading.  For 
example, perhaps people vary in the degree to which they are inclined 
to call music played by a string quartet ``classical''.  This suggests, 
e.g., a random effect of the form {\tt (Instrument|Subject)} or {\tt (0 + Instrument|Subject)} (and similarly for \verb|Harmony| and \verb|Voice|).  One could argue for a similar random effect for each person/harmony combination, and for each person/voice 
leading combination. %% \\ {\sl [Continued on next page...]}  
\begin{enumerate}

\item Determine whether the model in \ref{p:lmer.1.iii} is improved by adding one or more of these three new random effect terms.  Find the best combination of these terms\footnote{A good rule of thumb is to only include a term like (Voice$|$Subject) in the model if Voice is already a fixed effect.} and provide suitable evidence for your answer.

\item Re-examine the influence of the three main experimental factors 
(Instrument, Harmony \& Voice) on Classical ratings, using the 
random effects you found in part (i). Comment briefly on 
your findings, providing suitable brief evidence for each result. In 
addition, comment on the sizes of the variances of whichever random effects you added to the model, 
with respect to each other and with respect to the estimated residual 
variance.

\item Carefully write this model in mathematical terms as a hierarchical
linear model, using notation from class. Indicate values of estimated parameters (and their SE's, if available).

\end{enumerate}

\end{enumerate}
Because they are design variables in the experiment, the main effects for the three experimental
factors, Instrument, Harmony, and Voice, should be included in all models 
for the remainder of this homework, regardless of what you found about their
influence or lack of influence on ratings.  (Interactions among the design variables need not be included if you do not find them important, but the main effects for the design variables should {\bf always} be in your models.)

%\clearpage

\item\label{p:2} \ul{\em Person covariates.}  For this problem, begin 
with your best model from problem \ref{p:1}.
\begin{enumerate}

\item Determine which person covariates should be added to the 
model as fixed effects.  %% If you do this by hand, check your work with an automatic method.   If you do this with an automatic method, check to make sure that the choices make sense, musically and statistically.  
Show a suitable summary of your work, 
and list the final set of variables that you would 
include in the model.  {\em Hint: Some covariates that are actually
factor variables are coded as numeric.  Be careful to treat them as factors!}

\item Once the fixed effects are settled, go back and check to see whether
there should be any change in the random effects; focus on random slopes for design variables and their interactions.  Provide suitable 
evidence to justify your results.

\item\label{p:final-model} Briefly interpret the effect of each variable 
kept in the final model, on Classical ratings.

\end{enumerate}

\item\label{p:3} \ul{\em Musicians vs. Non-musicians.}  One of the secondary 
hypotheses of the researchers is that people who self-identify as musicians 
may be influenced by things that do not influence non-musicians.  Dichotomize
 ``Selfdeclare'' (``are you a musician?'') so 
that about half the participants are categorized as self-declared musicians, 
and half not. Examine and report on any interactions between the dichotomized 
musician variable and other predictors in the model.  Check to see if the results are sensitive to where you dichotomize.  Provide suitable evidence 
for, and comment on, your results.
% 
% \begin{enumerate}
% 
% \item Begin by dichotomizing ``Selfdeclare'' (``are you a musician?'') so 
% that about half the participants are categorized as self-declared musicians, 
% and half not. Examine and report on any interactions between the dichotomized 
% musician variable and other predictors in the model.  Provide suitable evidence 
% for, and comment on, your results.
% 
% \item Are there any other variables in the data set that could help to address
% this question?  If so, pursue a similar analysis with them.
%
% \end{enumerate}

\item\label{p:4} \ul{\em Classical vs. Popular.}  It would be totally appropriate to repeat problems \ref{p:2}, \ref{p:3} and \ref{p:4} using ``Popular'' ratings as the response, rather than classical, and then compare the results for ``Classical'' ratings, vs. ``Popular'' ratings.  {\em In the interest of time, {\bf you do not have to do this} for this hw/project.  It would make good ``future work'' for the paper; and/or it is something you could pursue on your own, if you are curious about it.}


\item \ul{\em Brief writeup.} Write an approximately one or two page professional-quality summary\footnote{This should form raw materials for the {\bf Discussion} section of your final IDMRAD paper, so please review what should be in the {\bf Discussion} section of an IMDRAD or IMRAD paper.  Also, try to keep this close to one or two pages.} 
of your findings, suitable for Dr. Jimenez.  
Be sure to address: %% t{\sl [Continues on next page...]}\clearpage
\begin{itemize}
\item The influence of the three main experimental 
factors (Instrument, Harmony \& Voice);
\item A brief discussion of variance components---is this a standard repeated
measures model, or did we need to include other variance components?  
\item A discussion of other indivdual covariates in the model.
\end{itemize}
You may
refer to your earlier work (e.g. ``As I showed in my answer to part \ref{p:lmer.1}, 
blah-blah-blah\ldots'').  Don't be sloppy about the statistical findings, but 
try to highlight things that will be of substantive interest to Dr. Jimenez.  Make
your summary very readable and clear.

\end{enumerate}

\end{document}
