\documentclass[12pt]{report}
\usepackage{graphicx}
\usepackage{psfrag}

\usepackage{psfig}
\setlength{\textwidth}{6.3in}
\setlength{\textheight}{8.5in}
\setlength{\topmargin}{-0.5in}
\setlength{\oddsidemargin}{0in}
\setlength{\evensidemargin}{0in}

\newcommand{\mbf}[1]{\mathbf{#1}}
\newcommand{\bvec}{\left[ \begin{array}{c}}
\newcommand{\evec}{\end{array} \right]}
\newcommand{\bmat}[1]{\left[ \begin{array}{#1}}
\newcommand{\emat}{\end{array} \right]}
\newcommand{\bg}[1]{\mbox{\boldmath $#1$}}
\newcommand{\cs}{, \ }

\begin{document}
\parindent0pt

\centerline{\bf \Large36-401 \ \ Chapter 5: Fitting
Curves (11/1/01)}
\centerline{\bf Kim Sellers and Brian Junker}
\bigskip

We experienced a bivariate form of a curvilinear regression in
Chapter 2.  In essence, we are interested in models that have an
{\it intrinsically linear form}:
$$
Y^*_i = \beta_0 + \beta_1 X^*_{i1} + \beta_2 X^*_{i2} + \ldots +
\beta_{K-1} X^*_{i,K-1} + \epsilon_i, 
$$
where $Y^*$ and $X^*_j, j=1,\ldots,K-1$ are functions of $Y$ and
$X_j, j=1,\ldots,K-1$ (e.g., $Y^* = Y^m$ for some $m$).  Such a
form allows us to use our current methodologies to solve
curvilinear regression problems.  However, models that cannot be
expressed as shown above require nonlinear regression
techniques.\\

Ways to fit curves to data:
\begin{itemize}
  \item Exploratory Band Regression: non-parametric guidance for 
  parametric model fitting 
    \begin{itemize}
      \item Steps:
         \begin{enumerate}
         \item Divide the scatterplot into a series of vertical
               bands
         \item Find central point (e.g., median) within each band
         \item Connect the resulting points to form curve
         \end{enumerate}
      \item Useful to search for patterns in any two-variable
scatterplot
      \item Guide for parametric model fitting
    \end{itemize}

%    \vspace*{-0.8in}
    \centerline{\psfig{file=crfe.ps,height=3.0in,angle=270}}

%  \vspace*{-1in}
  \item Regression with Transformed Variables
  \begin{itemize}
    \item Transformations are considered to linearize ${\bf X},Y$
relationship, sometimes at the risk of sacrificing normality,
homoscedasticity, etc.
    \item Steps:
    \begin{enumerate}
      \item Select appropriate transformations
      \item Generate transformed variables
      \item Using transformed variables, analyze data under new
model
      \item After obtaining results regarding relationships,
transform back to better interpret relations
    \end{enumerate}
  \end{itemize}

  \item Curvilinear Regression Models
    \begin{itemize}
      \item Example: Semilog curves and log-log curves have the forms 
            $\hat{Y}_i = b_0 + b_1 \log X_i$, $\log {\hat{Y_i}} = b_0 +
            b_1 X_i$, or $\log {\hat{Y_i}} = b_0 + b_1 \log X_i$

    \vspace*{-1in}
    \centerline{\resizebox{6.5in}{4.9in}{\includegraphics{semilog.ps}}}

    \vspace*{-0.8in}

    \item The models $Y_i=\alpha e^{\beta X_i}\epsilon_i$ and 
    $Y_i=\alpha  X_i^{\beta} \epsilon_i$ can be transformed using logs
    to form intrinsically linear models.
  
    \item Polynomial regression is a way to fit nonmonotonic curves
      \begin{itemize}
         \item Second order: ${\hat Y}_i = b_0 + b_1 X_i + b_2 X^2_i$
  \vspace*{1in}

         \item Third order: ${\hat Y}_i = b_0 + b_1 X_i + b_2 X^2_i +
b_3 X^3_i$ 
  \vspace*{1in}

         \item Higher order models may better fit the data, but are
difficult to understand
         \item Problems with polynomial regression:
            \begin{itemize}
              \item[] influence from outliers
              \item[] multicollinearity
            \end{itemize} 
               Thus, model is case dependent on its sample and may not
effectively represent the big picture.  Centering the $X$'s about their
mean $\bar X$ will reduce multicollinearity among powers of $X$ but
does not necessarily provide more precise estimates
      \end{itemize}
   \end{itemize}

\item Conditional Effect Plots
\begin{itemize}
\item Plots $X_j$ against predicted $Y$ with $X_i, i \ne j$ constant 
(substitute mean values for $X_i$'s) to develop relationship
between $X_j$ and $Y$; note that a different set of choices for $X_i$'s
only affects the intercept 
\item Comparing effects is easier for graphs containing the same $Y$
scale; this allows us to compare effect strength for linear,
curvilinear, or nonlinear models.

\end{itemize}

\item Non-linear Regression
\begin{itemize}
  \item Arguments for nonlinear modeling:
  \begin{itemize}
     \item Data suggest a specific nonlinear model
     \item Nonlinear equation expresses our understanding of the
mechanism causing changes in $Y$
  \end{itemize}

  \vspace{0.5in}
  \item Many nonlinear models to choose from.  Below are a few
examples:
\begin{itemize}
  \item[] Non-linear exponential models
   $$ Y_i=\alpha e^{\beta X_i} + \epsilon_i $$
  \vspace{0.5in}

  \item[] Negative exponential models (rise to asymptote with $\alpha,
\beta>0$)
   $$ Y_i=\alpha (1-e^{-\beta X_i}) + \epsilon_i $$
  \vspace{0.5in}

  \item[] Two term exponential models
   $$ Y_i=\frac{\theta_1}{\theta_1-\theta_2}
    \left(e^{-\theta_2 X_i}-e^{-\theta_1 X_i}\right) + \epsilon_i $$
  \vspace{0.5in}

  \item[] Logistic models (example of sigmoid or S-shaped curve)
   $$ Y_i=\frac{\alpha} {(1+\gamma e^{-\beta X_i})} + \epsilon_i $$
  \vspace{0.5in}

  \item[] Gompertz models (asymmetric S shape)
   $$ Y_i=\alpha e^{-\gamma e^{-\beta X_i}} + \epsilon_i $$
  \vspace{0.5in}

\end{itemize}

  \item Minimizing residual sum of squares requires iterative
techniques.  Unfortunately, methods aren't foolproof:
\begin{itemize}
      \item   Need starting values.
\item  May not converge.
\item  May converge to local minimum instead of global minimum.
\end{itemize}
Good initial estimates improve chances of an optimal nonlinear fit.
Approaches to obtain initial parameter estimates include using
comparable transformed linear form or estimating parameters from
scatterplots
  \item R$^2$ defined as $1-\frac{\mbox{RSS}}{\mbox{corrected TSS}}$.
May be negative.
\vspace*{1in}

  \item Under normal i.i.d. errors, estimates and tests are {\em
asymptotically} correct.  Use the usual graphs to check model assumptions. 

\end{itemize}


\end{itemize}




\end{document}



