Comments on: Thurs Mar 22

By: Thomas Kraynak

Thomas Kraynak — Thu, 22 Mar 2012 13:30:27 +0000

The procedures seem pretty straightforward when it comes to performing multiple regression using two, three, or four explanatory variables as related to doing a simple linear regression with one variable. Are there issues associated with using more variables than that? What is the “limit” of number of multiple regression variables typically used in the literature?

By: Noah

Noah — Thu, 22 Mar 2012 13:05:07 +0000

When the F-statistic does not follow a F-distribution, is it completely useless? Or, can a different distribution be fit to the available F-statistics in order to make conclusions about the data? I would assume this would require being able to generate a large number of F-statistics (like in the HW), is that correct?

By: Amanda Markey

Amanda Markey — Thu, 22 Mar 2012 12:42:53 +0000

We talked about corrections for multiple hypothesis tests yesterday. Is there a penalty for generating many multiple regressions (each time changing which x’s are included)?

By: David Zhou

David Zhou — Thu, 22 Mar 2012 12:40:56 +0000

Does the least squares fit principle basically say that if you have over constraining variables, you’re never going to find a linear combination of them to exact regress to a line, so you project to as many dimensions as you can? I think I’m familiar with the linear algebra behind it, but can you go over the statistical principles involved?

(This is one section ahead I realize.) How do you use polynomial and cosine regression? Do you have to assume that some model fits ad hoc? Is there any way to compute the order of the polynomials, or is it something you just fit and compare? I guess this goes back to what you say – all models are wrong, but some are useful?

By: Yijuan Du

Yijuan Du — Thu, 22 Mar 2012 12:33:51 +0000

How to represent the interaction of these explanatory variables?

By: Matt Panico

Matt Panico — Thu, 22 Mar 2012 08:07:45 +0000

When bootstrapping multivariate regressions, the bootstrapped sample sets will almost always generate a better fit (and smaller residuals) than the original data, due to repeated data. Does this cause an issue with analysis?

By: Rex Tien

Rex Tien — Thu, 22 Mar 2012 05:38:09 +0000

I am still not quite sure how to calculate the sigma^2 and s^2 in equation 12.56. Which x-values does it arise from?

Also, I appreciated the geometric argument in helping understand the matrix form of the least-squares fit. However, why does this insight apply to least squares and not other methods like least-absolute value?

By: Ben Dichter

Ben Dichter — Thu, 22 Mar 2012 05:38:08 +0000

Is linear regression always performed assuming normal noise? Are there instances when you would use a different noise model and would you be able to find a closed form solution? Would you be able to perform an analogous ANOVA?

By: Ben Dichter

Ben Dichter — Thu, 22 Mar 2012 05:37:23 +0000

Is linear regression always performed assuming normal error? Are there instances when you would use a different error estimate and would you be able to find a closed form solution?

By: Shubham Debnath

Shubham Debnath — Thu, 22 Mar 2012 04:56:51 +0000

When using matrices for multiple regression, is it helpful to reduce a matrix to upper echelon or echelon form? And how can one apply nonlinear regression models similarly?