Regression Analysis-Fall ’01: Solutions to HW5

November 16, 2001

1. RPD 4.3

(a) Regression Equation:

Y = 51.5697 + 1.4974 X_{1} + 6.7233 X_{2}

β_{0} : plant weight when there is neither organic matter nor supplemental nitrogen, measure in grams. β_{1} : increase in plant weight when organic matter increases by 1% (nitrogen supplement is kept fixed), measure in g/% β_{2} : increase in plant weight when nitrogen supplement is increased by 1 kg (organic matter is kept fixed), measure in g/kg

(b) SS(Res) has 7-2-1=4 df. s^{2}=SS(Res)/df=6.89, v̂ar( ˆ ) β = s^{2}(X^{T} X)^{−1} → var(β̂_{1}) = .0695, Cov(β̂_{1}, β̂_{2}) = −.0074

(c) Confidence Intervals:

• Univariate CI: β_{i} ± t_{(.025,4)}s(β̂_{i}). β_{1}: 1.4974 ± 0.7319. β_{2} : 6.7233 ± 1.7418

• Bonferroni CI: β_{i} ± t_{(.025/3,4)}s(β̂_{i}). β_{1} : 1.4974 ± 1.0441, β_{2} : 3.7233 ± 2.4848

• Scheffe CI: β̂_{i} ± \sqrt{3F_{(.05,3,4)}}s(β̂_{i}). β_{1} : 1.4974 ± 1.1722, β_{2} : 6.7233 ± 2.7897

(d) H_{0} : β_{1} = .5β_{2}, H_{1} : β_{1} ≠ .5β_{2} K = [0 1 − .5]^{T} , m = 0, t = (K^{T} β̂ − m)/\sqrt{(K^{T} (X^{T} X)^{−1}K)}s = −4.454. t_{critical} = −2.77 → reject H_{0}.

(e) H_{0} : β_{1} = .5β_{2}, H_{1} : β_{1} ≠ .5β_{2} Q = (K^{T} β̂−m)^{T} (K^{T} (X^{T} X)K)^{−1}(K^{T} β̂−m) = 136.79. F = Q/(rank(K)s^{2}) = 19.84. F_{critical} = 0.004 → reject H_{0}.

(f) Q = SSR(small) - SSR(big) = 164.3325-27.5808 = 136.75, same result as in (e).

2. RPD 4.12

• Sequential:

– R(β_{3}|β_{0}): Testing β_{3}=0 when only the intercept is already in the model.

– R(β_{1}|β_{0}, β_{3}): Testing β_{1}=0 when the intercept and X_{3} are already in the model.

– R(β_{2}|β_{0}, β_{1}, β_{3}): Testing β_{2}=0 when intecept, X_{1} and X_{3} are already in the model.

• Partial:

– R(β_{3}|β_{0}, β_{1}, β_{2}): Testing β_{3}=0 when intercept, X_{1} and X_{2} are already in the model.

– R(β_{2}|β_{0}, β_{1}, β_{3}): Testing β_{2}=0 when intercept, X_{1} and X_{3} are already in the model.

– R(β_{1}|β_{0}, β_{2}, β_{3}): Testing β_{1}=0 when intercept, X_{2} and X_{3} are already in the model.

3. Bank profits

(a) Profit or loss (Y) vs. size of bank (X_{1}) and type of bank (X_{2}, X_{3}).

Y = β + 0 + β_{1}X_{1} + β_{2}X_{2} + β_{3}X_{3}.

(b) Response functions:

• For commercial banks: Y = β_{0} + β_{1}X_{1} + β_{2}

• For mutual savings: Y = β_{0} + β_{1}X_{1} + β_{3}

• For savnigs and loan: Y = β_{0} + β_{1}X_{1} − β_{2} − β_{3}

(c) Interpretation: Additional gain or loss for being a particular type of bank.

4. RPD 7.13

(a) See HW3

(b) E[Y^{T} AY] = tr(A V ar(Y)) + E(Y^{T}) A E(Y)

E(σ̂^{2}) = E(Y^{T} (I − P_{X_{1}})Y/(n − p′))
= \frac{1}{n−p′} [tr[(I − P_{X_{1}})V ar(Y)] + E(Y^{T})(I − P_{X−1})E(Y)]
= \frac{1}{n−p′} [(n − tr(X_{1}(X_{1}^{T} X_{1})^{−1}X_{1}^{T}))σ^{2} + (X_{1}β_{1} + X_{2}β_{2})^{T} (I − P_{X_{1}})(X_{1}β_{1} + X_{2}β_{2})]
= \frac{1}{n−p′} [(n − p′)σ^{2} + (X_{1}β_{1})^{T} (I − P_{X_{1}})(X_{1}β_{1}) + (X_{2}β_{2})^{T} (I − P_{X_{1}})(X_{2}β_{2})]
= σ^{2} + (β_{2}^{T} X_{2}^{T} X_{2}β_{2} − β_{2}^{T} X_{2}^{T} P_{X_{1}}X_{2}β_{2})/(n − p′)
= σ^{2} + (X_{2}β_{2})^{T} (I − P_{X_{1}})(X_{2}β_{2})/(n − p′)
≥ σ^{2}since (I − P_{X_{1}})is nonegative definite andn > p′

When unbiased? (X_{2}β_{2})^{T} (I − P_{X_{1}})(X_{2}β_{2}) = 0: columns of X_{2} is in the span{columns of X_{1}}.

(c) X = [X_{1} X_{2}] → (X^{T} X)^{−1} = [ \begin{matrix}X_{1}^{T} X_{1} & X_{1}^{T} X_{2} \\ X_{2}^{T} X_{1} & X_{2}^{T} X_{2} \end{matrix} ]^{−1}

[ \begin{matrix}X_{1}^{T} X_{1} & X_{1}^{T} X_{2} \\ X_{2}^{T} X_{1} & X_{2}^{T} X_{2} \end{matrix} ] [ \begin{matrix}(X_{1}^{T} X_{1})^{−1} + AQ^{−1}A^{T} & −AQ^{−1} \\ −Q^{−1}A^{T} & Q^{−1} \end{matrix} ] = [ \begin{matrix}(1, 1) & (1, 2) \\ (2, 1) & (2, 2) \end{matrix} ]

Just carry out the matrix multiplication, we have: (1,1) = (X_{1}^{T} X_{1})(X_{1}^{T} X_{1})^{−1} + X_{1}^{T} X_{1}AQ^{−1}A^{T} + X_{1}^{T} X_{2}Q^{−1}A^{T} = I_{n−p′} (1,2) = … = 0_{n−p′} (2,1) = … = 0_{p′} (2,2) = (−X_{2}^{T} X_{1} + X_{2}T X_{2})Q^{−1} = QQ^{−1} = I_{p′}

Therefore, [ \begin{matrix}(X_{1}^{T} X_{1})^{−1} + AQ^{−1}A^{T} & −AQ^{−1} \\ −Q^{−1}A^{T} & Q^{−1} \end{matrix} ] = [ \begin{matrix}X_{1}^{T} X_{1} & X_{1}^{T} X_{2} \\ X_{2}^{T} X_{1} & X_{2}^{T} X_{2} \end{matrix} ]^{−1}

(d) Var(β̂_{1}) = (X_{1}^{T} X_{1})^{−1}V ar(Y)X_{1}(X_{1}^{T} X_{1})^{−1} = σ^{2}(X_{1}^{T} X_{1})^{−1} (subset model). From (c),

V ar(β̂_{1}) = σ^{2}(X_{1}^{T} X_{1})^{−1} + σ^{2}AQ^{−1}A^{T}
= σ^{2}(X_{1}^{T} X_{1})^{−1} + σ^{2}(X_{1}^{T} X_{1})^{−1}X_{1}T (I − P_{X_{1}})^{−1}X_{1}(X_{1}^{T} X_{1})^{−1}
≥ σ^{2}(X_{1}^{T} X_{1})^{−1}

As a result, β_{1} in the subset model has a smaller variance than that in the full model. Bias/Variance tradeoff: I personally like estimators with smaller variances because I have a better feel of the variance than the expected value. However, if I include too many extra variables in the model, I will end up other problems such as colinearity.