# unbiased estimator

• Sep 13th 2008, 06:36 AM
PvtBillPilgrim
unbiased estimator
I want to show that the following estimator is unbiased:

s^2 = SSE/(n-2) = [sum from i = 1 to n of (yi - yi*)^2]/(n-2) = [sum from i = 1 to n of (yi - b0 - b1xi)^2]/(n-2)

For it to be unbiased E(s^2) must equal sigma^2.

I know E(yi) = beta0 + beta1*xi, Var(yi) = sigma^2, E(b1) = beta1, E(b0) = beta0

I've tried working on this (it's difficult for me to write out all my work), but I get lost trying to calculate E(b0b1) and other E's for instance.

Can someone show me how to do this?
Thanks in advance for any help.
• Sep 13th 2008, 09:39 PM
mr fantastic
Quote:

Originally Posted by PvtBillPilgrim
I want to show that the following estimator is unbiased:

s^2 = SSE/(n-2) = [sum from i = 1 to n of (yi - yi*)^2]/(n-2) = [sum from i = 1 to n of (yi - b0 - b1xi)^2]/(n-2)

For it to be unbiased E(s^2) must equal sigma^2.

I know E(yi) = beta0 + beta1*xi, Var(yi) = sigma^2, E(b1) = beta1, E(b0) = beta0

I've tried working on this (it's difficult for me to write out all my work), but I get lost trying to calculate E(b0b1) and other E's for instance.

Can someone show me how to do this?
Thanks in advance for any help.

You need to show that $\displaystyle E[SSE] = (n-2) \sigma^2$. I'll take you most of the way but there'll be a point where I become latex intolerant and that's where you'll need to finish it off .....

The following standard and well known results (0, 1, 4 and 5 are standard linear regression formulae) will be used:

0. $\displaystyle b_0 = \bar{y} - b_1 \bar{x}$.

1. $\displaystyle \sum_{i=1}^{n} (x_i - \bar{x}) (y_i - \bar{y}) = b_1 \, \sum_{i=1}^{n} (x_i - \bar{x})^2$.

2. $\displaystyle \sum_{i=1}^{n} (y_i - \bar{y})^2 = \left( \sum_{i=1}^{n} y_i^2 \right) - n \bar{y}^2$.

3. $\displaystyle Var[W] = E[W^2] - (E[W])^2$ where $\displaystyle W$ is any random variable. (I would've used $\displaystyle U$ here but it looks like there's some sort of latex glitch which keeps turning $\displaystyle U$ into $\displaystyle u$ inside Var - quite annoying).

4. $\displaystyle E[b_1] = \beta_1$.

5. $\displaystyle Var[b_1] = \frac{\sigma^2}{\sum_{i=1}^{n} (x_i - \bar{x})^2}$.

$\displaystyle E[SSE] = E\left[ \sum_{i=1}^{n} (y_i - b_0 - b_1 x_i)^2\right]$

Substitute standard and well known result 0:

$\displaystyle = E\left[ \sum_{i=1}^{n} (y_i - ( \bar{y} - b_1 \bar{x} ) - b_1 x_i)^2\right] = E\left[ \sum_{i=1}^{n} (y_i - \bar{y} + b_1 \bar{x} - b_1 x_i)^2\right]$

$\displaystyle = E\left[ \sum_{i=1}^{n} ( [y_i - \bar{y}] - b_1 [x_i - \bar{x}])^2\right]$

$\displaystyle = E\left[ \sum_{i=1}^{n} [y_i - \bar{y}]^2 + b_1^2 \sum_{i=1}^{n} [x_i - \bar{x}]^2 - 2 b_1 \sum_{i=1}^{n} (x_i - \bar{x}) (y_i - \bar{y}) \right]$

Substitute from standard and well known results 1. and 2:

$\displaystyle = E\left[ \left( \sum_{i=1}^{n} y_i^2 \right) - n \bar{y}^2 - b_1^2 \sum_{i=1}^{n} (x_i - \bar{x})^2 \right]$

$\displaystyle = \left( \sum_{i=1}^{n} E\left[ y_i^2 \right] \right) - n E\left[\bar{y}^2\right] - \sum_{i=1}^{n} (x_i - \bar{x})^2 E\left[ b_1^2 \right]$

Use standard and well known result 3:

$\displaystyle = \left( \sum_{i=1}^{n} Var[y_i] + \left( E\left[ y_i \right]\right)^2 \right) - n \left( Var [\bar{y}] + \left( E \left[ \bar{y}\right] \right)^2 \right) - \sum_{i=1}^{n} (x_i - \bar{x})^2 \left( Var[ b_1] + \left( E[b_1]\right)^2 \right)$

$\displaystyle = n \sigma^2 + \sum_{i=1}^{n} (\beta_0 + \beta_1 x_i)^2 - n \left( \frac{\sigma^2}{n} + (\beta_0 + \beta_1 \bar{x})^2 \right) - \sum_{i=1}^{n} (x_i - \bar{x})^2 \left( \frac{\sigma^2}{\sum_{i=1}^{n} (x_i - \bar{x})^2} + \beta_1^2\right)$

where I've substituted standard and well known results 4. and 5. Note that $\displaystyle E[y_i] = E[\beta_0 + \beta_1 x_i] = \beta_0 + \beta_1 x_i$.

And at this point I've become quite latex intolerant. You should be able to show that all this simplifies to $\displaystyle (n-2) \sigma^2 \, ....$
• Oct 24th 2012, 01:12 PM
ShaunRLS
Re: unbiased estimator
Hey, I managed to get all the way to the last step on my own but i cant figure out the silliest thing! X_X
Could you please show all your steps regarding the last step (1/(n-2))(sigma^2)
What happens with the Sxx(b1^2)??