unbiased estimator

**PvtBillPilgrim** · September 13th 2008, 06:36 AM

I want to show that the following estimator is unbiased:

s^2 = SSE/(n-2) = [sum from i = 1 to n of (yi - yi*)^2]/(n-2) = [sum from i = 1 to n of (yi - b0 - b1xi)^2]/(n-2)

For it to be unbiased E(s^2) must equal sigma^2.

I know E(yi) = beta0 + beta1*xi, Var(yi) = sigma^2, E(b1) = beta1, E(b0) = beta0

I've tried working on this (it's difficult for me to write out all my work), but I get lost trying to calculate E(b0b1) and other E's for instance.

Can someone show me how to do this?
Thanks in advance for any help.

**mr fantastic** · September 13th 2008, 09:39 PM

Originally Posted by PvtBillPilgrim

I want to show that the following estimator is unbiased:

s^2 = SSE/(n-2) = [sum from i = 1 to n of (yi - yi*)^2]/(n-2) = [sum from i = 1 to n of (yi - b0 - b1xi)^2]/(n-2)

For it to be unbiased E(s^2) must equal sigma^2.

I know E(yi) = beta0 + beta1*xi, Var(yi) = sigma^2, E(b1) = beta1, E(b0) = beta0

I've tried working on this (it's difficult for me to write out all my work), but I get lost trying to calculate E(b0b1) and other E's for instance.

Can someone show me how to do this?
Thanks in advance for any help.

You need to show that $E[SSE] = (n-2) \sigma^2$ . I'll take you most of the way but there'll be a point where I become latex intolerant and that's where you'll need to finish it off .....

The following standard and well known results (0, 1, 4 and 5 are standard linear regression formulae) will be used:

0. $b_0 = \bar{y} - b_1 \bar{x}$ .

1. $\sum_{i=1}^{n} (x_i - \bar{x}) (y_i - \bar{y}) = b_1 \, \sum_{i=1}^{n} (x_i - \bar{x})^2$ .

2. $\sum_{i=1}^{n} (y_i - \bar{y})^2 = \left( \sum_{i=1}^{n} y_i^2 \right) - n \bar{y}^2$ .

3. $Var[W] = E[W^2] - (E[W])^2$ where $W$ is any random variable. (I would've used $U$ here but it looks like there's some sort of latex glitch which keeps turning $U$ into $u$ inside Var - quite annoying).

4. $E[b_1] = \beta_1$ .

5. $Var[b_1] = \frac{\sigma^2}{\sum_{i=1}^{n} (x_i - \bar{x})^2}$ .

$E[SSE] = E\left[ \sum_{i=1}^{n} (y_i - b_0 - b_1 x_i)^2\right]$

Substitute standard and well known result 0:

$= E\left[ \sum_{i=1}^{n} (y_i - ( \bar{y} - b_1 \bar{x} ) - b_1 x_i)^2\right] = E\left[ \sum_{i=1}^{n} (y_i - \bar{y} + b_1 \bar{x} - b_1 x_i)^2\right]$

$= E\left[ \sum_{i=1}^{n} ( [y_i - \bar{y}] - b_1 [x_i - \bar{x}])^2\right]$

$= E\left[ \sum_{i=1}^{n} [y_i - \bar{y}]^2 + b_1^2 \sum_{i=1}^{n} [x_i - \bar{x}]^2 - 2 b_1 \sum_{i=1}^{n} (x_i - \bar{x}) (y_i - \bar{y}) \right]$

Substitute from standard and well known results 1. and 2:

$= E\left[ \left( \sum_{i=1}^{n} y_i^2 \right) - n \bar{y}^2 - b_1^2 \sum_{i=1}^{n} (x_i - \bar{x})^2 \right]$

$= \left( \sum_{i=1}^{n} E\left[ y_i^2 \right] \right) - n E\left[\bar{y}^2\right] - \sum_{i=1}^{n} (x_i - \bar{x})^2 E\left[ b_1^2 \right]$

Use standard and well known result 3:

$= \left( \sum_{i=1}^{n} Var[y_i] + \left( E\left[ y_i \right]\right)^2 \right) - n \left( Var [\bar{y}] + \left( E \left[ \bar{y}\right] \right)^2 \right) - \sum_{i=1}^{n} (x_i - \bar{x})^2 \left( Var[ b_1] + \left( E[b_1]\right)^2 \right)$

$= n \sigma^2 + \sum_{i=1}^{n} (\beta_0 + \beta_1 x_i)^2 - n \left( \frac{\sigma^2}{n} + (\beta_0 + \beta_1 \bar{x})^2 \right) - \sum_{i=1}^{n} (x_i - \bar{x})^2 \left( \frac{\sigma^2}{\sum_{i=1}^{n} (x_i - \bar{x})^2} + \beta_1^2\right)$

where I've substituted standard and well known results 4. and 5. Note that $E[y_i] = E[\beta_0 + \beta_1 x_i] = \beta_0 + \beta_1 x_i$ .

And at this point I've become quite latex intolerant. You should be able to show that all this simplifies to $(n-2) \sigma^2 \, ....$

**ShaunRLS** · October 24th 2012, 01:12 PM

Hey, I managed to get all the way to the last step on my own but i cant figure out the silliest thing! X_X
Could you please show all your steps regarding the last step (1/(n-2))(sigma^2)
What happens with the Sxx(b1^2)??

Thread: unbiased estimator

LinkBack

Thread Tools

Display

unbiased estimator

Re: unbiased estimator

Similar Math Help Forum Discussions

unbiased estimator

unbiased estimator of μ1 – μ2

Estimator Unbiased?

Unbiased Estimator

Help me about best unbiased estimator, thanks

Search Tags