Results 1 to 3 of 3

Math Help - unbiased estimator

  1. #1
    Member
    Joined
    Nov 2006
    Posts
    142

    unbiased estimator

    I want to show that the following estimator is unbiased:

    s^2 = SSE/(n-2) = [sum from i = 1 to n of (yi - yi*)^2]/(n-2) = [sum from i = 1 to n of (yi - b0 - b1xi)^2]/(n-2)

    For it to be unbiased E(s^2) must equal sigma^2.

    I know E(yi) = beta0 + beta1*xi, Var(yi) = sigma^2, E(b1) = beta1, E(b0) = beta0

    I've tried working on this (it's difficult for me to write out all my work), but I get lost trying to calculate E(b0b1) and other E's for instance.

    Can someone show me how to do this?
    Thanks in advance for any help.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Flow Master
    mr fantastic's Avatar
    Joined
    Dec 2007
    From
    Zeitgeist
    Posts
    16,948
    Thanks
    5
    Quote Originally Posted by PvtBillPilgrim View Post
    I want to show that the following estimator is unbiased:

    s^2 = SSE/(n-2) = [sum from i = 1 to n of (yi - yi*)^2]/(n-2) = [sum from i = 1 to n of (yi - b0 - b1xi)^2]/(n-2)

    For it to be unbiased E(s^2) must equal sigma^2.

    I know E(yi) = beta0 + beta1*xi, Var(yi) = sigma^2, E(b1) = beta1, E(b0) = beta0

    I've tried working on this (it's difficult for me to write out all my work), but I get lost trying to calculate E(b0b1) and other E's for instance.

    Can someone show me how to do this?
    Thanks in advance for any help.
    You need to show that E[SSE] = (n-2) \sigma^2. I'll take you most of the way but there'll be a point where I become latex intolerant and that's where you'll need to finish it off .....

    The following standard and well known results (0, 1, 4 and 5 are standard linear regression formulae) will be used:

    0. b_0 = \bar{y} - b_1 \bar{x}.


    1. \sum_{i=1}^{n} (x_i - \bar{x}) (y_i - \bar{y}) = b_1 \, \sum_{i=1}^{n} (x_i - \bar{x})^2.


    2. \sum_{i=1}^{n} (y_i - \bar{y})^2 = \left( \sum_{i=1}^{n} y_i^2 \right) - n \bar{y}^2.


    3. Var[W] = E[W^2] - (E[W])^2 where W is any random variable. (I would've used U here but it looks like there's some sort of latex glitch which keeps turning U into u inside Var - quite annoying).


    4. E[b_1] = \beta_1.


    5. Var[b_1] = \frac{\sigma^2}{\sum_{i=1}^{n} (x_i - \bar{x})^2}.



    E[SSE] = E\left[ \sum_{i=1}^{n} (y_i - b_0 - b_1 x_i)^2\right]

    Substitute standard and well known result 0:


    = E\left[ \sum_{i=1}^{n} (y_i - ( \bar{y} - b_1 \bar{x} ) - b_1 x_i)^2\right] = E\left[ \sum_{i=1}^{n} (y_i - \bar{y} + b_1 \bar{x} - b_1 x_i)^2\right]


    = E\left[ \sum_{i=1}^{n} ( [y_i - \bar{y}] - b_1 [x_i - \bar{x}])^2\right]


    = E\left[ \sum_{i=1}^{n} [y_i - \bar{y}]^2 + b_1^2 \sum_{i=1}^{n} [x_i - \bar{x}]^2 - 2 b_1 \sum_{i=1}^{n} (x_i - \bar{x}) (y_i - \bar{y}) \right]

    Substitute from standard and well known results 1. and 2:


    = E\left[ \left( \sum_{i=1}^{n} y_i^2 \right) - n \bar{y}^2 - b_1^2 \sum_{i=1}^{n} (x_i - \bar{x})^2 \right]


    = \left( \sum_{i=1}^{n} E\left[ y_i^2 \right] \right) - n E\left[\bar{y}^2\right] - \sum_{i=1}^{n} (x_i - \bar{x})^2 E\left[ b_1^2 \right]

    Use standard and well known result 3:


    = \left( \sum_{i=1}^{n} Var[y_i] + \left( E\left[ y_i \right]\right)^2 \right) - n \left( Var [\bar{y}]  + \left( E \left[ \bar{y}\right] \right)^2 \right) - \sum_{i=1}^{n} (x_i - \bar{x})^2 \left( Var[ b_1] + \left( E[b_1]\right)^2 \right)


    = n \sigma^2 + \sum_{i=1}^{n} (\beta_0 + \beta_1 x_i)^2 - n \left( \frac{\sigma^2}{n} + (\beta_0 + \beta_1 \bar{x})^2 \right) -  \sum_{i=1}^{n} (x_i - \bar{x})^2 \left( \frac{\sigma^2}{\sum_{i=1}^{n} (x_i - \bar{x})^2} + \beta_1^2\right)


    where I've substituted standard and well known results 4. and 5. Note that E[y_i] = E[\beta_0 + \beta_1 x_i] = \beta_0 + \beta_1 x_i.

    And at this point I've become quite latex intolerant. You should be able to show that all this simplifies to (n-2) \sigma^2 \, ....
    Last edited by mr fantastic; September 14th 2008 at 02:46 AM. Reason: Slight re-organisation, content unchanged
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Oct 2012
    From
    South Africa
    Posts
    2

    Re: unbiased estimator

    Hey, I managed to get all the way to the last step on my own but i cant figure out the silliest thing! X_X
    Could you please show all your steps regarding the last step (1/(n-2))(sigma^2)
    What happens with the Sxx(b1^2)??
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. unbiased estimator
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: November 23rd 2010, 07:36 AM
  2. unbiased estimator of μ1 μ2
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: March 24th 2010, 12:45 AM
  3. Estimator Unbiased?
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: February 18th 2010, 04:58 PM
  4. Unbiased Estimator
    Posted in the Advanced Statistics Forum
    Replies: 4
    Last Post: December 2nd 2009, 12:51 AM
  5. Help me about best unbiased estimator, thanks
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: May 3rd 2009, 02:52 PM

Search Tags


/mathhelpforum @mathhelpforum