Results 1 to 2 of 2

Math Help - Linear Regression: Mean square error (MSE)

  1. #1
    Senior Member
    Joined
    Jan 2009
    Posts
    404

    Linear Regression: Mean square error (MSE)

    Simple linear regression model:
    Y_i = β0 + β1*X_i + ε_i , i=1,...,n
    where n is the number of data points, ε_i is random error


    Let σ^2 = V(ε_i) = V(Y_i)


    Then an unbiased estimator of σ^2 is
    s^2 = (1/n-2)[∑(e_i)^2]

    where e_i's are the residuals

    s^2 is called the "mean square error" (MSE).


    My concerns:
    1) The GENERAL formula for sample variance is s^2 = (1/n-1)[∑(y_i - y bar)^2], it's defined on the first pages of my statistics textbook, I've been using this again and again, now I don't see how this general formula (which always holds) can reduce to the formula in red above? How come we have (n-2) and e_i in the formula for s^2?


    2) From what I've learnt in previous stat courses, the "mean square error" of a point estimator is by definition
    MSE(θ hat) = E[(θ hat - θ)^2]

    Is this the same MSE as the one in red above? Are they related at all?


    Any help is greatly appreciated!

    note: also under discussion in Talk Stats forum
    Last edited by kingwinner; May 21st 2009 at 11:49 PM.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Senior Member
    Joined
    Jan 2009
    Posts
    404
    My textbook also says that the sample s^2 = (1/n-1)[∑(y_i - y bar)^2] has n-1 in the denominator because it has n-1 degrees of freedom.

    And s^2 = (1/n-2)[∑(e_i)^2] has n-2 in the denominator because it has n-2 degrees of freedom.
    Now I am puzzled...what is "degrees of freedom"? Why does it have n-2 degrees of freedom? What is the simplest way to understand this?

    Thanks!
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Replies: 1
    Last Post: October 14th 2011, 11:20 AM
  2. [SOLVED] Regression analysis and the error term
    Posted in the Statistics Forum
    Replies: 2
    Last Post: July 6th 2011, 09:56 AM
  3. Linear Regression Equation - error term
    Posted in the Statistics Forum
    Replies: 2
    Last Post: April 23rd 2011, 09:55 PM
  4. Error analysis on linear regression
    Posted in the Statistics Forum
    Replies: 1
    Last Post: April 15th 2010, 11:35 PM
  5. Replies: 0
    Last Post: April 1st 2009, 05:22 AM

Search Tags


/mathhelpforum @mathhelpforum