Results 1 to 3 of 3

Math Help - Need help on simple linear regression proof

  1. #1
    Newbie
    Joined
    Sep 2012
    From
    us
    Posts
    2

    Question Need help on simple linear regression proof

    I want to derive the least square estimation of the coefficients for y=B_0+B_1*x_1+e

    Can someone walk me through how to get from B to D in the image below?

    http://i.imgur.com/9zwvO.jpg


    I can do it through taking the derivative of the residual sum of squares, but I want to know how to do the proof through the general formula.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    MHF Contributor
    Prove It's Avatar
    Joined
    Aug 2008
    Posts
    11,589
    Thanks
    1445

    Re: Need help on simple linear regression proof

    I'm going to call \displaystyle \begin{align*} \mathbf{Y} = \left[ \begin{matrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{matrix} \right], \mathbf{X} = \left[ \begin{matrix} 1 & x_1 \\ 1 & x_2 \\ \vdots & \vdots \\ 1 & x_n \end{matrix} \right] \end{align*} and \displaystyle \begin{align*} \mathbf{\beta} = \left[ \begin{matrix} \beta_1 \\ \beta_2 \\ \vdots \\ \beta_n \end{matrix} \right] \end{align*}, then, assuming no error, we have the equation \displaystyle \begin{align*} \mathbf{Y} &= \mathbf{X}\mathbf{\beta} \end{align*}, with the aim to solve for \displaystyle \begin{align*} \mathbf{\beta} \end{align*}.

    Normally, what could be done is to premultiply both sides by \displaystyle \begin{align*} \mathbf{X}^{-1} \end{align*}, but we can't do that in this case because \displaystyle \begin{align*} \mathbf{X} \end{align*} is not a square matrix. But we can CREATE a square matrix by premultiplying both sides by \displaystyle \begin{align*} \mathbf{X}^T \end{align*} first. This gives the equation \displaystyle \begin{align*} \mathbf{X}^T\mathbf{Y} = \mathbf{X}^T\mathbf{X}\mathbf{\beta} \end{align*}.

    Now that \displaystyle \begin{align*} \mathbf{X}^T\mathbf{X} \end{align*} is a square matrix, to solve this equation for \displaystyle \begin{align*} \mathbf{\beta} \end{align*} we need to premultiply both sides by \displaystyle \begin{align*} \left( \mathbf{X}^T \mathbf{X} \right)^{-1}  \end{align*}, giving

    \displaystyle \begin{align*} \left( \mathbf{X}^T\mathbf{X} \right)^{-1} \mathbf{X}^T \mathbf{Y} &= \left( \mathbf{X}^T \mathbf{X} \right)^{-1} \mathbf{X}^T\mathbf{X} \mathbf{\beta} \\ \left( \mathbf{X}^T \mathbf{X} \right)^{-1} \mathbf{X}^T \mathbf{Y} &= \mathbf{I} \mathbf{\beta} \\ \left( \mathbf{X}^T \mathbf{X} \right)^{-1} \mathbf{X}^T \mathbf{Y} &= \mathbf{\beta} \end{align*}

    Did this help?
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Sep 2012
    From
    us
    Posts
    2

    Re: Need help on simple linear regression proof

    Edit. Thanks for the help, I solved the proof.
    Last edited by sabercn; September 16th 2012 at 06:40 AM.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. finding the p-value in simple linear regression
    Posted in the Statistics Forum
    Replies: 1
    Last Post: December 20th 2012, 05:37 AM
  2. Simple Linear Regression
    Posted in the Statistics Forum
    Replies: 0
    Last Post: October 28th 2010, 07:02 AM
  3. Regression fundamentals (linear, non linear, simple, multiple)
    Posted in the Advanced Statistics Forum
    Replies: 13
    Last Post: November 12th 2009, 06:47 AM
  4. SSE simple linear regression
    Posted in the Advanced Statistics Forum
    Replies: 2
    Last Post: October 27th 2009, 05:44 PM
  5. Simple Linear Regression
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: March 22nd 2008, 01:24 AM

Search Tags


/mathhelpforum @mathhelpforum