So I can prove that the sum of the residuals equals zero.. but how do I prove that the mean of e equals zero? (e is the residuals in this case, not the irrational e)
I've tried but all I end up doing is going around in a circle.
So I can prove that the sum of the residuals equals zero.. but how do I prove that the mean of e equals zero? (e is the residuals in this case, not the irrational e)
I've tried but all I end up doing is going around in a circle.
Let R be the random variable denoting the residual error. Then
$\displaystyle R_i = X_i - \frac{1}{n}\sum_{j=1}^n X_j, i=1, ... ,n$
Then it follows that
$\displaystyle E[R_i] = E[X_i] - \frac{1}{n} \sum_{j=1}^n E[X_j]$
$\displaystyle E[R_i] = \mu - \frac{1}{n} \sum_{j=1}^n \mu = 0$
I proved the sum of the residuals is zero in another post here.
BUT it's not true if you don't have an intercept term in your model.
HERE it is
http://www.mathhelpforum.com/math-he...-question.html
Did you even look at http://www.mathhelpforum.com/math-he...-question.html?
I do not even know what your model is, nor if you understand how to do least squares with matrices.
But here's simple proof for ANY model that the expected value of the residuals is zero.
The general model is $\displaystyle Y=X\beta +\epsilon$ where the $\displaystyle \epsilon$ is a column vector usually normal, with ZERO means.
HENCE $\displaystyle E(Y)=X\beta +E(\epsilon)=X\beta$
NOW the least squares fit is $\displaystyle \hat\beta=(X^tX)^{-1}X^tY$
and it's real easy to prove that $\displaystyle E(\hat\beta)=\beta$.
Our least squares fit is $\displaystyle \hat Y=X\hat\beta$ and $\displaystyle E(\hat Y)=XE(\hat\beta)=X\beta$.
SO the expected value of our residuals is $\displaystyle X\beta -X\beta=0$.
I'm flattered, but I don't know what I did.
Don't let Mr Fantasy see your comments, he thinks I'm a prima dona.
(And MOO may also be jealous too, so keep your comments to a minimum.)
I did misread your earlier question.
I thought that you wanted to see that the sum was zero.
That's the usual question here.
Without knowing your model I wasn't sure what you wanted me to do.
I can easily prove this for any model w/o matrices.
BUT matrices is the way to go.
I'm typing the exam I'm giving on this topic right now.
Is it, uhh... at all possible for you to prove it without matrices?
Just that this question was given to us before we started using matrices, so I can't use matrices.
And as for the model, our model for the regression line is:
$\displaystyle y=\hat\beta_0+\hat\beta_1 x$
with residual:
$\displaystyle e_i=y_i-\hat y_i$
Sorry I didn't give that before.
Sure, it works either way.
BUT it doesn't matter what your model is.
Have you proved that $\displaystyle E(\hat\beta_0) =\beta_0$ and $\displaystyle E(\hat\beta_1) =\beta_1$ that the estimators are unbiased for these parameters?
Your original model is $\displaystyle Y=\beta_0+\beta_1 x+\epsilon$
where the only random variable is the $\displaystyle \epsilon$ and it has mean zero.
THUS $\displaystyle E(Y)=\beta_0+\beta_1 x$.
NOW if you have proved $\displaystyle E(\hat\beta_0) =\beta_0$ and $\displaystyle E(\hat\beta_1) =\beta_1$ it follows from $\displaystyle \hat Y=\hat\beta_0+\hat\beta_1 x$ that $\displaystyle E(\hat Y)=\beta_0+\beta_1 x$.
I just did the scratch work to prove $\displaystyle E(\hat\beta_0) =\beta_0$ and $\displaystyle E(\hat\beta_1) =\beta_1$ via the SSxy and SSxx formulas.
do you need this with $\displaystyle \hat\beta_0 =\bar Y-\hat\beta_1\bar x$ and $\displaystyle \hat\beta_1 ={SS_{xy}\over SS_{xx}}$...
Start with $\displaystyle Y_i=\beta_0 +\beta_1 x_i+\epsilon_i$ and $\displaystyle \bar Y=\beta_0 +\beta_1 \bar x+\bar\epsilon$
$\displaystyle E(Y_i)=\beta_0 +\beta_1 x_i+0$
so $\displaystyle E(\hat\beta_1) ={\sum_{i=1}^n(x_i-\bar x)E(Y_i)\over \sum_{i=1}^n(x_i-\bar x)^2}$
$\displaystyle ={\sum_{i=1}^n(x_i-\bar x)(\beta_0 +\beta_1 x_i)\over \sum_{i=1}^n(x_i-\bar x)^2} =\beta_0 {\sum_{i=1}^n(x_i-\bar x)\over \sum_{i=1}^n(x_i-\bar x)^2}+\beta_1{\sum_{i=1}^n(x_i-\bar x)^2\over \sum_{i=1}^n(x_i-\bar x)^2}=\beta_1$
Next, $\displaystyle E(\bar Y)=\beta_0 +\beta_1 \bar x+0$
SO, using the fact that $\displaystyle \hat\beta_1$ is unbiased for $\displaystyle \beta_1$
we have $\displaystyle E(\hat\beta_0)=E(\bar Y) -E(\hat\beta_1)\bar x= \beta_0 +\beta_1 \bar x-\beta_1 \bar x= \beta_0 $.
Corrections here...
the model is $\displaystyle y=\beta_0+\beta_1 x+\epsilon$
and the least squares fit (line through the data) is $\displaystyle \hat y=\hat\beta_0+\hat\beta_1 x$
The $\displaystyle \hat\beta$'s are the estimators (random variables) of the unknown parameters (constants) $\displaystyle \beta$'s.