# Linear Regression, Residual Sum of Squares

• Nov 25th 2011, 12:16 PM
garunas
Linear Regression, Residual Sum of Squares
Suppose the columns of a rank 4 design matrix
$X = [X_{1}, X_{2}, X_{3}, X_{4}]$ come in three groups
Group 1: $\{X_{1}\}$,
Group 2: $\{X_{2}, X_{3}\}$ and
Group 3: $\{X_{4}\}$ so that $X^T_{i}X_{j} = 0$ if i and j are indicies from different groups.
Next consider 4 models, with design matricies:
Model 1: $X_{(1)} = [X_{1}]$
Model 2: $X_{(2)} = [X_{1}, X_{2}, X_{3}]$
Model 3: $X_{(3)} = [X_{1}, X_{2}, X_{3}, X_{4}]$
Model 4: $X_{(4)} = [X_{1}, X_{4}]$

Now then the question asks to show that $SS_{2} - SS_{3} = SS_{1} - SS_{4}$ where $SS_{i}$ is the residual sum of squares (SS error) for the ith model.

$SS_{error} = Y^T Y - \hat\theta X^T Y$ is the only formula for SS error that I know, which I believe does not apply in this situation. So it means that I'm absolutely stuck! I think I'm looking at this the wrong way to be honest. Where should I look into? Any hints?