## Linear Regression, Residual Sum of Squares

Suppose the columns of a rank 4 design matrix
$\displaystyle X = [X_{1}, X_{2}, X_{3}, X_{4}]$ come in three groups
Group 1: $\displaystyle \{X_{1}\}$,
Group 2: $\displaystyle \{X_{2}, X_{3}\}$ and
Group 3: $\displaystyle \{X_{4}\}$ so that $\displaystyle X^T_{i}X_{j} = 0$ if i and j are indicies from different groups.
Next consider 4 models, with design matricies:
Model 1: $\displaystyle X_{(1)} = [X_{1}]$
Model 2: $\displaystyle X_{(2)} = [X_{1}, X_{2}, X_{3}]$
Model 3: $\displaystyle X_{(3)} = [X_{1}, X_{2}, X_{3}, X_{4}]$
Model 4: $\displaystyle X_{(4)} = [X_{1}, X_{4}]$

Now then the question asks to show that $\displaystyle SS_{2} - SS_{3} = SS_{1} - SS_{4}$ where $\displaystyle SS_{i}$ is the residual sum of squares (SS error) for the ith model.

$\displaystyle SS_{error} = Y^T Y - \hat\theta X^T Y$ is the only formula for SS error that I know, which I believe does not apply in this situation. So it means that I'm absolutely stuck! I think I'm looking at this the wrong way to be honest. Where should I look into? Any hints?