Least Squares Approximation Theorem:

Let be continuous on , and let be a finite-dimensional subspace of . The least square approximating function of with respect to is given by

Proof:

To show that is the least squares approximating function of , prove that the inequality

is true for any vector in . By writing as

you can see that is orthogonal to each , which in turn implies that it is orthogonal to each vector in . In particular, is orthogonal to

-------------Proof Ends------------

My question has two parts.

First question:

Why proving

is enough for this proof? There might be other vectors that make the least square approximation smaller. Why choose particular orthonormal basis?

Second question:

What is the reason behind the implicit statement is in the subspace ?

What reasoning make ? Is there any theorem I can't remember now for this question?