Least Squares Approximation Theorem:
Let be continuous on , and let be a finite-dimensional subspace of . The least square approximating function of with respect to is given by
To show that is the least squares approximating function of , prove that the inequality
is true for any vector in . By writing as
you can see that is orthogonal to each , which in turn implies that it is orthogonal to each vector in . In particular, is orthogonal to
My question has two parts.
is enough for this proof? There might be other vectors that make the least square approximation smaller. Why choose particular orthonormal basis?
What is the reason behind the implicit statement is in the subspace ?
What reasoning make ? Is there any theorem I can't remember now for this question?