This is a relatively quick question.
Suppose we have a vector space with and subspace U with and we want to find some vector u such that is at small as possible which I know implies that u is a good approximation of v.
It is shown that
where is the orthogonal projection for
and I understand how to prove this inequality. Supposedly finding gives me a good approximation of v (which evidently it actually does), but I don't quite understand why.
I have some speculations on why however. I believe the inequality implies that unless , then will always be a better approximation than u.
However, I don't really have a good understanding on why the orthogonal projection is truly a good approximation of v.