Results 1 to 8 of 8

Math Help - Orthogonal Projection and Minimization

  1. #1
    Junior Member
    Joined
    Apr 2010
    Posts
    50

    Orthogonal Projection and Minimization

    This is a relatively quick question.

    Suppose we have a vector space V with v\in{V} and subspace U with u\in{U} and we want to find some vector u such that ||v-u|| is at small as possible which I know implies that u is a good approximation of v.

    It is shown that
    ||v-P_U(v)||\leq||v-u||

    where P_U(v) is the orthogonal projection for V=U\oplus{U^{\perp}
    and I understand how to prove this inequality. Supposedly finding P_U(v) gives me a good approximation of v (which evidently it actually does), but I don't quite understand why.

    I have some speculations on why however. I believe the inequality implies that unless P_U(v)=u, then P_U(v) will always be a better approximation than u.

    However, I don't really have a good understanding on why the orthogonal projection is truly a good approximation of v.

    Thank you.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    5
    Awards
    2
    Think of a simple example: V is the xy plane, and U is the subspace consisting of all points on the line y=x. You can show that it is a subspace, because it contains the origin, and is closed under scalar multiplication and vector addition.

    Now then. If your vector v\in U, then obviously the best approximation is v itself, because then 0\le\|v-u\|=0. If, on the other hand, v\not\in U, then what is the vector in U closest to v? Well, it would be the point on the line that is closest to v. And that point is the point you would get by dropping the perpendicular from v to the line y=x. That also turns out to be the orthogonal projection onto the subspace U.

    The essential idea here is that the straight line (in this case, the perpendicular), is the shortest path between two points, at least in the Euclidean norm. And that "perpendicular" is equivalent to the orthogonal projection onto the subspace.

    Make sense? Does this help?
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Junior Member
    Joined
    Apr 2010
    Posts
    50
    Quote Originally Posted by Ackbeet View Post
    Think of a simple example: V is the xy plane, and U is the subspace consisting of all points on the line y=x. You can show that it is a subspace, because it contains the origin, and is closed under scalar multiplication and vector addition.

    Now then. If your vector v\in U, then obviously the best approximation is v itself, because then 0\le\|v-u\|=0. If, on the other hand, v\not\in U, then what is the vector in U closest to v? Well, it would be the point on the line that is closest to v. And that point is the point you would get by dropping the perpendicular from v to the line y=x. That also turns out to be the orthogonal projection onto the subspace U.

    The essential idea here is that the straight line (in this case, the perpendicular), is the shortest path between two points, at least in the Euclidean norm. And that "perpendicular" is equivalent to the orthogonal projection onto the subspace.

    Make sense? Does this help?
    Thanks for the reply. I see how the projection gives a good approximation in your example with the point and line. I was also wondering how the inequality with the norms show that the orthogonal projection is a good approximation. I understand the intuition behind it, but what does the inequality show?

    Thank you
    Follow Math Help Forum on Facebook and Google+

  4. #4
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    5
    Awards
    2
    Are you talking about in the case when v\in U, and I got the expression

    0\le\|u-v\|=0?

    In that case, the inequality shows that this is the best approximation possible, because the expression \|u-v\| is a measure of how good the approximation is, and it is always non-negative, by the axioms of a norm.

    Make sense?
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Junior Member
    Joined
    Apr 2010
    Posts
    50
    Quote Originally Posted by Ackbeet View Post
    Are you talking about in the case when v\in U, and I got the expression

    0\le\|u-v\|=0?

    In that case, the inequality shows that this is the best approximation possible, because the expression \|u-v\| is a measure of how good the approximation is, and it is always non-negative, by the axioms of a norm.

    Make sense?
    .

    Sorry, I meant this inequality:
    ||v-P_U(v)||\leq||v-u||

    The expression 0\le\|u-v\|=0 does make sense though.
    Follow Math Help Forum on Facebook and Google+

  6. #6
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    5
    Awards
    2
    Ah. That inequality shows that the orthogonal projection is the best approximation, because the distance between the vector v and the vector P_{U}(v) is smaller than the distance \|v-u\| for any other vector u\in U. (Note that P_{U}(v)\in U.)

    A normed vector space is also a metric space, right? You have the induced metric

    d(u,v)=\|u-v\|,

    which obeys all the requirements for a metric. So the expression \|u-v\| is the distance between the two vectors u and v.

    Does that make sense?
    Follow Math Help Forum on Facebook and Google+

  7. #7
    Junior Member
    Joined
    Apr 2010
    Posts
    50
    Thanks!

    Yes, this makes perfect sense now. Thank you.
    Follow Math Help Forum on Facebook and Google+

  8. #8
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    5
    Awards
    2
    You're welcome! Have a good one.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Orthogonal projection
    Posted in the Advanced Algebra Forum
    Replies: 1
    Last Post: August 9th 2010, 07:40 PM
  2. Orthogonal Projection
    Posted in the Advanced Algebra Forum
    Replies: 1
    Last Post: February 17th 2010, 07:06 AM
  3. Replies: 1
    Last Post: December 2nd 2009, 11:14 PM
  4. orthogonal projection
    Posted in the Advanced Algebra Forum
    Replies: 1
    Last Post: December 2nd 2009, 01:32 PM
  5. orthogonal projection
    Posted in the Advanced Algebra Forum
    Replies: 1
    Last Post: September 6th 2008, 08:00 PM

Search Tags


/mathhelpforum @mathhelpforum