Results 1 to 6 of 6

Math Help - Least square approximation of a function and being in subspace W

  1. #1
    Senior Member x3bnm's Avatar
    Joined
    Nov 2009
    Posts
    300
    Thanks
    16

    Least square approximation of a function and being in subspace W

    Least Squares Approximation Theorem:


    Let f be continuous on [a,b], and let W be a finite-dimensional subspace of C[a,b]. The least square approximating function of f with respect to W is given by

     g = \langle f,\mathbf{w_1} \rangle \mathbf{w_1} + \langle f,\mathbf{w_2} \rangle \mathbf{w_2} + ... + \langle f,\mathbf{w_n} \rangle \mathbf{w_n},

     \text{where } B = \{\mathbf{w_1}, \mathbf{w_2}, ..., \mathbf{w_n}\} \text{ is an orthonormal basis for } W.


    Proof:
    To show that g is the least squares approximating function of f, prove that the inequality

      \parallel f-g \parallel \,\, \leq \,\, \parallel f-\mathbf{w} \parallel
    is true for any vector \mathbf{w} in W. By writing f-g as


     f - g = f - \langle f,\mathbf{w_1} \rangle \mathbf{w_1} + \langle f,\mathbf{w_2} \rangle \mathbf{w_2} + ... + \langle f,\mathbf{w_n} \rangle \mathbf{w_n}


    you can see that f-g is orthogonal to each \mathbf{w_i}, which in turn implies that it is orthogonal to each vector in W. In particular, f-g is orthogonal to g-\mathbf{w}


      \text{...the rest of the proof continues}

    -------------Proof Ends------------



    My question has two parts.

    First question:

    Why proving

      \parallel f-g \parallel \,\, \leq \,\, \parallel f-\mathbf{w} \parallel

    is enough for this proof? There might be other vectors that make the least square approximation smaller. Why choose particular orthonormal basis?

    Second question:

    What is the reason behind the implicit statement g-\mathbf{w} is in the subspace W ?

    What reasoning make g-\mathbf{w} \in W ? Is there any theorem I can't remember now for this question?
    Follow Math Help Forum on Facebook and Google+

  2. #2
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2

    Re: Least square approximation of a function and being in subspace W

    1. Because w is left arbitrary. It's true for ALL vectors w in the subspace W.

    2. Because g is in W, and w is in W. Since W is a subspace, linear combinations of vectors in the subspace remain in the subspace.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Senior Member x3bnm's Avatar
    Joined
    Nov 2009
    Posts
    300
    Thanks
    16

    Re: Least square approximation of a function and being in subspace W

    Quote Originally Posted by Ackbeet View Post
    1. Because w is left arbitrary. It's true for ALL vectors w in the subspace W.

    2. Because g is in W, and w is in W. Since W is a subspace, linear combinations of vectors in the subspace remain in the subspace.
    Thank you for your answers. I get the answer no. 2. But having difficulty in answer no. 1.

    I'll tell you my understanding.

    \mathbf{w_i} is not a generalized vector. \mathbf{w_i} is a orthonormal basis that are normalized and orthogonal to each other. Most vectors in subspace W are not normalized and orthogonal. So why a particular class of vector speaks for all generalized vectors in subspace W? I don't understand that.

    Can you kindly tell me what you meant by "left arbitrary"?
    Follow Math Help Forum on Facebook and Google+

  4. #4
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2

    Re: Least square approximation of a function and being in subspace W

    Hmm. Well, I could be wrong, but it looks to me like you might be confusing the arbitrary \mathbf{w} vector chosen at the beginning of the proof, and the basis vectors \mathbf{w}_{i} of the subspace W. They are not the same thing. It might have been clearer in the writing of the proof to pick an arbitrary vector \mathbf{v}\in W, instead of \mathbf{w}\in W. Does that clear things up a bit?
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Senior Member x3bnm's Avatar
    Joined
    Nov 2009
    Posts
    300
    Thanks
    16

    Re: Least square approximation of a function and being in subspace W

    Quote Originally Posted by Ackbeet View Post
    Hmm. Well, I could be wrong, but it looks to me like you might be confusing the arbitrary \mathbf{w} vector chosen at the beginning of the proof, and the basis vectors \mathbf{w}_{i} of the subspace W. They are not the same thing. It might have been clearer in the writing of the proof to pick an arbitrary vector \mathbf{v}\in W, instead of \mathbf{w}\in W. Does that clear things up a bit?
    Sorry Ackbeet. I'm extremely sorry. You're right. I mixed up \mathbf{w} with \mathbf{w_i}. Now it's clear.

    In the proof it's clearly written "is true for any vector \mathbf{w}". I wish I were more attentive. Sorry. Again thanks.
    Follow Math Help Forum on Facebook and Google+

  6. #6
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2

    Re: Least square approximation of a function and being in subspace W

    No apologies needed, man. The questions you asked are what this forum is for.

    Have a good one!
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Replies: 2
    Last Post: January 3rd 2012, 10:53 AM
  2. Replies: 2
    Last Post: September 2nd 2010, 10:28 AM
  3. Least square approximation parabola
    Posted in the Advanced Algebra Forum
    Replies: 1
    Last Post: April 25th 2010, 09:20 AM
  4. Replies: 0
    Last Post: April 1st 2009, 04:22 AM
  5. Show subspace Ua = span {a} is square on hyperplane H
    Posted in the Advanced Algebra Forum
    Replies: 0
    Last Post: September 22nd 2007, 06:53 AM

/mathhelpforum @mathhelpforum