Results 1 to 6 of 6

Thread: Least square approximation of a function and being in subspace W

  1. #1
    Senior Member x3bnm's Avatar
    Joined
    Nov 2009
    Posts
    308
    Thanks
    16

    Least square approximation of a function and being in subspace W

    Least Squares Approximation Theorem:


    Let $\displaystyle f$ be continuous on $\displaystyle [a,b]$, and let $\displaystyle W$ be a finite-dimensional subspace of $\displaystyle C[a,b]$. The least square approximating function of $\displaystyle f$ with respect to $\displaystyle W$ is given by

    $\displaystyle g = \langle f,\mathbf{w_1} \rangle \mathbf{w_1} + \langle f,\mathbf{w_2} \rangle \mathbf{w_2} + ... + \langle f,\mathbf{w_n} \rangle \mathbf{w_n}, $

    $\displaystyle \text{where } B = \{\mathbf{w_1}, \mathbf{w_2}, ..., \mathbf{w_n}\} \text{ is an orthonormal basis for } W.$


    Proof:
    To show that $\displaystyle g$ is the least squares approximating function of $\displaystyle f$, prove that the inequality

    $\displaystyle \parallel f-g \parallel \,\, \leq \,\, \parallel f-\mathbf{w} \parallel $
    is true for any vector $\displaystyle \mathbf{w}$ in $\displaystyle W$. By writing $\displaystyle f-g$ as


    $\displaystyle f - g = f - \langle f,\mathbf{w_1} \rangle \mathbf{w_1} + \langle f,\mathbf{w_2} \rangle \mathbf{w_2} + ... + \langle f,\mathbf{w_n} \rangle \mathbf{w_n} $


    you can see that $\displaystyle f-g$ is orthogonal to each $\displaystyle \mathbf{w_i}$, which in turn implies that it is orthogonal to each vector in $\displaystyle W$. In particular, $\displaystyle f-g$ is orthogonal to $\displaystyle g-\mathbf{w}$


    $\displaystyle \text{...the rest of the proof continues} $

    -------------Proof Ends------------



    My question has two parts.

    First question:

    Why proving

    $\displaystyle \parallel f-g \parallel \,\, \leq \,\, \parallel f-\mathbf{w} \parallel $

    is enough for this proof? There might be other vectors that make the least square approximation smaller. Why choose particular orthonormal basis?

    Second question:

    What is the reason behind the implicit statement $\displaystyle g-\mathbf{w}$ is in the subspace $\displaystyle W$ ?

    What reasoning make $\displaystyle g-\mathbf{w} \in W $? Is there any theorem I can't remember now for this question?
    Follow Math Help Forum on Facebook and Google+

  2. #2
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    7
    Awards
    2

    Re: Least square approximation of a function and being in subspace W

    1. Because w is left arbitrary. It's true for ALL vectors w in the subspace W.

    2. Because g is in W, and w is in W. Since W is a subspace, linear combinations of vectors in the subspace remain in the subspace.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Senior Member x3bnm's Avatar
    Joined
    Nov 2009
    Posts
    308
    Thanks
    16

    Re: Least square approximation of a function and being in subspace W

    Quote Originally Posted by Ackbeet View Post
    1. Because w is left arbitrary. It's true for ALL vectors w in the subspace W.

    2. Because g is in W, and w is in W. Since W is a subspace, linear combinations of vectors in the subspace remain in the subspace.
    Thank you for your answers. I get the answer no. 2. But having difficulty in answer no. 1.

    I'll tell you my understanding.

    $\displaystyle \mathbf{w_i}$ is not a generalized vector. $\displaystyle \mathbf{w_i}$ is a orthonormal basis that are normalized and orthogonal to each other. Most vectors in subspace $\displaystyle W$ are not normalized and orthogonal. So why a particular class of vector speaks for all generalized vectors in subspace $\displaystyle W$? I don't understand that.

    Can you kindly tell me what you meant by "left arbitrary"?
    Follow Math Help Forum on Facebook and Google+

  4. #4
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    7
    Awards
    2

    Re: Least square approximation of a function and being in subspace W

    Hmm. Well, I could be wrong, but it looks to me like you might be confusing the arbitrary $\displaystyle \mathbf{w}$ vector chosen at the beginning of the proof, and the basis vectors $\displaystyle \mathbf{w}_{i}$ of the subspace $\displaystyle W.$ They are not the same thing. It might have been clearer in the writing of the proof to pick an arbitrary vector $\displaystyle \mathbf{v}\in W,$ instead of $\displaystyle \mathbf{w}\in W.$ Does that clear things up a bit?
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Senior Member x3bnm's Avatar
    Joined
    Nov 2009
    Posts
    308
    Thanks
    16

    Re: Least square approximation of a function and being in subspace W

    Quote Originally Posted by Ackbeet View Post
    Hmm. Well, I could be wrong, but it looks to me like you might be confusing the arbitrary $\displaystyle \mathbf{w}$ vector chosen at the beginning of the proof, and the basis vectors $\displaystyle \mathbf{w}_{i}$ of the subspace $\displaystyle W.$ They are not the same thing. It might have been clearer in the writing of the proof to pick an arbitrary vector $\displaystyle \mathbf{v}\in W,$ instead of $\displaystyle \mathbf{w}\in W.$ Does that clear things up a bit?
    Sorry Ackbeet. I'm extremely sorry. You're right. I mixed up $\displaystyle \mathbf{w}$ with $\displaystyle \mathbf{w_i}$. Now it's clear.

    In the proof it's clearly written "is true for any vector $\displaystyle \mathbf{w}$". I wish I were more attentive. Sorry. Again thanks.
    Follow Math Help Forum on Facebook and Google+

  6. #6
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    7
    Awards
    2

    Re: Least square approximation of a function and being in subspace W

    No apologies needed, man. The questions you asked are what this forum is for.

    Have a good one!
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Replies: 2
    Last Post: Jan 3rd 2012, 10:53 AM
  2. Replies: 2
    Last Post: Sep 2nd 2010, 10:28 AM
  3. Least square approximation parabola
    Posted in the Advanced Algebra Forum
    Replies: 1
    Last Post: Apr 25th 2010, 09:20 AM
  4. Replies: 0
    Last Post: Apr 1st 2009, 04:22 AM
  5. Show subspace Ua = span {a} is square on hyperplane H
    Posted in the Advanced Algebra Forum
    Replies: 0
    Last Post: Sep 22nd 2007, 06:53 AM

Search tags for this page

Click on a term to search for related topics.

/mathhelpforum @mathhelpforum