Thread: Least squares approximation

First, some definitions:
The least squares approximation of a function f is a function $\displaystyle \phi\epsilon\theta_n$ such as:
$\displaystyle ||f-\phi||\le||f-\phi_n||$, for every $\displaystyle \phi_n\epsilon\theta_n$.
Also: $\displaystyle ||u||=(\int_R|u(t)|^2d\lambda t)^{1/2}$

Now the problem:
Let's consider $\displaystyle \theta_n$ a class of approximations with the following properties:
- all functions $\displaystyle \phi \in \theta_n$ are defined on a symmetric interval $\displaystyle [-a, a]$;
- if $\displaystyle \phi(t)\in\theta_n$, then $\displaystyle \phi(-t)\in\theta_n$;
Consider $\displaystyle d\lambda t=\omega(t)dt$, where $\displaystyle \omega$ is an even function.
Prove that if f is an even function on $\displaystyle [-a, a]$, its least squares approximation $\displaystyle \phi\epsilon\theta_n$ is even.

My attempt of a solution started by trying to prove that the approximation is a linear combination of even functions. So we consider an orthogonal basis $\displaystyle \lbrace\pi_j\rbrace_{j=1}^n$ for the given class of approximations. In this case, the least squares approximation is:
$\displaystyle \phi=\sum_{j=1}^nc_j*\pi_j$, where: $\displaystyle c_j=(\pi_j,f)/(\pi_j,\pi_j)$
(u,v) is the dot product of functions u and v. More precisely:
$\displaystyle (u,v)=\int_{-a}^a u(t)v(t)\omega(t)dt$
Using this formula and the hypothesis, it can be proven that if $\displaystyle \pi_j$ is an odd function, then $\displaystyle c_j=0$, because $\displaystyle (\pi_j,f)$ is the integral of an odd function on a symmetric interval.
This way, we eliminate from the approximation $\displaystyle \phi$ all the odd basis functions. So now $\displaystyle \phi$ is a linear combinations of even functions and functions that are neither even, nor odd. The even functions are good, because a linear combination of even functions is also an even function. But what happens when the basis function $\displaystyle \pi_j$ is neither even, nor odd? I tried decomposing $\displaystyle \pi_j$ in combinations of even and odd functions, but this approach does not seem to lead anywhere...

Any help would be appreciated.

Originally Posted by Anomander

First, some definitions:
The least squares approximation of a function f is a function $\displaystyle \phi\epsilon\theta_n$ such as:
$\displaystyle ||f-\phi||\le||f-\phi_n||$, for every $\displaystyle \phi_n\epsilon\theta_n$.
Also: $\displaystyle ||u||=(\int_R|u(t)|^2d\lambda t)^{1/2}$

Now the problem:
Let's consider $\displaystyle \theta_n$ a class of approximations with the following properties:
- all functions $\displaystyle \phi \in \theta_n$ are defined on a symmetric interval $\displaystyle [-a, a]$;
- if $\displaystyle \phi(t)\in\theta_n$, then $\displaystyle \phi(-t)\in\theta_n$;
Consider $\displaystyle d\lambda t=\omega(t)dt$, where $\displaystyle \omega$ is an even function.
Prove that if f is an even function on $\displaystyle [-a, a]$, its least squares approximation $\displaystyle \phi\epsilon\theta_n$ is even.

My attempt of a solution started by trying to prove that the approximation is a linear combination of even functions. So we consider an orthogonal basis $\displaystyle \lbrace\pi_j\rbrace_{j=1}^n$ for the given class of approximations. In this case, the least squares approximation is:
$\displaystyle \phi=\sum_{j=1}^nc_j*\pi_j$, where: $\displaystyle c_j=(\pi_j,f)/(\pi_j,\pi_j)$
(u,v) is the dot product of functions u and v. More precisely:
$\displaystyle (u,v)=\int_{-a}^a u(t)v(t)\omega(t)dt$
Using this formula and the hypothesis, it can be proven that if $\displaystyle \pi_j$ is an odd function, then $\displaystyle c_j=0$, because $\displaystyle (\pi_j,f)$ is the integral of an odd function on a symmetric interval.
This way, we eliminate from the approximation $\displaystyle \phi$ all the odd basis functions. So now $\displaystyle \phi$ is a linear combinations of even functions and functions that are neither even, nor odd. The even functions are good, because a linear combination of even function is also a even function. But what happens when the basis function $\displaystyle \pi_j$ is neither even, nor odd? I tried decomposing $\displaystyle \pi_j$ in combinations of even and odd functions, but this approach does not seem to lead anywhere...

Any help would be appreciated.

Best not to think in terms of basis functions, I think.

Suppose that $\displaystyle \phi$ is a least-squares approximation to $\displaystyle f$. Define $\displaystyle \psi$ by $\displaystyle \psi(t) = \phi(-t)$. Then $\displaystyle \psi\in\theta_n$. Use the fact that $\displaystyle f$ is even to show that $\displaystyle \|f-\psi\| = \|f-\phi\|$. Assuming you know that a least-squares approximation is unique, it follows that $\displaystyle \psi = \phi$, and therefore $\displaystyle \phi$ is even.

Okay, I'll try this. So, keeping your notations, we must prove that:

$\displaystyle ||f - \psi|| = ||f - \phi|| \Leftrightarrow ||f - \psi||^2 = ||f - \phi||^2$ (based on the fact that the norm is a positive integer).
$\displaystyle ||f - \psi||^2 = \int_{-a}^a |f(t) - \psi(t)|^2 * \omega(t)dt$

I'm thinking of making the substition $\displaystyle v \leftarrow -t$. With that we obtain:

$\displaystyle \int_{-a}^a |f(t) - \psi(t)|^2 * \omega(t)dt = -\int_a^{-a} |f(v) - \psi(v)|^2 * \omega(v)dv$
$\displaystyle = \int_{-a}^a |f(v) - \psi(v)|^2 * \omega(v)dv$

Now I'm tempted to replace $\displaystyle v$ with $\displaystyle -t$ and obtain the final result:

$\displaystyle \int_{-a}^a |f(v) - \psi(v)|^2 * \omega(v)dv = \int_{-a}^a |f(-t) - \psi(-t)|^2 * \omega(-t)d(-t) $
$\displaystyle = \int_{-a}^a |f(t) - \psi(-t)|^2 * \omega(t)dt$(because $\displaystyle f$ and $\displaystyle \omega$ are even)

But integrals aren't my strong point, so I'm wondering if the last step is ok. If it is, the problem is solved. But if it isn't... How else could I use the fact that $\displaystyle f$ is even?

Yes that's basically correct. The only other thing that you might want to clarify when making the substitution v = –t in the integral is that dv becomes –dt, and the limits of integration become $\displaystyle \textstyle\int_a^{-a}$ rather than $\displaystyle \textstyle\int_{-a}^a$. But when you switch the limits of integration, that has the effect of changing the sign of the integral. Those two changes of sign (one in in –dt and the other in the limits of integration) cancel each other out, so the t-integral ends up looking identical to the v-integral.

Wow. I didn't expect it to be so... straightforward. Thank you a lot, Opalg.