# Math Help - Discrete Least Squares

1. ## Discrete Least Squares

Ok so we were given this really long crazy function to be the model of a an arbitrary data set defined by vector f (function values) and vector x (data points) for $x_i > 0 for All i = 0,1,2,.....,n$

And were asked to find the best fit in the least squares sense for this model -- Find the normal equation?

$h(x)=a + bx + xe^a^r^c^t^a^n^(^x^)+ dcos(sin(T_1_4(x)))$ where $T_1_4(x)$ is the 14th degree Chebyshev polynomial

So I was wondering how do u find the the normal equation for something as crazy as this... or is this more of a theoretical question

2. Ok how would we find the 14th degree Chebyshev polynomial? lets start with that

3. ## Discrete Least Squares, reply.

Originally Posted by ductiletoaster
OK, so we were given this really long crazy function ...
to be the model of a an arbitrary data set defined by vector f (function values) and vector x (data points) for $x_i > 0 \text{ for All }i = 0,1,2,.....,n$

And we're asked to find the best fit in the least squares sense for this model -- Find the normal equation?

$h(x)=a + bx + xe^{\arctan (x)}+ d(\cos(\sin(T_{14}(x)))),$ where $T_{14}(x)$ is the 14th degree Chebyshev polynomial (of the first kind).

Originally Posted by ductiletoaster
So I was wondering how do you find the the normal equation for something as crazy as this... or is this more of a theoretical question
You certainly can show HOW to get the normal matrix, and normal equation for this, but you have no specific values for the $\displaystyle x_i$ and no measured dependent values $\displaystyle y_i,$ so you certainly can't give a numerical answer.

I'm pretty sure that $\displaystyle h(x)$ should have a coefficient, $\displaystyle c$ in it.

I suspect that you typed $\displaystyle x$ instead of $\displaystyle c$ in the third term, which would make

$\displaystyle h(x)=a + b\,x + c\,e^{\arctan (x)}+ d(\cos(\sin(T_{14}(x))))$

On the other hand, if you merely omitted the $\displaystyle c$, that would make

$\displaystyle h(x)=a + b\,x + c\,xe^{\arctan (x)}+ d(\cos(\sin(T_{14}(x))))$

As for Chebyshev polynomials, Check out "Chebyshev polynomials" in Wikipedia and/or "Chebyshev polynomial" at WolframAlpha. More specifically, check out "ChebyshevT[14, x]" at WolframAlpha.

Work on that. I'll write an additional response as time permits.

4. yes i meant to type a c INSTEAD of the x... And thank you for your help.

The question that was posed to use was find the best fit for this model in the least squares sense. "Find the normal equation". I looked through our lecture notes for hours but I just couldn't figure it out.

5. Originally Posted by SammyS
... to be the model of a an arbitrary data set defined by vector f (function values) and vector x (data points) for $x_i > 0 \text{ for All }i = 0,1,2,.....,n$

And we're asked to find the best fit in the least squares sense for this model -- Find the normal equation?

$h(x)=a + b\,x + c\,e^{\arctan (x)}+ d\,(\cos(\sin(T_{14}(x)))),$ (corrected) where $T_{14}(x)$ is the 14th degree Chebyshev polynomial (of the first kind).

You certainly can show HOW to get the normal matrix, and normal equation for this, but you have no specific values for the $\displaystyle x_i$ and no observed dependent values $\displaystyle y_i,$ so you certainly can't give a numerical answer.

I'm going to rename the coefficients to make setting up matricies more convenient, calling them $\displaystyle b_1,\ b_2,\ b_3,\ b_4$ for $\displaystyle a,\ b,\ c,\ d$ respectively. We then have:

$\displaystyle h(x)=b_1 + b_2\,x + b_3\,e^{\arctan (x)}+ b_4\cos(\sin( T_{14}(x)))$

Define the following matrix and vectors.
Vector f:

$\displaystyle \text{f}\ =\ \left[ \begin{array}{c}
h(x_1)\\
h(x_2)\\
\vdots\\
h(x_n) \end{array} \right]$

Vector b:

$\displaystyle \text{b}\ =\ \left[ \begin{array}{c}
b_1\\
b_2\\
b_3\\
b_4 \end{array} \right]$

Matrix X, an $n\times 4$ matrix, such that: $\displaystyle \text{x}_{i,j}={{\partial\ \,}\over{\partial b_j}}\,h(x_i),$ for example: $\displaystyle \text{x}_{5,3}=e^{\arctan (x_5)}.$
The transpose of X times X is often called the Normal matrix. It is a 4x4 matrix.

Notice that $\text{f}=\text{X}\times \text{b}$

We also need a vector, y, of observed values, each $\displaystyle y_i$ corresponding to $\displaystyle x_i .$

$\displaystyle \text{y}\ =\ \left[ \begin{array}{c}
y_1\\
y_2\\
\vdots\\
y_n \end{array} \right]$

The normal equations resulting from this are: $\displaystyle \text{X}^T\,\text{X}\,\text{b}=\text{X}^T\text{y} .$

Solve this for b to get the least squares result.