# Thread: Help with nonlinear regression / curve fitting

1. ## Help with nonlinear regression / curve fitting

Hi everyone,

I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

Thanks

Steve

2. Originally Posted by khayman
Hi everyone,

I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

Thanks

Steve
Give my the points. I have a program on my Turing machine which can do regressions.

3. Originally Posted by ThePerfectHacker
Give my the points. I have a program on my Turing machine which can do regressions.
what program is that?

4. Originally Posted by Jhevon
what program is that?
The same one you have. Heir.

5. Originally Posted by ThePerfectHacker
The same one you have. Heir.
wow. this program can do a lot more than i thought it could. i should sit down one day and just explore its features.

6. I do not know if you have ever taken Statistics. But perhaps you heard of "method of least squares". (It is explained a little bit in my Calculus mini-book).

To test you can do this.
1)FUNCTION
2)INSERT POINT SERIES
3)For example (1,1) (2,5) (3,10) (4,14)
5)FUNCTION ---> INSERT TREDLINE
6)Select LINEAR.

You get the equation of "the line of best fit" and you also get the R^2 = .9979.... That is the number which measures how good your approximate is. (Called "coefficient of regession")

For example if instead you choose EXPONENTIAL it is R^2=.6958 which is much much more worse.

The best is when R^2=1 and worst is when R^2=0.

7. Originally Posted by khayman
Hi everyone,

I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

Thanks

Steve
Originally Posted by ThePerfectHacker
Give my the points. I have a program on my Turing machine which can do regressions.
This doesn't look like a standard regression problem.

From the Wiki page, the curve is generated by these equations

but the picture in the (x,y) plane looks something something like this

,

and the values for t are not observed.

This does not appear to be a regression problem (linear or nonlinear) because that is for fitting a function to a set of points. But that picture in the (x,y) plane is not the graph of a well-defined function: there is more than one y value for each x value. (The regression can handle any set of points, but the function you're trying to fit must be a well-defined function.)

At this point, I don't see any good solution.

8. Originally Posted by JakeD
This doesn't look like a standard regression problem.

From the Wiki page, the curve is generated by these equations

but the picture in the (x,y) plane looks something something like this

,

and the values for t are not observed.

This does not appear to be a regression problem (linear or nonlinear) because that is for fitting a function to a set of points. But that picture in the (x,y) plane is not the graph of a well-defined function: there is more than one y value for each x value. (The regression can handle any set of points, but the function you're trying to fit must be a well-defined function.)

At this point, I don't see any good solution.
Given a set of points $\displaystyle (x_i,y_i),\ i=1,\ ..,\ N$ which we hope lie
on a curve:

$\displaystyle x=A\sin(at+\delta),\ y=B\cos(bt)$

Now if we tabulate:

$\displaystyle u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$

for $\displaystyle k=1,\ ..\ ,\ 2\pi / (\min(a,b)\delta t)$

for $\displaystyle \delta t$ suitably small. We can find $\displaystyle A,\ B,\ a,\ b,\ \delta$ which minimises the sum of the
squares of minimum distances of $\displaystyle (x_i,y_i)$ from the $\displaystyle (u_k,\ v_k)'s$.

The Excell solver sould be up to this job.

(note the solution will not necessarily be unique but that won't bother
Excell )

RonL

9. Originally Posted by CaptainBlack
Given a set of points $\displaystyle (x_i,y_i),\ i=1,\ ..,\ N$ which we hope lie
on a curve:

$\displaystyle x=A\sin(at+\delta),\ y=B\cos(bt)$

Now if we tabulate:

$\displaystyle u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$

for $\displaystyle k=1,\ ..\ ,\ 2\pi / (\min(a,b)\delta t)$

for $\displaystyle \delta t$ suitably small. We can find $\displaystyle A,\ B,\ a,\ b,\ \delta$ which minimises the sum of the
squares of minimum distances of $\displaystyle (x_i,y_i)$ from the $\displaystyle (u_k,\ v_k)'s$.

The Excell solver sould be up to this job.

(note the solution will not necessarily be unique but that won't bother
Excell )

RonL
I think this is a clever solution, however I don't think Excel's solver can handle it. But let me make sure I understand the algorithm.

First, get coordinates $\displaystyle (x_i,y_i),\ i = 1,\ldots,N$ from the picture. Fix a value for $\displaystyle \delta t.$

Next, define a function $\displaystyle Dist(A,\ B,\ a,\ b,\ \delta)$ with the following 4 steps.
1. Set $\displaystyle M = 2\pi / (\min(a,b)\delta t).$
2. Tabulate $\displaystyle u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$ for $\displaystyle k=1,\ldots, M.$
3. For each $\displaystyle i = 1,\ldots,N$, find a point $\displaystyle (u_j,v_j)$ out of the $\displaystyle M\ (u_k,\ v_k)'s$ which has the minimum distance from $\displaystyle (x_i,y_i)$ to $\displaystyle (u_j,v_j)$.
4. Return the sum of the squares of the distances in step 3 as the value of $\displaystyle Dist.$

Finally, minimize $\displaystyle Dist(A,\ B,\ a,\ b,\ \delta).$

I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives. But the $\displaystyle Dist$ function, although continuous, will not be differentiable. Another problem with using Excel is that programming the $\displaystyle Dist$ function in a spreadsheet looks difficult.

However with a good programming language and a minimization routine that can handle a non-differentiable function, this could work. Good job!

10. Originally Posted by JakeD
I think this is a clever solution, however I don't think Excel's solver can handle it. But let me make sure I understand the algorithm.

First, get coordinates $\displaystyle (x_i,y_i),\ i = 1,\ldots,N$ from the picture. Fix a value for $\displaystyle \delta t.$

Next, define a function $\displaystyle Dist(A,\ B,\ a,\ b,\ \delta)$ with the following 4 steps.
1. Set $\displaystyle M = 2\pi / (\min(a,b)\delta t).$
2. Tabulate $\displaystyle u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$ for $\displaystyle k=1,\ldots, M.$
3. For each $\displaystyle i = 1,\ldots,N$, find a point $\displaystyle (u_j,v_j)$ out of the $\displaystyle M\ (u_k,\ v_k)'s$ which has the minimum distance from $\displaystyle (x_i,y_i)$ to $\displaystyle (u_j,v_j)$.
4. Return the sum of the squares of the distances in step 3 as the value of $\displaystyle Dist.$
Finally, minimize $\displaystyle Dist(A,\ B,\ a,\ b,\ \delta).$

I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives. But the $\displaystyle Dist$ function, although continuous, will not be differentiable. Another problem with using Excel is that programming the $\displaystyle Dist$ function in a spreadsheet looks difficult.

However with a good programming language and a minimization routine that can handle a non-differentiable function, this could work. Good job!
I don't believe that Excell's solver assumes the objective is differentiable,
(it's not a single algorithm but a suite IIRC) but it does help. Even if it does the
optimisation algorithm in Euler uses the Nelder-Mead simplex algorithm that
makes no such assumption.

Also Simulated Annealing and similar methods will do the job.

We will also need a penalty function added onto the objective to
stop the frequencies going too high.

RonL

11. Originally Posted by JakeD
I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives.
Originally Posted by CaptainBlack
I don't believe that Excell's solver assumes the objective is differentiable, (it's not a single algorithm but a suite IIRC) but it does help.
Actually, solver does assume differentiability and uses numerical approximations to the derivatives. See XL2000: Solver Uses Generalized Reduced Gradient Algorithm where it says "Derivatives (and gradients) play a crucial role in iterative methods in Microsoft Excel Solver" and describes the methods.

12. Originally Posted by JakeD
Actually, solver does assume differentiability and uses numerical approximations to the derivatives. See XL2000: Solver Uses Generalized Reduced Gradient Algorithm where it says "Derivatives (and gradients) play a crucial role in iterative methods in Microsoft Excel Solver" and describes the methods.
Ok.. but I would still like to see some data and try a few things either
Excel or Euler, and if deperate Matlab (sp.. yuck, just off to wash my mouth out).

RonL

13. Originally Posted by CaptainBlack
Ok.. but I would still like to see some data and try a few things either
Excel or Euler, and if deperate Matlab (sp.. yuck, just off to wash my mouth out).

RonL
lol, you dislike MatLab that much, Captain?

14. Originally Posted by Jhevon
lol, you dislike MatLab that much, Captain?
It provides prepackaged solutions, so you don't have to understand what
is going on or how your problem is being solved.

Also it is over priced, the three installations that I have cost my employer
~£10000 each plus annual maintenance charges (but its their own fault its
their policy that says we don't use free-ware)

(Now it does so happen that I don't use Matlab much because we are
allowed to use tools we build ourselves, and I have been building/modifying
Euler on our machines now for a decade - its like matlab, but leaner).

RonL

15. Originally Posted by CaptainBlack
It provides prepackaged solutions, so you don't have to understand what
is going on or how your problem is being solved.
i see, but aren't all similar utilities like that? Is Euler any different in that respect?

Also it is over priced, the three installations that I have cost my employer
~£10000 each plus annual maintenance charges (but its their own fault its
their policy that says we don't use free-ware)
I remember you mentioning this before

(Now it does so happen that I don't use Matlab much because we are
allowed to use tools we build ourselves, and I have been building/modifying
Euler on our machines now for a decade - its like matlab, but leaner).
is Euler available to the general public? if so, can you provide a link to where i can find it (obviously if i search for Euler i will get webpages about the mathematician Euler)

Page 1 of 2 12 Last