Help with nonlinear regression / curve fitting

Printable View

Show 40 post(s) from this thread on one page

June 2nd 2007, 05:06 PM
khayman

Help with nonlinear regression / curve fitting

Hi everyone,

I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

Thanks

Steve
June 2nd 2007, 07:10 PM
ThePerfectHacker

Quote:

Originally Posted by khayman

Hi everyone,

I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

Thanks

Steve

Give my the points. I have a program on my Turing machine which can do regressions.
June 2nd 2007, 07:15 PM
Jhevon

Quote:

Originally Posted by ThePerfectHacker

Give my the points. I have a program on my Turing machine which can do regressions.

what program is that?
June 2nd 2007, 07:27 PM
ThePerfectHacker

Quote:

Originally Posted by Jhevon

what program is that?

The same one you have. Heir.
June 2nd 2007, 07:35 PM
Jhevon

Quote:

Originally Posted by ThePerfectHacker

The same one you have. Heir.

wow. this program can do a lot more than i thought it could. i should sit down one day and just explore its features.
June 2nd 2007, 07:41 PM
ThePerfectHacker

I do not know if you have ever taken Statistics. But perhaps you heard of "method of least squares". (It is explained a little bit in my Calculus mini-book).

To test you can do this.
1)FUNCTION
2)INSERT POINT SERIES
3)For example (1,1) (2,5) (3,10) (4,14)
5)FUNCTION ---> INSERT TREDLINE
6)Select LINEAR.

You get the equation of "the line of best fit" and you also get the R^2 = .9979.... That is the number which measures how good your approximate is. (Called "coefficient of regession")

For example if instead you choose EXPONENTIAL it is R^2=.6958 which is much much more worse.

The best is when R^2=1 and worst is when R^2=0.
June 2nd 2007, 07:54 PM
JakeD

Quote:

Originally Posted by khayman

Hi everyone,

I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

Thanks

Steve

Quote:

Originally Posted by ThePerfectHacker

Give my the points. I have a program on my Turing machine which can do regressions.

This doesn't look like a standard regression problem.

From the Wiki page, the curve is generated by these equations

http://upload.wikimedia.org/math/e/4...e0cef4e09f.png

but the picture in the (x,y) plane looks something something like this

http://upload.wikimedia.org/wikipedi...e_3by2.svg.png,

and the values for t are not observed.

This does not appear to be a regression problem (linear or nonlinear) because that is for fitting a function to a set of points. But that picture in the (x,y) plane is not the graph of a well-defined function: there is more than one y value for each x value. (The regression can handle any set of points, but the function you're trying to fit must be a well-defined function.)

At this point, I don't see any good solution.
June 3rd 2007, 04:14 AM
CaptainBlack

Quote:

Originally Posted by JakeD

This doesn't look like a standard regression problem.

From the Wiki page, the curve is generated by these equations

http://upload.wikimedia.org/math/e/4...e0cef4e09f.png

but the picture in the (x,y) plane looks something something like this

http://upload.wikimedia.org/wikipedi...e_3by2.svg.png,

and the values for t are not observed.

This does not appear to be a regression problem (linear or nonlinear) because that is for fitting a function to a set of points. But that picture in the (x,y) plane is not the graph of a well-defined function: there is more than one y value for each x value. (The regression can handle any set of points, but the function you're trying to fit must be a well-defined function.)

At this point, I don't see any good solution.

Given a set of points $(x_i,y_i),\ i=1,\ ..,\ N$ which we hope lie
on a curve:

$x=A\sin(at+\delta),\ y=B\cos(bt)$

Now if we tabulate:

$u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$

for $k=1,\ ..\ ,\ 2\pi / (\min(a,b)\delta t)$

for $\delta t$ suitably small. We can find $A,\ B,\ a,\ b,\ \delta$ which minimises the sum of the
squares of minimum distances of $(x_i,y_i)$ from the $(u_k,\ v_k)'s$ .

The Excell solver sould be up to this job.

(note the solution will not necessarily be unique but that won't bother
Excell:eek: )

RonL
June 3rd 2007, 06:34 AM
JakeD
Quote:

Originally Posted by CaptainBlack

Given a set of points $(x_i,y_i),\ i=1,\ ..,\ N$ which we hope lie
on a curve:

$x=A\sin(at+\delta),\ y=B\cos(bt)$

Now if we tabulate:

$u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$

for $k=1,\ ..\ ,\ 2\pi / (\min(a,b)\delta t)$

for $\delta t$ suitably small. We can find $A,\ B,\ a,\ b,\ \delta$ which minimises the sum of the
squares of minimum distances of $(x_i,y_i)$ from the $(u_k,\ v_k)'s$ .

The Excell solver sould be up to this job.

(note the solution will not necessarily be unique but that won't bother
Excell:eek: )

RonL

I think this is a clever solution, however I don't think Excel's solver can handle it. But let me make sure I understand the algorithm.

First, get coordinates from the picture. Fix a value for

Next, define a function with the following 4 steps.
1. Set $M = 2\pi / (\min(a,b)\delta t).$
2. Tabulate $u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$ for $k=1,\ldots, M.$
3. For each $i = 1,\ldots,N$ , find a point $(u_j,v_j)$ out of the $M\ (u_k,\ v_k)'s$ which has the minimum distance from $(x_i,y_i)$ to $(u_j,v_j)$ .
4. Return the sum of the squares of the distances in step 3 as the value of $Dist.$
Finally, minimize

I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives. But the function, although continuous, will not be differentiable. Another problem with using Excel is that programming the function in a spreadsheet looks difficult.

However with a good programming language and a minimization routine that can handle a non-differentiable function, this could work. Good job!
June 3rd 2007, 07:33 AM
CaptainBlack
Quote:
Originally Posted by JakeD

I think this is a clever solution, however I don't think Excel's solver can handle it. But let me make sure I understand the algorithm.

First, get coordinates $(x_i,y_i),\ i = 1,\ldots,N$ from the picture. Fix a value for $\delta t.$

Next, define a function $Dist(A,\ B,\ a,\ b,\ \delta)$ with the following 4 steps.

Set $M = 2\pi / (\min(a,b)\delta t).$
Tabulate $u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$ for $k=1,\ldots, M.$
For each $i = 1,\ldots,N$ , find a point $(u_j,v_j)$ out of the $M\ (u_k,\ v_k)'s$ which has the minimum distance from $(x_i,y_i)$ to $(u_j,v_j)$ .
Return the sum of the squares of the distances in step 3 as the value of $Dist.$
Finally, minimize $Dist(A,\ B,\ a,\ b,\ \delta).$

I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives. But the $Dist$ function, although continuous, will not be differentiable. Another problem with using Excel is that programming the $Dist$ function in a spreadsheet looks difficult.

However with a good programming language and a minimization routine that can handle a non-differentiable function, this could work. Good job!
I don't believe that Excell's solver assumes the objective is differentiable,
(it's not a single algorithm but a suite IIRC) but it does help. Even if it does the
optimisation algorithm in Euler uses the Nelder-Mead simplex algorithm that
makes no such assumption.

Also Simulated Annealing and similar methods will do the job.

We will also need a penalty function added onto the objective to
stop the frequencies going too high.

RonL
June 3rd 2007, 08:26 AM
JakeD

Quote:

Originally Posted by JakeD

I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives.

Quote:

Originally Posted by CaptainBlack

I don't believe that Excell's solver assumes the objective is differentiable, (it's not a single algorithm but a suite IIRC) but it does help.

Actually, solver does assume differentiability and uses numerical approximations to the derivatives. See XL2000: Solver Uses Generalized Reduced Gradient Algorithm where it says "Derivatives (and gradients) play a crucial role in iterative methods in Microsoft Excel Solver" and describes the methods.
June 3rd 2007, 11:22 AM
CaptainBlack

Quote:

Originally Posted by JakeD

Actually, solver does assume differentiability and uses numerical approximations to the derivatives. See XL2000: Solver Uses Generalized Reduced Gradient Algorithm where it says "Derivatives (and gradients) play a crucial role in iterative methods in Microsoft Excel Solver" and describes the methods.

Ok.. but I would still like to see some data and try a few things either
Excel or Euler, and if deperate Matlab (sp.. yuck, just off to wash my mouth out).

RonL
June 3rd 2007, 11:27 AM
Jhevon

Quote:

Originally Posted by CaptainBlack

Ok.. but I would still like to see some data and try a few things either
Excel or Euler, and if deperate Matlab (sp.. yuck, just off to wash my mouth out).

RonL

lol, you dislike MatLab that much, Captain?
June 3rd 2007, 11:40 AM
CaptainBlack

Quote:

Originally Posted by Jhevon

lol, you dislike MatLab that much, Captain?

It provides prepackaged solutions, so you don't have to understand what
is going on or how your problem is being solved.

Also it is over priced, the three installations that I have cost my employer
~£10000 each plus annual maintenance charges (but its their own fault its
their policy that says we don't use free-ware)

(Now it does so happen that I don't use Matlab much because we are
allowed to use tools we build ourselves, and I have been building/modifying
Euler on our machines now for a decade - its like matlab, but leaner).

RonL
June 3rd 2007, 11:50 AM
Jhevon

Quote:

Originally Posted by CaptainBlack

It provides prepackaged solutions, so you don't have to understand what
is going on or how your problem is being solved.

i see, but aren't all similar utilities like that? Is Euler any different in that respect?

Quote:

Also it is over priced, the three installations that I have cost my employer
~£10000 each plus annual maintenance charges (but its their own fault its
their policy that says we don't use free-ware)

I remember you mentioning this before

Quote:

(Now it does so happen that I don't use Matlab much because we are
allowed to use tools we build ourselves, and I have been building/modifying
Euler on our machines now for a decade - its like matlab, but leaner).

is Euler available to the general public? if so, can you provide a link to where i can find it (obviously if i search for Euler i will get webpages about the mathematician Euler)

Show 40 post(s) from this thread on one page

All times are GMT -8. The time now is 01:40 PM.

Copyright © 2005-2013 Math Help Forum. All rights reserved.

Copyright © 2005-2013 Math Help Forum. All rights reserved.