Help with nonlinear regression / curve fitting

Hi everyone,

I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

Thanks

Steve

Quote:

---

Originally Posted by khayman

Hi everyone,

I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

Thanks

Steve

---

Give my the points. I have a program on my Turing machine which can do regressions.

Quote:

---

Originally Posted by ThePerfectHacker

Give my the points. I have a program on my Turing machine which can do regressions.

---

what program is that?

Quote:

---

Originally Posted by Jhevon

what program is that?

---

The same one you have. Heir.

Quote:

---

Originally Posted by ThePerfectHacker

The same one you have. Heir.

---

wow. this program can do a lot more than i thought it could. i should sit down one day and just explore its features.

I do not know if you have ever taken Statistics. But perhaps you heard of "method of least squares". (It is explained a little bit in my Calculus mini-book).

To test you can do this.
1)FUNCTION
2)INSERT POINT SERIES
3)For example (1,1) (2,5) (3,10) (4,14)
5)FUNCTION ---> INSERT TREDLINE
6)Select LINEAR.

You get the equation of "the line of best fit" and you also get the R^2 = .9979.... That is the number which measures how good your approximate is. (Called "coefficient of regession")

For example if instead you choose EXPONENTIAL it is R^2=.6958 which is much much more worse.

The best is when R^2=1 and worst is when R^2=0.

Quote:

---

Originally Posted by khayman

Hi everyone,

I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

Thanks

Steve

---

Quote:

---

Originally Posted by ThePerfectHacker

Give my the points. I have a program on my Turing machine which can do regressions.

---

This doesn't look like a standard regression problem.

From the Wiki page, the curve is generated by these equations

http://upload.wikimedia.org/math/e/4...e0cef4e09f.png

but the picture in the (x,y) plane looks something something like this

http://upload.wikimedia.org/wikipedi...e_3by2.svg.png,

and the values for t are not observed.

This does not appear to be a regression problem (linear or nonlinear) because that is for fitting a function to a set of points. But that picture in the (x,y) plane is not the graph of a well-defined function: there is more than one y value for each x value. (The regression can handle any set of points, but the function you're trying to fit must be a well-defined function.)

At this point, I don't see any good solution.

Quote:

---

Originally Posted by JakeD

This doesn't look like a standard regression problem.

From the Wiki page, the curve is generated by these equations

http://upload.wikimedia.org/math/e/4...e0cef4e09f.png

but the picture in the (x,y) plane looks something something like this

http://upload.wikimedia.org/wikipedi...e_3by2.svg.png,

and the values for t are not observed.

This does not appear to be a regression problem (linear or nonlinear) because that is for fitting a function to a set of points. But that picture in the (x,y) plane is not the graph of a well-defined function: there is more than one y value for each x value. (The regression can handle any set of points, but the function you're trying to fit must be a well-defined function.)

At this point, I don't see any good solution.

---

Given a set of points $\displaystyle (x_i,y_i),\ i=1,\ ..,\ N$ which we hope lie
on a curve:

$\displaystyle x=A\sin(at+\delta),\ y=B\cos(bt)$

Now if we tabulate:

$\displaystyle u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$

for $\displaystyle k=1,\ ..\ ,\ 2\pi / (\min(a,b)\delta t)$

for $\displaystyle \delta t$ suitably small. We can find $\displaystyle A,\ B,\ a,\ b,\ \delta$ which minimises the sum of the
squares of minimum distances of $\displaystyle (x_i,y_i)$ from the $\displaystyle (u_k,\ v_k)'s$.

The Excell solver sould be up to this job.

(note the solution will not necessarily be unique but that won't bother
Excell:eek: )

RonL

Quote:

---

Originally Posted by CaptainBlack

Given a set of points $\displaystyle (x_i,y_i),\ i=1,\ ..,\ N$ which we hope lie
on a curve:

$\displaystyle x=A\sin(at+\delta),\ y=B\cos(bt)$

Now if we tabulate:

$\displaystyle u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$

for $\displaystyle k=1,\ ..\ ,\ 2\pi / (\min(a,b)\delta t)$

for $\displaystyle \delta t$ suitably small. We can find $\displaystyle A,\ B,\ a,\ b,\ \delta$ which minimises the sum of the
squares of minimum distances of $\displaystyle (x_i,y_i)$ from the $\displaystyle (u_k,\ v_k)'s$.

The Excell solver sould be up to this job.

(note the solution will not necessarily be unique but that won't bother
Excell:eek: )

RonL

---

I think this is a clever solution, however I don't think Excel's solver can handle it. But let me make sure I understand the algorithm.

First, get coordinates $\displaystyle (x_i,y_i),\ i = 1,\ldots,N$ from the picture. Fix a value for $\displaystyle \delta t.$

Next, define a function $\displaystyle Dist(A,\ B,\ a,\ b,\ \delta)$ with the following 4 steps.

1. Set $\displaystyle M = 2\pi / (\min(a,b)\delta t).$
2. Tabulate $\displaystyle u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$ for $\displaystyle k=1,\ldots, M.$
3. For each $\displaystyle i = 1,\ldots,N$, find a point $\displaystyle (u_j,v_j)$ out of the $\displaystyle M\ (u_k,\ v_k)'s$ which has the minimum distance from $\displaystyle (x_i,y_i)$ to $\displaystyle (u_j,v_j)$.
4. Return the sum of the squares of the distances in step 3 as the value of $\displaystyle Dist.$

Finally, minimize $\displaystyle Dist(A,\ B,\ a,\ b,\ \delta).$

I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives. But the $\displaystyle Dist$ function, although continuous, will not be differentiable. Another problem with using Excel is that programming the $\displaystyle Dist$ function in a spreadsheet looks difficult.

However with a good programming language and a minimization routine that can handle a non-differentiable function, this could work. Good job!

Quote:

---

Originally Posted by JakeD

I think this is a clever solution, however I don't think Excel's solver can handle it. But let me make sure I understand the algorithm.

First, get coordinates $\displaystyle (x_i,y_i),\ i = 1,\ldots,N$ from the picture. Fix a value for $\displaystyle \delta t.$

Next, define a function $\displaystyle Dist(A,\ B,\ a,\ b,\ \delta)$ with the following 4 steps.

1. Set $\displaystyle M = 2\pi / (\min(a,b)\delta t).$
2. Tabulate $\displaystyle u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))$ for $\displaystyle k=1,\ldots, M.$
3. For each $\displaystyle i = 1,\ldots,N$, find a point $\displaystyle (u_j,v_j)$ out of the $\displaystyle M\ (u_k,\ v_k)'s$ which has the minimum distance from $\displaystyle (x_i,y_i)$ to $\displaystyle (u_j,v_j)$.
4. Return the sum of the squares of the distances in step 3 as the value of $\displaystyle Dist.$

Finally, minimize $\displaystyle Dist(A,\ B,\ a,\ b,\ \delta).$

I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives. But the $\displaystyle Dist$ function, although continuous, will not be differentiable. Another problem with using Excel is that programming the $\displaystyle Dist$ function in a spreadsheet looks difficult.

However with a good programming language and a minimization routine that can handle a non-differentiable function, this could work. Good job!

---

I don't believe that Excell's solver assumes the objective is differentiable,
(it's not a single algorithm but a suite IIRC) but it does help. Even if it does the
optimisation algorithm in Euler uses the Nelder-Mead simplex algorithm that
makes no such assumption.

Also Simulated Annealing and similar methods will do the job.

We will also need a penalty function added onto the objective to
stop the frequencies going too high.

RonL

Quote:

---

Originally Posted by JakeD

I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives.

---

Quote:

---

Originally Posted by CaptainBlack

I don't believe that Excell's solver assumes the objective is differentiable, (it's not a single algorithm but a suite IIRC) but it does help.

---

Actually, solver does assume differentiability and uses numerical approximations to the derivatives. See XL2000: Solver Uses Generalized Reduced Gradient Algorithm where it says "Derivatives (and gradients) play a crucial role in iterative methods in Microsoft Excel Solver" and describes the methods.

Quote:

---

Originally Posted by JakeD

Actually, solver does assume differentiability and uses numerical approximations to the derivatives. See XL2000: Solver Uses Generalized Reduced Gradient Algorithm where it says "Derivatives (and gradients) play a crucial role in iterative methods in Microsoft Excel Solver" and describes the methods.

---

Ok.. but I would still like to see some data and try a few things either
Excel or Euler, and if deperate Matlab (sp.. yuck, just off to wash my mouth out).

RonL

Quote:

---

Originally Posted by CaptainBlack

Ok.. but I would still like to see some data and try a few things either
Excel or Euler, and if deperate Matlab (sp.. yuck, just off to wash my mouth out).

RonL

---

lol, you dislike MatLab that much, Captain?

Quote:

---

Originally Posted by Jhevon

lol, you dislike MatLab that much, Captain?

---

It provides prepackaged solutions, so you don't have to understand what
is going on or how your problem is being solved.

Also it is over priced, the three installations that I have cost my employer
~£10000 each plus annual maintenance charges (but its their own fault its
their policy that says we don't use free-ware)

(Now it does so happen that I don't use Matlab much because we are
allowed to use tools we build ourselves, and I have been building/modifying
Euler on our machines now for a decade - its like matlab, but leaner).

RonL

Quote:

---

Originally Posted by CaptainBlack

It provides prepackaged solutions, so you don't have to understand what
is going on or how your problem is being solved.

---

i see, but aren't all similar utilities like that? Is Euler any different in that respect?

Quote:

---

Also it is over priced, the three installations that I have cost my employer
~£10000 each plus annual maintenance charges (but its their own fault its
their policy that says we don't use free-ware)

---

I remember you mentioning this before

Quote:

---

(Now it does so happen that I don't use Matlab much because we are
allowed to use tools we build ourselves, and I have been building/modifying
Euler on our machines now for a decade - its like matlab, but leaner).

---

is Euler available to the general public? if so, can you provide a link to where i can find it (obviously if i search for Euler i will get webpages about the mathematician Euler)