Page 1 of 2 12 LastLast
Results 1 to 15 of 18

Math Help - Help with nonlinear regression / curve fitting

  1. #1
    khayman
    Guest

    Help with nonlinear regression / curve fitting

    Hi everyone,

    I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

    I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

    Thanks

    Steve
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Global Moderator

    Joined
    Nov 2005
    From
    New York City
    Posts
    10,616
    Thanks
    9
    Quote Originally Posted by khayman View Post
    Hi everyone,

    I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

    I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

    Thanks

    Steve
    Give my the points. I have a program on my Turing machine which can do regressions.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    is up to his old tricks again! Jhevon's Avatar
    Joined
    Feb 2007
    From
    New York, USA
    Posts
    11,663
    Thanks
    3
    Quote Originally Posted by ThePerfectHacker View Post
    Give my the points. I have a program on my Turing machine which can do regressions.
    what program is that?
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Global Moderator

    Joined
    Nov 2005
    From
    New York City
    Posts
    10,616
    Thanks
    9
    Quote Originally Posted by Jhevon View Post
    what program is that?
    The same one you have. Heir.
    Follow Math Help Forum on Facebook and Google+

  5. #5
    is up to his old tricks again! Jhevon's Avatar
    Joined
    Feb 2007
    From
    New York, USA
    Posts
    11,663
    Thanks
    3
    Quote Originally Posted by ThePerfectHacker View Post
    The same one you have. Heir.
    wow. this program can do a lot more than i thought it could. i should sit down one day and just explore its features.
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Global Moderator

    Joined
    Nov 2005
    From
    New York City
    Posts
    10,616
    Thanks
    9
    I do not know if you have ever taken Statistics. But perhaps you heard of "method of least squares". (It is explained a little bit in my Calculus mini-book).

    To test you can do this.
    1)FUNCTION
    2)INSERT POINT SERIES
    3)For example (1,1) (2,5) (3,10) (4,14)
    5)FUNCTION ---> INSERT TREDLINE
    6)Select LINEAR.

    You get the equation of "the line of best fit" and you also get the R^2 = .9979.... That is the number which measures how good your approximate is. (Called "coefficient of regession")

    For example if instead you choose EXPONENTIAL it is R^2=.6958 which is much much more worse.

    The best is when R^2=1 and worst is when R^2=0.
    Follow Math Help Forum on Facebook and Google+

  7. #7
    Senior Member
    Joined
    Apr 2006
    Posts
    399
    Awards
    1
    Quote Originally Posted by khayman View Post
    Hi everyone,

    I have recently been working on a project regarding lissajous figures and juggling (poi). I have reached a standpoint and was hoping someone could point me in the right direction.

    I have a picture of a curve traced into the air, I would like to analyse this curve to see if it really fits a lissajou ratio. Any ideas?

    Thanks

    Steve
    Quote Originally Posted by ThePerfectHacker View Post
    Give my the points. I have a program on my Turing machine which can do regressions.
    This doesn't look like a standard regression problem.

    From the Wiki page, the curve is generated by these equations



    but the picture in the (x,y) plane looks something something like this

    ,

    and the values for t are not observed.

    This does not appear to be a regression problem (linear or nonlinear) because that is for fitting a function to a set of points. But that picture in the (x,y) plane is not the graph of a well-defined function: there is more than one y value for each x value. (The regression can handle any set of points, but the function you're trying to fit must be a well-defined function.)

    At this point, I don't see any good solution.
    Follow Math Help Forum on Facebook and Google+

  8. #8
    Grand Panjandrum
    Joined
    Nov 2005
    From
    someplace
    Posts
    14,972
    Thanks
    4
    Quote Originally Posted by JakeD View Post
    This doesn't look like a standard regression problem.

    From the Wiki page, the curve is generated by these equations



    but the picture in the (x,y) plane looks something something like this

    ,

    and the values for t are not observed.

    This does not appear to be a regression problem (linear or nonlinear) because that is for fitting a function to a set of points. But that picture in the (x,y) plane is not the graph of a well-defined function: there is more than one y value for each x value. (The regression can handle any set of points, but the function you're trying to fit must be a well-defined function.)

    At this point, I don't see any good solution.
    Given a set of points (x_i,y_i),\ i=1,\ ..,\ N which we hope lie
    on a curve:

    x=A\sin(at+\delta),\ y=B\cos(bt)

    Now if we tabulate:

    u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))

    for k=1,\ ..\ ,\ 2\pi / (\min(a,b)\delta t)

    for \delta t suitably small. We can find A,\ B,\ a,\ b,\ \delta which minimises the sum of the
    squares of minimum distances of (x_i,y_i) from the (u_k,\ v_k)'s.

    The Excell solver sould be up to this job.

    (note the solution will not necessarily be unique but that won't bother
    Excell )

    RonL
    Follow Math Help Forum on Facebook and Google+

  9. #9
    Senior Member
    Joined
    Apr 2006
    Posts
    399
    Awards
    1
    Quote Originally Posted by CaptainBlack View Post
    Given a set of points (x_i,y_i),\ i=1,\ ..,\ N which we hope lie
    on a curve:

    x=A\sin(at+\delta),\ y=B\cos(bt)

    Now if we tabulate:

    u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t))

    for k=1,\ ..\ ,\ 2\pi / (\min(a,b)\delta t)

    for \delta t suitably small. We can find A,\ B,\ a,\ b,\ \delta which minimises the sum of the
    squares of minimum distances of (x_i,y_i) from the (u_k,\ v_k)'s.

    The Excell solver sould be up to this job.

    (note the solution will not necessarily be unique but that won't bother
    Excell )

    RonL
    I think this is a clever solution, however I don't think Excel's solver can handle it. But let me make sure I understand the algorithm.

    First, get coordinates (x_i,y_i),\ i = 1,\ldots,N from the picture. Fix a value for \delta t.

    Next, define a function Dist(A,\ B,\ a,\ b,\ \delta) with the following 4 steps.
    1. Set M = 2\pi / (\min(a,b)\delta t).
    2. Tabulate u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t)) for k=1,\ldots, M.
    3. For each i = 1,\ldots,N, find a point (u_j,v_j) out of the M\ (u_k,\ v_k)'s which has the minimum distance from (x_i,y_i) to (u_j,v_j).
    4. Return the sum of the squares of the distances in step 3 as the value of Dist.


    Finally, minimize Dist(A,\ B,\ a,\ b,\ \delta).

    I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives. But the Dist function, although continuous, will not be differentiable. Another problem with using Excel is that programming the Dist function in a spreadsheet looks difficult.

    However with a good programming language and a minimization routine that can handle a non-differentiable function, this could work. Good job!
    Follow Math Help Forum on Facebook and Google+

  10. #10
    Grand Panjandrum
    Joined
    Nov 2005
    From
    someplace
    Posts
    14,972
    Thanks
    4
    Quote Originally Posted by JakeD View Post
    I think this is a clever solution, however I don't think Excel's solver can handle it. But let me make sure I understand the algorithm.

    First, get coordinates (x_i,y_i),\ i = 1,\ldots,N from the picture. Fix a value for \delta t.

    Next, define a function Dist(A,\ B,\ a,\ b,\ \delta) with the following 4 steps.
    1. Set M = 2\pi / (\min(a,b)\delta t).
    2. Tabulate u_k=A\sin(a(k\ \delta t)+\delta),\ v_k=B\cos(b(k\ \delta t)) for k=1,\ldots, M.
    3. For each i = 1,\ldots,N, find a point (u_j,v_j) out of the M\ (u_k,\ v_k)'s which has the minimum distance from (x_i,y_i) to (u_j,v_j).
    4. Return the sum of the squares of the distances in step 3 as the value of Dist.
    Finally, minimize Dist(A,\ B,\ a,\ b,\ \delta).

    I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives. But the Dist function, although continuous, will not be differentiable. Another problem with using Excel is that programming the Dist function in a spreadsheet looks difficult.

    However with a good programming language and a minimization routine that can handle a non-differentiable function, this could work. Good job!
    I don't believe that Excell's solver assumes the objective is differentiable,
    (it's not a single algorithm but a suite IIRC) but it does help. Even if it does the
    optimisation algorithm in Euler uses the Nelder-Mead simplex algorithm that
    makes no such assumption.

    Also Simulated Annealing and similar methods will do the job.

    We will also need a penalty function added onto the objective to
    stop the frequencies going too high.

    RonL
    Follow Math Help Forum on Facebook and Google+

  11. #11
    Senior Member
    Joined
    Apr 2006
    Posts
    399
    Awards
    1
    Quote Originally Posted by JakeD View Post
    I don't think Excel's solver can handle this minimization because I believe it assumes the minimized function is differentiable and numerically approximates the derivatives.
    Quote Originally Posted by CaptainBlack View Post
    I don't believe that Excell's solver assumes the objective is differentiable, (it's not a single algorithm but a suite IIRC) but it does help.
    Actually, solver does assume differentiability and uses numerical approximations to the derivatives. See XL2000: Solver Uses Generalized Reduced Gradient Algorithm where it says "Derivatives (and gradients) play a crucial role in iterative methods in Microsoft Excel Solver" and describes the methods.
    Follow Math Help Forum on Facebook and Google+

  12. #12
    Grand Panjandrum
    Joined
    Nov 2005
    From
    someplace
    Posts
    14,972
    Thanks
    4
    Quote Originally Posted by JakeD View Post
    Actually, solver does assume differentiability and uses numerical approximations to the derivatives. See XL2000: Solver Uses Generalized Reduced Gradient Algorithm where it says "Derivatives (and gradients) play a crucial role in iterative methods in Microsoft Excel Solver" and describes the methods.
    Ok.. but I would still like to see some data and try a few things either
    Excel or Euler, and if deperate Matlab (sp.. yuck, just off to wash my mouth out).

    RonL
    Follow Math Help Forum on Facebook and Google+

  13. #13
    is up to his old tricks again! Jhevon's Avatar
    Joined
    Feb 2007
    From
    New York, USA
    Posts
    11,663
    Thanks
    3
    Quote Originally Posted by CaptainBlack View Post
    Ok.. but I would still like to see some data and try a few things either
    Excel or Euler, and if deperate Matlab (sp.. yuck, just off to wash my mouth out).

    RonL
    lol, you dislike MatLab that much, Captain?
    Follow Math Help Forum on Facebook and Google+

  14. #14
    Grand Panjandrum
    Joined
    Nov 2005
    From
    someplace
    Posts
    14,972
    Thanks
    4
    Quote Originally Posted by Jhevon View Post
    lol, you dislike MatLab that much, Captain?
    It provides prepackaged solutions, so you don't have to understand what
    is going on or how your problem is being solved.

    Also it is over priced, the three installations that I have cost my employer
    ~10000 each plus annual maintenance charges (but its their own fault its
    their policy that says we don't use free-ware)

    (Now it does so happen that I don't use Matlab much because we are
    allowed to use tools we build ourselves, and I have been building/modifying
    Euler on our machines now for a decade - its like matlab, but leaner).

    RonL
    Follow Math Help Forum on Facebook and Google+

  15. #15
    is up to his old tricks again! Jhevon's Avatar
    Joined
    Feb 2007
    From
    New York, USA
    Posts
    11,663
    Thanks
    3
    Quote Originally Posted by CaptainBlack View Post
    It provides prepackaged solutions, so you don't have to understand what
    is going on or how your problem is being solved.
    i see, but aren't all similar utilities like that? Is Euler any different in that respect?


    Also it is over priced, the three installations that I have cost my employer
    ~10000 each plus annual maintenance charges (but its their own fault its
    their policy that says we don't use free-ware)
    I remember you mentioning this before


    (Now it does so happen that I don't use Matlab much because we are
    allowed to use tools we build ourselves, and I have been building/modifying
    Euler on our machines now for a decade - its like matlab, but leaner).
    is Euler available to the general public? if so, can you provide a link to where i can find it (obviously if i search for Euler i will get webpages about the mathematician Euler)
    Follow Math Help Forum on Facebook and Google+

Page 1 of 2 12 LastLast

Similar Math Help Forum Discussions

  1. Curve fitting nonlinear polynomial
    Posted in the Differential Geometry Forum
    Replies: 8
    Last Post: November 6th 2012, 08:32 AM
  2. Nonlinear Fitting
    Posted in the Advanced Applied Math Forum
    Replies: 7
    Last Post: July 4th 2011, 12:40 PM
  3. Curve fitting when typical regression models fail
    Posted in the Business Math Forum
    Replies: 3
    Last Post: February 3rd 2011, 07:38 PM
  4. Curve Fitting
    Posted in the Calculus Forum
    Replies: 8
    Last Post: September 15th 2010, 02:09 PM
  5. Polynomial Fitting? Regression?
    Posted in the Pre-Calculus Forum
    Replies: 0
    Last Post: March 2nd 2009, 10:59 AM

Search Tags


/mathhelpforum @mathhelpforum