Thread: Help with non-linear least squares

I'm trying to find least squares coefficients using the Gauss-Newton method (using scilab as my numerical analysis software)

Given the following data vectors:

x = [57.9 108.2 149.6 227.9 778.3 1429.4] ' ;
y = [0.2 0.6 1.0 1.9 11.9 29.5] ' ;

(The ' sign means transposed in scilab notation, so they are column vectors)

I'm trying to adjust the data to the following equation:

(you will find this to be Kepler's third law on planetary motion)

,where a and b are the coefficients to find.

I'm using the following code to find , and an initial value of:

So here's the problem. In the first iteration, the Jacobian grows to be a very badly scaled matrix:



note that the left column is , which makes for a huge number compared to the right column, making the matrix very badly scaled, so i never get to the solution.

I've tried everything from QR decomposition to logarithmic scaling, but nothing seems to work for me.

Any help will be much appreciated.

Grinding through a non-linear least squares problem efficiently isn't a really a statistical problem per se. That being said, I'm pretty sure you can find this via iteratively re-weighted least squares, or else just straight-forward Newton's Method.

If you want to do it by hand, that's all I have to offer, but if my thinking is correct the least squares solution should be the same as the one you get if you fit a GLM with gaussian errors and a log link function, which is something I can do quickly, so I did. The solution I got was

If you're doing least squares, how about the following. Let be your data set, and define



We compute the partial derivatives of with respect to and hoping that we'll be able to set them equal to zero, and thus solve for and We get

and



Setting

yields that



Setting

yields that



Solve this second equation numerically for , then plug the result into the equation for and you're done.

Caveat: I have no idea how numerically stable your algorithm for solving for is going to be, but it does seem straight-forward that once you have solved for it, should be straight-forward.

Does this make sense?

And yes, theodds is right: this question probably belongs in the Analysis, Topology, and Differential Geometry forum. Or you could also put it in either Advanced Applied Math or in Other Advanced Topics.

First of all, thanks for your replies.

You may be right about the forum placing. To be honest I didn't really know where to put the question so, sorry for that.

theodds, the values you came up with seem pretty close to the ones I'm looking for. I'm not really familiar with the method you're mentioning, but I'll see if i can get the same results.

Ackbeet, I followed some pretty similar steps to get to the the initial values I mentioned above, but they come out with significant errors, that's why I went for the Gauss-Newton method.

Anyways, thanks a lot for your replies.

Cheers.