# Help/suggestions on optimization problem

• Jun 28th 2013, 12:21 PM
Lord Voldemort
Help/suggestions on optimization problem
Hi all, I'm currently working at a physics research group and I need to find a way to solve an optimization problem with multiple non linear equations. I don't have any significant background, but I've looked in a few books we have around the office and didn't find much - I'm hoping you all can help me out. Barring anything else, suggestions on good books are much appreciated as well. I've got equations of the form $\displaystyle \\ \hat{y}_{n} = k(x_{1} {{a_{1}}^{x_{2}}+ ... +x_{1} {{a_{n}}^{x_{2}})$ and I'm trying to find the values of $\displaystyle x_{1}$ and $\displaystyle x_{2}$ that minimize the total error $\displaystyle \sum{(y_{n} - \hat{y}_{n})^{2}}$. Can anyone point me in the right direction?
• Jun 28th 2013, 03:56 PM
Shakarri
Re: Help/suggestions on optimization problem
If $\displaystyle E=\sum(y_n-\hat{y}_n)$

The error will be smallest when both
$\displaystyle \frac{dE}{dx_1}=0$ and $\displaystyle \frac{dE}{dx_2}=0$

This gives you two non linear equations to solve for x1 and x2. You can use Newton's method for a system of equations to get an approximate solution (by approximate I mean the estimation will converge to the true solution but will unlikely every reach the solution so you can get very very close if you repeat the procedure enough times).
• Jul 1st 2013, 11:20 AM
Lord Voldemort
Re: Help/suggestions on optimization problem
Thank you for the suggestion! I'm going to start implementing that on some data sets. However, do you have suggestions besides Newton's method? That doesn't converge fast enough for the limited computing power I have and the large number of data sets I have to process. I'm going to have cases that I want to minimize with 8 different variables.