Results 1 to 2 of 2

Math Help - Multidimensional Newton's method optimization

  1. #1
    Newbie
    Joined
    Oct 2009
    Posts
    3

    Multidimensional Newton's method optimization

    The followings. I know this is Hessian matrix. But i still could not understand how it work. Could you help me expand these 2 formula. And tell me how this process work.
    ----------------------------------
    http://en.wikipedia.org/wiki/Newton%...n_optimization

    This iterative scheme can be generalized to several dimensions by replacing the derivative with the gradient, , and the reciprocal of the second derivative with the inverse of the Hessian matrix, . One obtains the iterative scheme

    Usually Newton's method is modified to include a small step size γ > 0 instead of γ = 1
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Grand Panjandrum
    Joined
    Nov 2005
    From
    someplace
    Posts
    14,972
    Thanks
    4
    Quote Originally Posted by jasonHuang View Post
    The followings. I know this is Hessian matrix. But i still could not understand how it work. Could you help me expand these 2 formula. And tell me how this process work.
    ----------------------------------
    Newton's method in optimization - Wikipedia, the free encyclopedia

    This iterative scheme can be generalized to several dimensions by replacing the derivative with the gradient, , and the reciprocal of the second derivative with the inverse of the Hessian matrix, . One obtains the iterative scheme

    Usually Newton's method is modified to include a small step size γ > 0 instead of γ = 1
    We assume that the (scalar) function that we seek the minimum of is locally quadratic, and take the two term Taylor expansion for the gradient of the function (we are going to use N-R to find a zero of the gradient).

    This gives:

    \nabla f(\bold{x})=\nabla f(\bold{a}) + \bold{H}f(\bold{a})(\bold{x}-\bold{a})

    Thus if \bold x is the minimum we seek then \nabla f(\bold x)=0, and so:

    \bold Hf(\bold a)(\bold x-\bold a)=-\nabla f(\bold a)

    or:

    \bold x=\bold a-[\bold Hf(\bold a)]^{-1}\nabla f(\bold a)

    Now we use this as an itteration formula (because the function is not really a quadratic surface):

    \bold x_{n+1}=\bold x_n-[\bold Hf(\bold x_n)]^{-1}\nabla f(\bold x_n)

    CB
    Last edited by CaptainBlack; October 3rd 2009 at 11:37 PM.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Newton's Method
    Posted in the Calculus Forum
    Replies: 5
    Last Post: December 21st 2009, 02:48 PM
  2. newton's method
    Posted in the Calculus Forum
    Replies: 1
    Last Post: December 8th 2009, 08:12 PM
  3. newton's method
    Posted in the Calculus Forum
    Replies: 8
    Last Post: December 30th 2008, 09:40 PM
  4. Newton's Method
    Posted in the Calculus Forum
    Replies: 1
    Last Post: October 26th 2007, 11:53 PM
  5. need help! using newton's method
    Posted in the Calculus Forum
    Replies: 2
    Last Post: October 23rd 2006, 10:41 PM

Search Tags


/mathhelpforum @mathhelpforum