Results 1 to 10 of 10

Math Help - How to get the Hessian Matrix please?

  1. #1
    Newbie
    Joined
    Aug 2008
    Posts
    12

    Question How to get the Hessian Matrix please?

    I have got the following function constructed by matrices and vectors:

     <br />
f(x, \alpha) = \frac{1}{2}[(Ax-y_1)^2+(AP(\alpha)x-y_2)^2]<br />

    in which, x, y_1, y_2 are vectors. A, P(\alpha) are matrices.
    The gradients with respect to x, \alpha are:

     <br />
\frac{\partial{f(x, \alpha)}}{\partial{x}} = A^T A x - A^T y_1 + P(\alpha)^T A^T A P(\alpha) x - P(\alpha)^T A^T y_2<br />

     <br />
\frac{\partial{f(x, \alpha)}}{\partial{\alpha}} = x^T P(\alpha)^T A^T A P(\alpha) x - x^T P'(\alpha)^T A^T y_2<br />

    in which, A^T is the transpose of A and P'(\alpha) is the first order derivative of P.
    Anyone knows how to get the Hessian of the f(x, \alpha)? Thanks a lot!
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Junior Member
    Joined
    Aug 2008
    Posts
    74
    Thanks
    1
    just take second derivatives the way you took first ones, and write the Hessian as the 2x2 matrix each entry of which is also a matrix (I guess it can be viewed as a tensor if needed)
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Aug 2008
    Posts
    12

    Exclamation

    Quote Originally Posted by choovuck View Post
    just take second derivatives the way you took first ones, and write the Hessian as the 2x2 matrix each entry of which is also a matrix (I guess it can be viewed as a tensor if needed)
    Thanks a lot! But could you please tell me more details? I guess it should be:

    <br />
HessianMatrix=<br />
\begin{bmatrix}<br />
\frac{\partial{f(x, \alpha)}}{\partial{x}\partial{x}} <br />
&\frac{\partial{f(x, \alpha)}}{\partial{\alpha}\partial{\alpha}} \\<br />
\frac{\partial{f(x, \alpha)}}{\partial{\alpha}\partial{\alpha}} <br />
&0<br />
\end{bmatrix}<br />

    because the Hessian should be symmetric, am I right?

    Moreover, due to the equation:

     <br />
\frac{\partial{f(x, \alpha)}}{\partial{\alpha}} = x^T P(\alpha)^T A^T A P(\alpha) x - x^T P'(\alpha)^T A^T y_2<br />

    then the second order partial derivative:

     <br />
\frac{\partial{f(x, \alpha)}}{\partial{\alpha}\partial{\alpha}}<br />

    will involve P''(\alpha). I do not know if I am correct. Please show me the analytic form of the Hessian. Thank you very much!
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Junior Member
    Joined
    Aug 2008
    Posts
    74
    Thanks
    1
    Quote Originally Posted by ggyyree View Post
    I guess it should be:

    <br />
HessianMatrix=<br />
\begin{bmatrix}<br />
\frac{\partial{f(x, \alpha)}}{\partial{x}\partial{x}} <br />
&\frac{\partial{f(x, \alpha)}}{\partial{\alpha}\partial{\alpha}} \\<br />
\frac{\partial{f(x, \alpha)}}{\partial{\alpha}\partial{\alpha}} <br />
&0<br />
\end{bmatrix}<br />

    because the Hessian should be symmetric, am I right?
    no, the Hessian matrix has as its (i,j)-th element the derivative \frac{\partial^2 f}{\partial x_i \partial x_j}, so in our case
    <br />
HessianMatrix=<br />
\begin{bmatrix}<br />
\frac{\partial^2{f(x, \alpha)}}{\partial{x}^2} <br />
&\frac{\partial^2{f(x, \alpha)}}{\partial{x}\partial{\alpha}} \\<br />
\frac{\partial^2{f(x, \alpha)}}{\partial{\alpha}\partial{x}} <br />
&\frac{\partial^2{f(x, \alpha)}}{\partial{\alpha}\partial{\alpha}} <br />
\end{bmatrix}<br />

    It is symmetric since \frac{\partial^2{f(x, \alpha)}}{\partial{\alpha}\partial{x}}=\frac{\part  ial^2{f(x, \alpha)}}{\partial{x}\partial{\alpha}}

    I'm not very comfortable with those matrix gradients (I can't even convince myself why your \frac{\partial f}{\partial \alpha} is correct...), so I won't help you with the actual computations, sorry
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Junior Member
    Joined
    Aug 2008
    Posts
    74
    Thanks
    1
    P.S. When you wrote f(x, \alpha) = \frac{1}{2}[(Ax-y_1)^2+(AP(\alpha)x-y_2)^2], I'm assuming you meant f(x, \alpha) = \frac{1}{2}[(Ax-y_1)^T (Ax-y_1)+(AP(\alpha)x-y_2)^T (AP(\alpha)x-y_2)]
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Newbie
    Joined
    Aug 2008
    Posts
    12
    Quote Originally Posted by choovuck View Post
    no, the Hessian matrix has as its (i,j)-th element the derivative \frac{\partial^2 f}{\partial x_i \partial x_j}, so in our case
    <br />
HessianMatrix=<br />
\begin{bmatrix}<br />
\frac{\partial^2{f(x, \alpha)}}{\partial{x}^2} <br />
&\frac{\partial^2{f(x, \alpha)}}{\partial{x}\partial{\alpha}} \\<br />
\frac{\partial^2{f(x, \alpha)}}{\partial{\alpha}\partial{x}} <br />
&\frac{\partial^2{f(x, \alpha)}}{\partial{\alpha}\partial{\alpha}} <br />
\end{bmatrix}<br />

    It is symmetric since \frac{\partial^2{f(x, \alpha)}}{\partial{\alpha}\partial{x}}=\frac{\part  ial^2{f(x, \alpha)}}{\partial{x}\partial{\alpha}}

    I'm not very comfortable with those matrix gradients (I can't even convince myself why your \frac{\partial f}{\partial \alpha} is correct...), so I won't help you with the actual computations, sorry

    Yes, you are quite right! Thanks a lot!
    Follow Math Help Forum on Facebook and Google+

  7. #7
    Newbie
    Joined
    Aug 2008
    Posts
    12

    Thumbs up

    Quote Originally Posted by choovuck View Post
    P.S. When you wrote f(x, \alpha) = \frac{1}{2}[(Ax-y_1)^2+(AP(\alpha)x-y_2)^2], I'm assuming you meant f(x, \alpha) = \frac{1}{2}[(Ax-y_1)^T (Ax-y_1)+(AP(\alpha)x-y_2)^T (AP(\alpha)x-y_2)]
    Yes, you are right!
    Follow Math Help Forum on Facebook and Google+

  8. #8
    Newbie
    Joined
    Aug 2008
    Posts
    12
    According to your suggestion I have derived the formula. Thank you very much!
    Follow Math Help Forum on Facebook and Google+

  9. #9
    Newbie
    Joined
    Aug 2008
    Posts
    12

    Question

    By the way, may I ask you another related question please?

    I want to know the relations between 'Gradient', 'Jacobian' and 'Hessian'. What I think they are:

    'Function': F
    'Gradient': G
    'Jacobian' : J
    'Hessian': H

    G= -J^T \times F
    H= J^T \times J

    Am I right? Are there any definition for these relations please? Thanks very much!
    Follow Math Help Forum on Facebook and Google+

  10. #10
    Junior Member
    Joined
    Aug 2008
    Posts
    74
    Thanks
    1
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Hessian Matrix help
    Posted in the Calculus Forum
    Replies: 1
    Last Post: December 10th 2011, 12:20 PM
  2. help with hessian matrix
    Posted in the Calculus Forum
    Replies: 2
    Last Post: November 14th 2011, 10:37 PM
  3. doubt hessian matrix
    Posted in the Calculus Forum
    Replies: 0
    Last Post: October 17th 2011, 12:37 PM
  4. Hessian matrix
    Posted in the Advanced Algebra Forum
    Replies: 0
    Last Post: December 22nd 2009, 01:46 PM
  5. Hessian matrix for 2d function
    Posted in the Differential Geometry Forum
    Replies: 2
    Last Post: September 27th 2009, 11:24 AM

Search Tags


/mathhelpforum @mathhelpforum