Results 1 to 13 of 13

Math Help - Finding The Gradient Of A Matrix Equation

  1. #1
    Newbie
    Joined
    May 2011
    Posts
    8

    Finding The Gradient Of A Matrix Equation

    f(x) = x' * E * x

    x' = x transpose
    E = a covariance matrix

    The problem is: prove that f(x) is convex. The solution has the equations:

    gradient of f = 2 * E * x <- How is the gradient calculated here from f(x)?
    gradient^2 of f = 2 * E

    E > 0 since it is a covariance matrix, therefore f is convex

    I want to learn how to calculate the gradient from the function f, the answer is given but I don't know how it was determined.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Master Of Puppets
    pickslides's Avatar
    Joined
    Sep 2008
    From
    Melbourne
    Posts
    5,236
    Thanks
    28
    Quote Originally Posted by jjpioli View Post

    gradient of f = 2 * E * x <- How is the gradient calculated here from f(x)?
    As \displastyle X'EX is quadratic. Multiply it out to convince yourself.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    May 2011
    Posts
    8
    By quadratic, do you mean if we have a 2x2 matrix:

    top left represents x^2
    top right represents x*y
    bottom left represents y*x
    bottom right represents y^2

    So then we can build a quadratic equation from that with x and y variables, then we simply find the gradient of that function?

    I still don't understand how you can simply go from x' * E * x to grad(f) = 2*E*x
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Master Of Puppets
    pickslides's Avatar
    Joined
    Sep 2008
    From
    Melbourne
    Posts
    5,236
    Thanks
    28
    There is a vector tranpose with your variables, say x1 and x2. Then E the 2x2 matrix is eij constants for covariance where e_{ij} = e_{ji}, where i \neq j. Finally x1 and x2 again. Expand them out!
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Newbie
    Joined
    May 2011
    Posts
    8
    I expanded it out, here is my work:

    http://i.imgur.com/iLcaS.jpg

    I have one resulting quadratic equation, do I find the gradient of that?
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Master Of Puppets
    pickslides's Avatar
    Joined
    Sep 2008
    From
    Melbourne
    Posts
    5,236
    Thanks
    28
    Very close, discard the C's altogether, now

    E = [ e11 e12
    e21 e21 ]

    Try again, remember e12 = e21
    Follow Math Help Forum on Facebook and Google+

  7. #7
    Master Of Puppets
    pickslides's Avatar
    Joined
    Sep 2008
    From
    Melbourne
    Posts
    5,236
    Thanks
    28
    You should get

    \displaystyle e_{11}x_1^2+2e_{12}x_1x_2+e_{22}x_2^2

    Differentiate this with respect to x_1 and x_2 and you should be able to see the system 2EX
    Follow Math Help Forum on Facebook and Google+

  8. #8
    Newbie
    Joined
    May 2011
    Posts
    8
    http://i.imgur.com/69Lgj.jpg

    After expanding x'Ex, to get the gradient, I would derive with respect to x1, and then derive with respect to x2, resulting in 2*(e12), but my solution says that the gradient is 2*E*x, which is not equal to 2*(e12).

    Is my understanding incorrect?
    Follow Math Help Forum on Facebook and Google+

  9. #9
    Master Of Puppets
    pickslides's Avatar
    Joined
    Sep 2008
    From
    Melbourne
    Posts
    5,236
    Thanks
    28
    I'm not following your work very well and where are your actual derivatives?

    I get

    \displaystyle f_{x_1} = 2e_{11}x_1+ 2e_{12}x_2

    and

    \displaystyle f_{x_2} = 2e_{21}x_1+ 2e_{22}x_2

    2 is clearly a factor for the whole system, now just write back to matrix form.
    Follow Math Help Forum on Facebook and Google+

  10. #10
    Newbie
    Joined
    May 2011
    Posts
    8
    I see:

    http://i.imgur.com/7nQ6z.jpg

    Now I have a few additional questions. The gradient is a vector, that represents the direction of greatest slope increase, so I have two elements in my vector, do the elements represent the (x1 or x2) coordinates or the (i or j) coordinates?

    Also, from the solution to this problem, it appears that the problem takes the following steps:

    f= x'Ex
    grad f = 2Ex
    grad^2 f = 2E

    Is there a rule with matrices where I can simply deduce the gradient without expanding everything out? I feel, now that I understand the underlying concept (through expansion) I should be able to use shortcuts to quicken my problem solving skills. THanks
    Follow Math Help Forum on Facebook and Google+

  11. #11
    Master Of Puppets
    pickslides's Avatar
    Joined
    Sep 2008
    From
    Melbourne
    Posts
    5,236
    Thanks
    28
    Not only the greatest slope increase but the gradient at any x_1 and x_2

    As for the generalisation, you have proven that this process maps back to

    \displaystyle y = ax^2

    \displaystyle y' = 2ax

    \displaystyle y'' = 2a

    Well done!
    Follow Math Help Forum on Facebook and Google+

  12. #12
    Newbie
    Joined
    May 2011
    Posts
    8
    y = ax^2
    y' = 2ax
    y'' = 2a

    If x is a 2x2 matrix, and E is our same covariance matrix, x'Ex does not appear to be equal to Ex^2. I'm still confused
    Follow Math Help Forum on Facebook and Google+

  13. #13
    Master Of Puppets
    pickslides's Avatar
    Joined
    Sep 2008
    From
    Melbourne
    Posts
    5,236
    Thanks
    28
    Quote Originally Posted by jjpioli View Post

    If x is a 2x2 matrix, and E is our same covariance matrix, x'Ex does not appear to be equal to Ex^2. I'm still confused
    Because X is not a 2x2, E is a 2x2 matrix.

    Here is your expression with their corresponding orders X' E X (1x2) (2x2) (2x1) which expands out to a 1x1 as shown in post #7. The matrix expression is a quadratic form. That's all you need to know.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Replies: 0
    Last Post: January 15th 2012, 01:32 PM
  2. Replies: 6
    Last Post: July 23rd 2010, 12:36 PM
  3. Finding the gradient
    Posted in the Pre-Calculus Forum
    Replies: 2
    Last Post: December 14th 2009, 05:53 PM
  4. Finding Gradient
    Posted in the Algebra Forum
    Replies: 1
    Last Post: October 14th 2009, 01:54 AM
  5. Finding the gradient function.... lost
    Posted in the Calculus Forum
    Replies: 6
    Last Post: July 29th 2009, 06:01 AM

Search Tags


/mathhelpforum @mathhelpforum