Results 1 to 6 of 6

Thread: Gradient of a Euclidean function

  1. #1
    Newbie
    Joined
    Aug 2015
    From
    Philippines
    Posts
    7

    Gradient of a Euclidean function

    Folks - I know this is too basic but I really need your help. I haven't mastered the properties of Matrix Differentiation. Haven't had any Matrix Calculus in college. Please share your knowledge. Thanks!
    Please help explain the solution, if possibly, in a clear and detailed way
    Seems like there's a relationship between differentiation of a matrix and its transposition. Please see the attached image.
    Attached Thumbnails Attached Thumbnails Gradient of a Euclidean function-gradient-euclidean-wrt-.jpg  
    Follow Math Help Forum on Facebook and Google+

  2. #2
    MHF Contributor
    Joined
    Nov 2013
    From
    California
    Posts
    6,581
    Thanks
    2913

    Re: Gradient of a Euclidean function

    Do you think you could post the problem so we can see what it is.

    Looking at the image you posted I have no idea what you're trying to do.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Aug 2015
    From
    Philippines
    Posts
    7

    Re: Gradient of a Euclidean function

    Sorry. I missed something. Anyways, please see the screenshot below for the exact cost function. I'm trying to get the gradient wrt to A & X. You may solve A and will do for X the soonest I get the idea. Thanks!
    Attached Thumbnails Attached Thumbnails Gradient of a Euclidean function-cf-frob-norm.jpg  
    Follow Math Help Forum on Facebook and Google+

  4. #4
    MHF Contributor
    Joined
    Nov 2013
    From
    California
    Posts
    6,581
    Thanks
    2913

    Re: Gradient of a Euclidean function

    Quote Originally Posted by romsek View Post
    Do you think you could post the problem so we can see what it is.

    Looking at the image you posted I have no idea what you're trying to do.
    I think the issue is that you want

    $\dfrac 1 2 \sum (Y-AX)^\dagger (Y-AX)$ rather than $\dfrac 1 2 \sum (Y-AX)^2$ but I'm really not sure.
    Follow Math Help Forum on Facebook and Google+

  5. #5
    MHF Contributor
    Joined
    Sep 2012
    From
    Washington DC USA
    Posts
    1,061
    Thanks
    410

    Re: Gradient of a Euclidean function

    What's varying with what? What's a function of what? What variable are you taking the derivative with respect to?

    ##########################

    The main thing to watch out for when it comes to derivatives of matrices is that matrix multiplication is non-commutative.

    For example, if you have matrices A(t) and B(t) (meaning the components of the matrix are functions of t), then the product rule for matrices looks like this: (d/dt)[AB] = (dA/dt)B + A(dB/dt).

    Proof is the same as with the the ordinary product rule, but now you have to be careful to keep the order of the multiplication correct:

    A(t + h)B(t + h) - A(t)B(t) = A(t + h)B(t + h) - A(t)B(t + h) + A(t)B(t + h) - A(t)B(t) = [ A(t + h) - A(t) ] B(t + h) + A(t) [ B(t + h) - B(t) ]. Now divide both sides by h and take the limit as h goes to 0.

    In ordinary calculus you might derive the formula for the derivative of the reciprocal like this:

    $\displaystyle \frac{f(x)}{f(x)} = 1$, so $\displaystyle \frac{d}{dx} \left( \frac{f(x)}{f(x)} \right) = 0$,

    $\displaystyle \frac{d}{dx} \left\[ \left( f(x) \right) \left( \frac{1}{f(x)} \right) \right] = 0$, so

    $\displaystyle \left\[ \frac{d}{dx} f(x)\right\] \left( \frac{1}{f(x)} \right) + f(x) \left\[ \frac{d}{dx} \left( \frac{1}{f(x)} \right) \right\] = 0$, so

    $\displaystyle f(x) \frac{d}{dx} \left( \frac{1}{f(x)} \right) = - \frac{1}{f(x)} \frac{df}{dx}$, so

    $\displaystyle \frac{d}{dx} \left( \frac{1}{f(x)} \right) = - \frac{\left( \frac{df}{dx} \right)}{f(x)^2} $.

    From there, you could then apply the product rule to get the quotient rule.

    With matrices, you do the same "thing", except now you have to keep an eye on the order of multiplication for everything.

    So what's the derivative of the inverse of a matrix?

    $\displaystyle A^{-1}(t)A(t) = I$, so $\displaystyle \frac{d}{dt} \left( A^{-1}(t)A(t) \right) = \frac{d}{dt}(I) = 0$ (that's the 0 matrix).

    From the product rule already shown, you get:

    $\displaystyle \frac{d}{dt} \left( A^{-1}(t)A(t) \right) = \left\[ \frac{d}{dt} A^{-1}(t) \right\] A(t) + A^{-1}(t) \frac{d}{dt}A(t)$, so

    $\displaystyle \left\[ \frac{d}{dt} A^{-1}(t) \right\] A(t) + A^{-1}(t) \frac{d}{dt}A(t) = 0$, so

    $\displaystyle \left\[ \frac{d}{dt} A^{-1}(t) \right\] A(t) = - A^{-1}(t) \frac{d}{dt}A(t)$, so

    $\displaystyle \frac{d}{dt} A^{-1}(t) = - A^{-1}(t) \left\[ \frac{d}{dt}A(t) \right\] A^{-1}(t)$.
    Last edited by johnsomeone; Aug 16th 2015 at 09:50 AM.
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Newbie
    Joined
    Aug 2015
    From
    Philippines
    Posts
    7

    Re: Gradient of a Euclidean function

    Solved already. Thanks!
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Gradient of a function?
    Posted in the Calculus Forum
    Replies: 4
    Last Post: Apr 1st 2013, 08:59 AM
  2. Convex function of Euclidean distance
    Posted in the Advanced Applied Math Forum
    Replies: 0
    Last Post: Oct 23rd 2012, 08:24 AM
  3. Gradient of function
    Posted in the Algebra Forum
    Replies: 1
    Last Post: Apr 28th 2008, 11:44 PM
  4. Need some help (gradient function?)
    Posted in the Trigonometry Forum
    Replies: 2
    Last Post: Aug 28th 2007, 10:24 PM
  5. gradient function
    Posted in the Calculus Forum
    Replies: 1
    Last Post: Oct 8th 2006, 12:14 PM

Search Tags


/mathhelpforum @mathhelpforum