f(x) = x' * E * x
x' = x transpose
E = a covariance matrix
The problem is: prove that f(x) is convex. The solution has the equations:
gradient of f = 2 * E * x <- How is the gradient calculated here from f(x)?
gradient^2 of f = 2 * E
E > 0 since it is a covariance matrix, therefore f is convex
I want to learn how to calculate the gradient from the function f, the answer is given but I don't know how it was determined.
By quadratic, do you mean if we have a 2x2 matrix:
top left represents x^2
top right represents x*y
bottom left represents y*x
bottom right represents y^2
So then we can build a quadratic equation from that with x and y variables, then we simply find the gradient of that function?
I still don't understand how you can simply go from x' * E * x to grad(f) = 2*E*x
I expanded it out, here is my work:
http://i.imgur.com/iLcaS.jpg
I have one resulting quadratic equation, do I find the gradient of that?
http://i.imgur.com/69Lgj.jpg
After expanding x'Ex, to get the gradient, I would derive with respect to x1, and then derive with respect to x2, resulting in 2*(e12), but my solution says that the gradient is 2*E*x, which is not equal to 2*(e12).
Is my understanding incorrect?
I see:
http://i.imgur.com/7nQ6z.jpg
Now I have a few additional questions. The gradient is a vector, that represents the direction of greatest slope increase, so I have two elements in my vector, do the elements represent the (x1 or x2) coordinates or the (i or j) coordinates?
Also, from the solution to this problem, it appears that the problem takes the following steps:
f= x'Ex
grad f = 2Ex
grad^2 f = 2E
Is there a rule with matrices where I can simply deduce the gradient without expanding everything out? I feel, now that I understand the underlying concept (through expansion) I should be able to use shortcuts to quicken my problem solving skills. THanks