Consider the function f(x,y) = x^2 * exp(y). Its Hessian is -2 * x^2 * exp(2 * y) which is negative for any non-zero x, while f is convex.

What is the story about Hessian being positive for convex functions?

Printable View

- August 16th 2010, 08:27 AMhtgNegative Hessian of a convex function
Consider the function f(x,y) = x^2 * exp(y). Its Hessian is -2 * x^2 * exp(2 * y) which is negative for any non-zero x, while f is convex.

What is the story about Hessian being positive for convex functions? - August 16th 2010, 09:23 AMAckbeet
Hello, htg.

Quote:

What is the story about Hessian being positive for convex functions?

- August 16th 2010, 09:33 AMhtg
d^2 f/dx^2 = 2*exp(y), d^2 f/(dx dy) =2*x*exp(y), d^2 f/ dy^2 = x^2 * exp(y)

so the Hessian is -2 * x^2 * exp(2 * y) - August 16th 2010, 09:35 AMAckbeet
Never mind. I buy your Hessian. Let me think a bit.

- August 16th 2010, 11:16 AMAckbeet
So, the wiki on convex functions says the following:

Quote:

More generally, a continuous, twice differentiable function of several variables is convex on a convex set if and only if its Hessian matrix is positive semidefinite on the interior of the convex set.

Your function certainly seems like it should be convex. However, it is not. The product of two convex functions is not necessarily convex. Counterexample: on the interval Both and are convex on that interval (indeed, for the whole real line).

If I were to produce a counterexample for your function, I would have to use two points whose x and y coordinates both differ, because obviously, is convex in for positive , and is convex in for positive . So if the function is not convex, the secret must lie with how the two variables interact.

Look at this plot of the function in Mathematica.

Attachment 18613

The flat regions really extend upward: this is Mathematica's way of chopping off the function. It's actually helpful in this circumstance. Here and You can see for yourself that if I take the two points and , and connect those two points with a line, the line would at some point pass under the graph of the function. Therefore, I claim this function is not convex. You would need to produce your own bona fide counterexample in order to provide a rigorous proof. But there it is.

Does this help? - August 16th 2010, 11:33 AMhtgAnother convex function with non-positive Hessian
I do not agree, x^2 * exp(y) is convex.

You can consider another example: g(x,y) = exp(x)*exp(y) = exp(x+y).

Its Hessian is identically zero. I am convinced that this is also a strictly convex function. - August 16th 2010, 11:47 AMAckbeetQuote:

I do not agree, x^2 * exp(y) is convex.

Even if your g(x,y) is convex, that would not prove that f(x,y) is convex. The product of two convex functions can be convex. Example: . I'm saying that the product of two convex functions is**not always convex**. I have given you an irrefutable example of that in post # 5.

Therefore, showing that f(x,y) is the product of two convex functions is not enough to prove that f(x,y) is convex! You have to do more than that. - August 16th 2010, 11:48 AMAckbeet
Incidentally, an identically zero Hessian means that the Hessian test fails. This is similar to the second derivative test, which similarly fails when it is zero. You don't know what you have in that case, and you have to do more work.

- August 16th 2010, 12:26 PMhtg
You would have to tell me which points in R^3 you want to connect. If these are points on the graph of f(x,y)=x^2*exp(y), then the segment will lie above the graph of f.

Tn the case of g(x,y)=exp(x+y) the Hessian is IDENTICALLY zero - it is like having the zecond derivative ALWAYS zero - then your function is linear.

I am waiting until you admit that the story with positive Hessian of a convex function is a lie. - August 16th 2010, 12:44 PMAckbeet
Connect the two points and The line segment connecting the two is given by

for Pick Then the line segment at that point has the vector

The -coordinate is approximately .

We can read off the and coordinates corresponding to as and . Therefore, we compute

The line segment has a -value less than the function value at the point corresponding to . Therefore, the function is not convex. QED. - August 16th 2010, 01:07 PMAckbeet
Incidentally, you were correct that my counter-example in post # 5 does not work. You can't choose the origin as one of the points. But my counterexample in post # 10 does work.

- August 16th 2010, 01:39 PMhtg
I think it does work, but the Hessian of f is negative semidefinite for all x,y , so f should be concave at least for all (x,y) except possibly (0,y). It is not.

- August 16th 2010, 03:08 PMDefunktQuote:

Originally Posted by**htg**

- August 17th 2010, 01:32 AMhtg
Hessian of a function of 2 variables does not change sign when we change the sign of the function.

- August 17th 2010, 01:45 AMAckbeet
Which is why, when applying the multi-dimensional 2nd Derivative Test, you also have to test at your points, if you're doing extrema-finding. does change sign when the function changes sign. Wiki doesn't mention this on the convexity page, but this might also be a requirement when testing for convexity.

Is there any confusion remaining?