# Thread: Negative Hessian of a convex function

1. ## Negative Hessian of a convex function

Consider the function f(x,y) = x^2 * exp(y). Its Hessian is -2 * x^2 * exp(2 * y) which is negative for any non-zero x, while f is convex.
What is the story about Hessian being positive for convex functions?

2. Hello, htg.

What is the story about Hessian being positive for convex functions?
I believe it is. I'm not sure I buy your Hessian, though. Can you show us your computations?

3. d^2 f/dx^2 = 2*exp(y), d^2 f/(dx dy) =2*x*exp(y), d^2 f/ dy^2 = x^2 * exp(y)
so the Hessian is -2 * x^2 * exp(2 * y)

4. Never mind. I buy your Hessian. Let me think a bit.

5. So, the wiki on convex functions says the following:

More generally, a continuous, twice differentiable function of several variables is convex on a convex set if and only if its Hessian matrix is positive semidefinite on the interior of the convex set.
Based on this statement, your function f is not convex. The Hessian is negative semidefinite.

Your function certainly seems like it should be convex. However, it is not. The product of two convex functions is not necessarily convex. Counterexample: $\displaystyle f(x)=x^{2}(x-1)^{2}$ on the interval $\displaystyle (-1,2).$ Both $\displaystyle x^{2}$ and $\displaystyle (x-1)^{2}$ are convex on that interval (indeed, for the whole real line).

If I were to produce a counterexample for your function, I would have to use two points whose x and y coordinates both differ, because obviously, $\displaystyle k x^{2}$ is convex in $\displaystyle x$ for positive $\displaystyle k$, and $\displaystyle p e^{y}$ is convex in $\displaystyle y$ for positive $\displaystyle p$. So if the function $\displaystyle f$ is not convex, the secret must lie with how the two variables interact.

Look at this plot of the function in Mathematica.

The flat regions really extend upward: this is Mathematica's way of chopping off the function. It's actually helpful in this circumstance. Here $\displaystyle x\in[-10,10],$ and $\displaystyle y\in[0,10].$ You can see for yourself that if I take the two points $\displaystyle (0,0)$ and $\displaystyle (10,6)$, and connect those two points with a line, the line would at some point pass under the graph of the function. Therefore, I claim this function is not convex. You would need to produce your own bona fide counterexample in order to provide a rigorous proof. But there it is.

Does this help?

6. ## Another convex function with non-positive Hessian

I do not agree, x^2 * exp(y) is convex.
You can consider another example: g(x,y) = exp(x)*exp(y) = exp(x+y).
Its Hessian is identically zero. I am convinced that this is also a strictly convex function.

7. I do not agree, x^2 * exp(y) is convex.
Ok. Then show me why my counterexample doesn't work (why it doesn't show that the function is not convex).

Even if your g(x,y) is convex, that would not prove that f(x,y) is convex. The product of two convex functions can be convex. Example: $\displaystyle f(x)=x^{2}x^{2}$. I'm saying that the product of two convex functions is not always convex. I have given you an irrefutable example of that in post # 5.

Therefore, showing that f(x,y) is the product of two convex functions is not enough to prove that f(x,y) is convex! You have to do more than that.

8. Incidentally, an identically zero Hessian means that the Hessian test fails. This is similar to the second derivative test, which similarly fails when it is zero. You don't know what you have in that case, and you have to do more work.

9. You would have to tell me which points in R^3 you want to connect. If these are points on the graph of f(x,y)=x^2*exp(y), then the segment will lie above the graph of f.
Tn the case of g(x,y)=exp(x+y) the Hessian is IDENTICALLY zero - it is like having the zecond derivative ALWAYS zero - then your function is linear.
I am waiting until you admit that the story with positive Hessian of a convex function is a lie.

10. Connect the two points $\displaystyle (1,10,f(1,10))$ and $\displaystyle (10,6,f(10,6)).$ The line segment connecting the two is given by

$\displaystyle \vec{r}(t)=t\begin{bmatrix}1\\10\\e^{10}\end{bmatr ix}+(1-t)\begin{bmatrix}10\\6\\100 e^{6}\end{bmatrix},$

for $\displaystyle t\in(0,1).$ Pick $\displaystyle t=0.5.$ Then the line segment at that point has the vector

$\displaystyle \vec{r}(0.5)=0.5\begin{bmatrix}1\\10\\e^{10}\end{b matrix}+0.5\begin{bmatrix}10\\6\\100 e^{6}\end{bmatrix}=\begin{bmatrix}0.5\\5\\0.5 e^{10}\end{bmatrix}+\begin{bmatrix}5\\3\\50 e^{6}\end{bmatrix}=\begin{bmatrix}5.5\\8\\0.5e^{10 }+50 e^{6}\end{bmatrix}.$

The $\displaystyle z$-coordinate is approximately $\displaystyle 31184.67$.

We can read off the $\displaystyle x$ and $\displaystyle y$ coordinates corresponding to $\displaystyle t=0.5$ as $\displaystyle x=5.5$ and $\displaystyle y=8$. Therefore, we compute

$\displaystyle f(5.5,8)=(5.5)^{2}e^{8}\approx 90173.98.$

The line segment has a $\displaystyle z$-value less than the function value at the point corresponding to $\displaystyle t=0.5$. Therefore, the function is not convex. QED.

11. Incidentally, you were correct that my counter-example in post # 5 does not work. You can't choose the origin as one of the points. But my counterexample in post # 10 does work.

12. I think it does work, but the Hessian of f is negative semidefinite for all x,y , so f should be concave at least for all (x,y) except possibly (0,y). It is not.

13. Originally Posted by htg
I am waiting until you admit that the story with positive Hessian of a convex function is a lie.
This is not the type of attitude you want to display if you're interested in someone answering your questions.

14. Hessian of a function of 2 variables does not change sign when we change the sign of the function.

15. Which is why, when applying the multi-dimensional 2nd Derivative Test, you also have to test $\displaystyle f_{xx}$ at your points, if you're doing extrema-finding. $\displaystyle f_{xx}$ does change sign when the function changes sign. Wiki doesn't mention this on the convexity page, but this might also be a requirement when testing for convexity.

Is there any confusion remaining?

Page 1 of 2 12 Last