Consider the function f(x,y) = x^2 * exp(y). Its Hessian is -2 * x^2 * exp(2 * y) which is negative for any non-zero x, while f is convex.

What is the story about Hessian being positive for convex functions?

Printable View

- Aug 16th 2010, 08:27 AMhtgNegative Hessian of a convex function
Consider the function f(x,y) = x^2 * exp(y). Its Hessian is -2 * x^2 * exp(2 * y) which is negative for any non-zero x, while f is convex.

What is the story about Hessian being positive for convex functions? - Aug 16th 2010, 09:23 AMAckbeet
Hello, htg.

Quote:

What is the story about Hessian being positive for convex functions?

- Aug 16th 2010, 09:33 AMhtg
d^2 f/dx^2 = 2*exp(y), d^2 f/(dx dy) =2*x*exp(y), d^2 f/ dy^2 = x^2 * exp(y)

so the Hessian is -2 * x^2 * exp(2 * y) - Aug 16th 2010, 09:35 AMAckbeet
Never mind. I buy your Hessian. Let me think a bit.

- Aug 16th 2010, 11:16 AMAckbeet
So, the wiki on convex functions says the following:

Quote:

More generally, a continuous, twice differentiable function of several variables is convex on a convex set if and only if its Hessian matrix is positive semidefinite on the interior of the convex set.

Your function certainly seems like it should be convex. However, it is not. The product of two convex functions is not necessarily convex. Counterexample: $\displaystyle f(x)=x^{2}(x-1)^{2}$ on the interval $\displaystyle (-1,2).$ Both $\displaystyle x^{2}$ and $\displaystyle (x-1)^{2}$ are convex on that interval (indeed, for the whole real line).

If I were to produce a counterexample for your function, I would have to use two points whose x and y coordinates both differ, because obviously, $\displaystyle k x^{2}$ is convex in $\displaystyle x$ for positive $\displaystyle k$, and $\displaystyle p e^{y}$ is convex in $\displaystyle y$ for positive $\displaystyle p$. So if the function $\displaystyle f$ is not convex, the secret must lie with how the two variables interact.

Look at this plot of the function in Mathematica.

Attachment 18613

The flat regions really extend upward: this is Mathematica's way of chopping off the function. It's actually helpful in this circumstance. Here $\displaystyle x\in[-10,10],$ and $\displaystyle y\in[0,10].$ You can see for yourself that if I take the two points $\displaystyle (0,0)$ and $\displaystyle (10,6)$, and connect those two points with a line, the line would at some point pass under the graph of the function. Therefore, I claim this function is not convex. You would need to produce your own bona fide counterexample in order to provide a rigorous proof. But there it is.

Does this help? - Aug 16th 2010, 11:33 AMhtgAnother convex function with non-positive Hessian
I do not agree, x^2 * exp(y) is convex.

You can consider another example: g(x,y) = exp(x)*exp(y) = exp(x+y).

Its Hessian is identically zero. I am convinced that this is also a strictly convex function. - Aug 16th 2010, 11:47 AMAckbeetQuote:

I do not agree, x^2 * exp(y) is convex.

Even if your g(x,y) is convex, that would not prove that f(x,y) is convex. The product of two convex functions can be convex. Example: $\displaystyle f(x)=x^{2}x^{2}$. I'm saying that the product of two convex functions is**not always convex**. I have given you an irrefutable example of that in post # 5.

Therefore, showing that f(x,y) is the product of two convex functions is not enough to prove that f(x,y) is convex! You have to do more than that. - Aug 16th 2010, 11:48 AMAckbeet
Incidentally, an identically zero Hessian means that the Hessian test fails. This is similar to the second derivative test, which similarly fails when it is zero. You don't know what you have in that case, and you have to do more work.

- Aug 16th 2010, 12:26 PMhtg
You would have to tell me which points in R^3 you want to connect. If these are points on the graph of f(x,y)=x^2*exp(y), then the segment will lie above the graph of f.

Tn the case of g(x,y)=exp(x+y) the Hessian is IDENTICALLY zero - it is like having the zecond derivative ALWAYS zero - then your function is linear.

I am waiting until you admit that the story with positive Hessian of a convex function is a lie. - Aug 16th 2010, 12:44 PMAckbeet
Connect the two points $\displaystyle (1,10,f(1,10))$ and $\displaystyle (10,6,f(10,6)).$ The line segment connecting the two is given by

$\displaystyle \vec{r}(t)=t\begin{bmatrix}1\\10\\e^{10}\end{bmatr ix}+(1-t)\begin{bmatrix}10\\6\\100 e^{6}\end{bmatrix},$

for $\displaystyle t\in(0,1).$ Pick $\displaystyle t=0.5.$ Then the line segment at that point has the vector

$\displaystyle \vec{r}(0.5)=0.5\begin{bmatrix}1\\10\\e^{10}\end{b matrix}+0.5\begin{bmatrix}10\\6\\100 e^{6}\end{bmatrix}=\begin{bmatrix}0.5\\5\\0.5 e^{10}\end{bmatrix}+\begin{bmatrix}5\\3\\50 e^{6}\end{bmatrix}=\begin{bmatrix}5.5\\8\\0.5e^{10 }+50 e^{6}\end{bmatrix}.$

The $\displaystyle z$-coordinate is approximately $\displaystyle 31184.67$.

We can read off the $\displaystyle x$ and $\displaystyle y$ coordinates corresponding to $\displaystyle t=0.5$ as $\displaystyle x=5.5$ and $\displaystyle y=8$. Therefore, we compute

$\displaystyle f(5.5,8)=(5.5)^{2}e^{8}\approx 90173.98.$

The line segment has a $\displaystyle z$-value less than the function value at the point corresponding to $\displaystyle t=0.5$. Therefore, the function is not convex. QED. - Aug 16th 2010, 01:07 PMAckbeet
Incidentally, you were correct that my counter-example in post # 5 does not work. You can't choose the origin as one of the points. But my counterexample in post # 10 does work.

- Aug 16th 2010, 01:39 PMhtg
I think it does work, but the Hessian of f is negative semidefinite for all x,y , so f should be concave at least for all (x,y) except possibly (0,y). It is not.

- Aug 16th 2010, 03:08 PMDefunktQuote:

Originally Posted by**htg**

- Aug 17th 2010, 01:32 AMhtg
Hessian of a function of 2 variables does not change sign when we change the sign of the function.

- Aug 17th 2010, 01:45 AMAckbeet
Which is why, when applying the multi-dimensional 2nd Derivative Test, you also have to test $\displaystyle f_{xx}$ at your points, if you're doing extrema-finding. $\displaystyle f_{xx}$ does change sign when the function changes sign. Wiki doesn't mention this on the convexity page, but this might also be a requirement when testing for convexity.

Is there any confusion remaining?