Page 1 of 2 12 LastLast
Results 1 to 15 of 17

Math Help - Negative Hessian of a convex function

  1. #1
    htg
    htg is offline
    Newbie
    Joined
    Aug 2010
    Posts
    7

    Negative Hessian of a convex function

    Consider the function f(x,y) = x^2 * exp(y). Its Hessian is -2 * x^2 * exp(2 * y) which is negative for any non-zero x, while f is convex.
    What is the story about Hessian being positive for convex functions?
    Follow Math Help Forum on Facebook and Google+

  2. #2
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2
    Hello, htg.

    What is the story about Hessian being positive for convex functions?
    I believe it is. I'm not sure I buy your Hessian, though. Can you show us your computations?
    Follow Math Help Forum on Facebook and Google+

  3. #3
    htg
    htg is offline
    Newbie
    Joined
    Aug 2010
    Posts
    7
    d^2 f/dx^2 = 2*exp(y), d^2 f/(dx dy) =2*x*exp(y), d^2 f/ dy^2 = x^2 * exp(y)
    so the Hessian is -2 * x^2 * exp(2 * y)
    Follow Math Help Forum on Facebook and Google+

  4. #4
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2
    Never mind. I buy your Hessian. Let me think a bit.
    Follow Math Help Forum on Facebook and Google+

  5. #5
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2
    So, the wiki on convex functions says the following:

    More generally, a continuous, twice differentiable function of several variables is convex on a convex set if and only if its Hessian matrix is positive semidefinite on the interior of the convex set.
    Based on this statement, your function f is not convex. The Hessian is negative semidefinite.

    Your function certainly seems like it should be convex. However, it is not. The product of two convex functions is not necessarily convex. Counterexample: f(x)=x^{2}(x-1)^{2} on the interval (-1,2). Both x^{2} and (x-1)^{2} are convex on that interval (indeed, for the whole real line).

    If I were to produce a counterexample for your function, I would have to use two points whose x and y coordinates both differ, because obviously, k x^{2} is convex in x for positive k, and p e^{y} is convex in y for positive p. So if the function f is not convex, the secret must lie with how the two variables interact.

    Look at this plot of the function in Mathematica.

    Negative Hessian of a convex function-x-2-e-y.jpg

    The flat regions really extend upward: this is Mathematica's way of chopping off the function. It's actually helpful in this circumstance. Here x\in[-10,10], and y\in[0,10]. You can see for yourself that if I take the two points (0,0) and (10,6), and connect those two points with a line, the line would at some point pass under the graph of the function. Therefore, I claim this function is not convex. You would need to produce your own bona fide counterexample in order to provide a rigorous proof. But there it is.

    Does this help?
    Follow Math Help Forum on Facebook and Google+

  6. #6
    htg
    htg is offline
    Newbie
    Joined
    Aug 2010
    Posts
    7

    Another convex function with non-positive Hessian

    I do not agree, x^2 * exp(y) is convex.
    You can consider another example: g(x,y) = exp(x)*exp(y) = exp(x+y).
    Its Hessian is identically zero. I am convinced that this is also a strictly convex function.
    Follow Math Help Forum on Facebook and Google+

  7. #7
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2
    I do not agree, x^2 * exp(y) is convex.
    Ok. Then show me why my counterexample doesn't work (why it doesn't show that the function is not convex).

    Even if your g(x,y) is convex, that would not prove that f(x,y) is convex. The product of two convex functions can be convex. Example: f(x)=x^{2}x^{2}. I'm saying that the product of two convex functions is not always convex. I have given you an irrefutable example of that in post # 5.

    Therefore, showing that f(x,y) is the product of two convex functions is not enough to prove that f(x,y) is convex! You have to do more than that.
    Follow Math Help Forum on Facebook and Google+

  8. #8
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2
    Incidentally, an identically zero Hessian means that the Hessian test fails. This is similar to the second derivative test, which similarly fails when it is zero. You don't know what you have in that case, and you have to do more work.
    Follow Math Help Forum on Facebook and Google+

  9. #9
    htg
    htg is offline
    Newbie
    Joined
    Aug 2010
    Posts
    7
    You would have to tell me which points in R^3 you want to connect. If these are points on the graph of f(x,y)=x^2*exp(y), then the segment will lie above the graph of f.
    Tn the case of g(x,y)=exp(x+y) the Hessian is IDENTICALLY zero - it is like having the zecond derivative ALWAYS zero - then your function is linear.
    I am waiting until you admit that the story with positive Hessian of a convex function is a lie.
    Follow Math Help Forum on Facebook and Google+

  10. #10
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2
    Connect the two points (1,10,f(1,10)) and (10,6,f(10,6)). The line segment connecting the two is given by

    \vec{r}(t)=t\begin{bmatrix}1\\10\\e^{10}\end{bmatr  ix}+(1-t)\begin{bmatrix}10\\6\\100 e^{6}\end{bmatrix},

    for t\in(0,1). Pick t=0.5. Then the line segment at that point has the vector

    \vec{r}(0.5)=0.5\begin{bmatrix}1\\10\\e^{10}\end{b  matrix}+0.5\begin{bmatrix}10\\6\\100 e^{6}\end{bmatrix}=\begin{bmatrix}0.5\\5\\0.5 e^{10}\end{bmatrix}+\begin{bmatrix}5\\3\\50 e^{6}\end{bmatrix}=\begin{bmatrix}5.5\\8\\0.5e^{10  }+50 e^{6}\end{bmatrix}.

    The z-coordinate is approximately 31184.67.

    We can read off the x and y coordinates corresponding to t=0.5 as x=5.5 and y=8. Therefore, we compute

    f(5.5,8)=(5.5)^{2}e^{8}\approx 90173.98.

    The line segment has a z-value less than the function value at the point corresponding to t=0.5. Therefore, the function is not convex. QED.
    Follow Math Help Forum on Facebook and Google+

  11. #11
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2
    Incidentally, you were correct that my counter-example in post # 5 does not work. You can't choose the origin as one of the points. But my counterexample in post # 10 does work.
    Follow Math Help Forum on Facebook and Google+

  12. #12
    htg
    htg is offline
    Newbie
    Joined
    Aug 2010
    Posts
    7
    I think it does work, but the Hessian of f is negative semidefinite for all x,y , so f should be concave at least for all (x,y) except possibly (0,y). It is not.
    Follow Math Help Forum on Facebook and Google+

  13. #13
    Super Member
    Joined
    Aug 2009
    From
    Israel
    Posts
    976
    Quote Originally Posted by htg
    I am waiting until you admit that the story with positive Hessian of a convex function is a lie.
    This is not the type of attitude you want to display if you're interested in someone answering your questions.
    Follow Math Help Forum on Facebook and Google+

  14. #14
    htg
    htg is offline
    Newbie
    Joined
    Aug 2010
    Posts
    7
    Hessian of a function of 2 variables does not change sign when we change the sign of the function.
    Follow Math Help Forum on Facebook and Google+

  15. #15
    A Plied Mathematician
    Joined
    Jun 2010
    From
    CT, USA
    Posts
    6,318
    Thanks
    4
    Awards
    2
    Which is why, when applying the multi-dimensional 2nd Derivative Test, you also have to test f_{xx} at your points, if you're doing extrema-finding. f_{xx} does change sign when the function changes sign. Wiki doesn't mention this on the convexity page, but this might also be a requirement when testing for convexity.

    Is there any confusion remaining?
    Follow Math Help Forum on Facebook and Google+

Page 1 of 2 12 LastLast

Similar Math Help Forum Discussions

  1. Hessian matrix for 2d function
    Posted in the Differential Geometry Forum
    Replies: 2
    Last Post: September 27th 2009, 12:24 PM
  2. Replies: 0
    Last Post: February 6th 2009, 03:06 PM
  3. Replies: 1
    Last Post: December 1st 2008, 05:34 AM
  4. convex due to nonnegative hessian proof
    Posted in the Calculus Forum
    Replies: 0
    Last Post: February 24th 2008, 09:50 PM
  5. A function and a Hessian
    Posted in the Calculus Forum
    Replies: 2
    Last Post: November 20th 2007, 09:15 PM

Search Tags


/mathhelpforum @mathhelpforum