Are you asserting that every point on the graph of a convex or concave function must be a local extremum?
If not, the fact that a saddle point is not a local extremum does not immediately imply that it cannot be a saddle point.
My question is:
Can a convex/concave function have a saddle point?
My answer would be:
Convex and concave function do not have saddle points, because a saddle point is not a local extremum.
Is this answer correct? How could I explain it better?
Are you asserting that every point on the graph of a convex or concave function must be a local extremum?
If not, the fact that a saddle point is not a local extremum does not immediately imply that it cannot be a saddle point.
Neither convex or concave functions can have saddle points.
Let's show this for convex functions, then a similar argument can be used for concave. First, we need some definitions.
1. denotes the -neighborhood of x.
2. Given a Frechet differentiable where X is a Hilbert space (which includes ), a stationary point is a point x where . I'm sure we could do this with Gatteaux (directional) derivatives, but it's simpler to assume we have a gradient.
3. A saddle point is a stationary point that is neither a local min or a local max.
4. Given , a local min is a point x such that there exists an -neighborhood such that for all .
5. A convex function is a function where for all and ,
Now, let us assume that f is Frechet differentiable and convex. Let us also assume that there exists a stationary point x that is a saddle point. Since x is a saddle point, x is not a local min. That means that for every -neighborhood of x there exists such that . Next, from convexity we have that
for . Then, since , we have that
Rearranging terms, we have
Taking the limit as , we have that
where denotes the inner product on X. In , we typically have that . In any case, since x is stationary, . Hene, we reduce the above inequality into
This contradicts the assumption that x is not a local min. Hence, if f is convex and x is stationary, then x is a local, and in fact global, min.