1. ## Locally convex set

Considering a constrained nonlinear programming (NLP) problem
$\displaystyle min \quad f({\bf x}) \quad {\bf x}\in \mathbb{R}^{n}$
$\displaystyle s.t. \quad g_{i}({\bf x})\leq 0 \quad i=1,2,...,N$
$\displaystyle \quad\quad\quad\quad h_{j}({\bf x})=0 \quad j=1,2,...,M$

Where $\displaystyle g_{i}({\bf x})$ and $\displaystyle h_{j}({\bf x})$ is twice continuously differentiable. The feasible region $\displaystyle S=\{{\bf x}|g_{i}({\bf x}),h_{j}({\bf x}),\forall i,j\}$. It is known that if $\displaystyle g_{i}({\bf x})$ is convex and $\displaystyle h_{j}({\bf x})$ is affinely linear for $\displaystyle {\bf x}\in \mathbb{R}^{n}$, $\displaystyle S$ is a convex set. However, in my problem, $\displaystyle g_{i}({\bf x})$ and $\displaystyle h_{j}({\bf x})$ is indefinite for $\displaystyle {\bf x}\in \mathbb{R}^{n}$. So I would like to ask if there is any theory may answer the following two questions:

(1)For any twice continuously differentiable but indefinite function $\displaystyle g_{i}({\bf x})$, on what condition, $\displaystyle g_{i}({\bf x})$ is convex in a neighborhood of a point $\displaystyle {\bf x_{0}}\in\mathbb{R}^{n}$ ? (A guess is that Hessian of $\displaystyle g_{i}$ at $\displaystyle {\bf x_{0}}$ is positive semidefinite. Is that the case?)

Just like the image above. The function is indefinite for all $\displaystyle x$, but is locally convex in the neighborhood of $\displaystyle x_{0}$, which is $\displaystyle (x_{1},x_{2})$.

(2)On what condition, a neighborhood in $\displaystyle S$ of a feasible point $\displaystyle {\bf x_{0}}\in S$ is a convex set? (I suppose a sufficient condition is that every $\displaystyle g_{i}({\bf x})$ and $\displaystyle h_{j}({\bf x})$ is convex in a neighborhood of $\displaystyle {\bf x_{0}}$. But is that necessary?)

Just like the image above. The set $\displaystyle S$ is not convex, but is locally convex in the neighborhood of $\displaystyle {\bf x_{0}}$ (the red triangle set).

2. ## Re: Locally convex set

The questions can also be put like the following. I believe these three questions make it easier for you to answer exactly.

(1) Given a twice continuously differentiable function $\displaystyle f(x),x\in\mathbb{R}$, it can be justified that $\displaystyle f''(x)$ is not always positive for $\displaystyle \forall x\in\mathbb{R}$. However, if $\displaystyle f''(x_0)>0$, is $\displaystyle f(x)$ ("locally") convex in some epsilon distance around $\displaystyle x_0$? (As shown in the 1st picutre in #1)

(2) Given a twice continously differentiable function $\displaystyle f({\bf x}),{\bf x}\in\mathbb{R}^{n}$, it can be justified that Hessian Matrix of $\displaystyle f({\bf x})$ is not always postive definite for $\displaystyle \forall x\in\mathbb{R}^{n}$. However, if Hessian of $\displaystyle f({\bf x})$ at $\displaystyle {\bf x_0}$ is positve definite, is $\displaystyle f({\bf x})$ ("locally") convex in some epsilon neighborhood of $\displaystyle {\bf x_0}$?

(3) Given a region $\displaystyle S$ defined by $\displaystyle g_{i}({\bf x})\leq 0 \quad i=1,2,...,N$ and $\displaystyle h_{j}({\bf x})=0 \quad j=1,2,...,M$ and $\displaystyle {\bf x}\in\mathbb{R}^{n}$ (usually $\displaystyle S$ defines the feasible region of a general constrained optimization problem), where every $\displaystyle g_{i}({\bf x})$ and $\displaystyle h_{j}({\bf x})$ is twice continously differentiable. Here $\displaystyle g_{i}({\bf x})$ is not convex for $\displaystyle \forall {\bf x}\in\mathbb{R}^{n}$, $\displaystyle h_{j}({\bf x})$ is not affinely linear, so $\displaystyle S$ is not a convex set "as a whole". But for a feasible point $\displaystyle {\bf x_0}\in S$, on what condition (I would like to know condition about $\displaystyle g_{i}({\bf x})$ and $\displaystyle h_{j}({\bf x})$, not just the "at + (1-t)b" definition of convexity set), a neighborhood of $\displaystyle {\bf x_0}$ in $\displaystyle S$ is ("locally") convex? (As shown in the 2nd picture in #1)
As to this question, if this kind of condition exists, Hessian of $\displaystyle g_{i}({\bf x_0})$ and $\displaystyle h_{j}({\bf x_0})$ is probably involved, as I guessed.

Thanks very much!