# Subspace Minimization

• Mar 21st 2008, 03:50 PM
syme.gabriel
Subspace Minimization
Let $\displaystyle f(x)$ be a $\displaystyle \mathcal{C}^1$ function and let $\displaystyle M$ be a subspace of $\displaystyle \mathbb{R}^n$. Suppose $\displaystyle x^\ast \in M$ minimizes $\displaystyle f(x)$ on $\displaystyle M$. Show $\displaystyle \nabla f(x^\ast) \in M^\perp$.

My only idea is to take any $\displaystyle x \in M$ and try to show something with the function $\displaystyle \varphi(t) = f(x^\ast + tx)$, but I haven't been able to get anywhere with that. Any help is greatly appreciated!
• Mar 23rd 2008, 12:55 AM
CaptainBlack
Quote:

Originally Posted by syme.gabriel
Let $\displaystyle f(x)$ be a $\displaystyle \mathcal{C}^1$ function and let $\displaystyle M$ be a subspace of $\displaystyle \mathbb{R}^n$. Suppose $\displaystyle x^\ast \in M$ minimizes $\displaystyle f(x)$ on $\displaystyle M$. Show $\displaystyle \nabla f(x^\ast) \in M^\perp$.

My only idea is to take any $\displaystyle x \in M$ and try to show something with the function $\displaystyle \varphi(t) = f(x^\ast + tx)$, but I haven't been able to get anywhere with that. Any help is greatly appreciated!

Suppose the gradient did not belong to the orthogonal complement of $\displaystyle M$.

Then the gradient at $\displaystyle \bold{x}^*$ has a component $\displaystyle \bold{u}$ in $\displaystyle M$, so moving form $\displaystyle \bold{x}^*$ to $\displaystyle \bold{x}^*+\delta \bold{\hat{u}}$
changes the value of the function to:

$\displaystyle f(\bold{x}^*)+ \delta [\nabla f(\bold{x}^*).\bold{\hat{u}}]=f(\bold{x}^*)+ \delta |\bold{u}|$,

so $\displaystyle \bold{x}^*$ will not be a mininmum in $\displaystyle M$.

RonL