# steepest descent

• Aug 8th 2006, 06:40 AM
AMR
steepest descent
how can we proof this :
direction of steepest descent of a differentiable function F of M variables is the vector

( $-cdF/dw_1,...,-cdF/dw_M$)

where c is a positive constant of proportionality.

thanks
• Aug 8th 2006, 08:41 AM
ThePerfectHacker
Quote:

Originally Posted by AMR
how can we proof this :
direction of steepest descent of a differentiable function F of M variables is the vector

( $-cdF/dw_1,...,-cdF/dw_M$)

where c is a positive constant of proportionality.

thanks

Because that is the gradient of a diffrenciable function.
---
I can prove it for $f(x,y)$.
Let $\nabla f_0\not = 0$ at a point.
Let, $\bold{u}$ be any unit vector.
Then, directional derivative of,
$D_u f=\nabla f_0 \cdot \bold{u} = ||\nabla f_0||\cdot ||\bold{u}||\cos \theta$
The largest possible value of,
$\cos \theta =1$
Thus, maximum is,
$||\nabla f_0||(1)$
Thus, it is in the in the direction of the gradient. Thus, it is a multiple of the vector, hence the constant of proportionality.
---
I am thinking about several variables.
I guess the proof is based on finding the critical points. Since the critical points are determined by partial derivatives they appear over here. I will think more about it.
• Aug 8th 2006, 11:15 AM
ThePerfectHacker
You have a function,
$f:\mathbb{R}^n\to \mathbb{R}$.
At point, $P(x_1,x_2,...,x_n)$
You have, $\nabla f\not = \bold{0}$.
Consider, a unit vector,
$\bold{u}=$ and, $\sum_{k=1}^n u_k^2 = 1$

You need to find, what the components of $\bold{u}$ such as, $D_u f$ (directional derivative along this vector) is extremized.
But,
$D_u f =\nabla f \cdot \bold{u}$
Thus, you want to maximize,
$g(u_1,...,u_n)=\nabla f\cdot \bold{u}$
Thus,
$g(u_1,...,u_n)=\sum_{k=1}^n \frac{\partial f}{\partial x_k}u_k$
With constraint curve,
$c(u_1,...u_n)=\sum_{k=1}^n u_k^2=1$
Thus, Lagrange Multipliers,
$\nabla g=\kappa \nabla c$
Thus,
$\left< \frac{\partial f}{\partial x_1},...,\frac{\partial f}{\partial x_n} \right>=\left<2\kappa u_1,...,2\kappa u_n\right>$
Since,
$\nabla f\not = \bold{0}$ thus, $\kappa \not = 0$
Thus,
$\bold{u}=\frac{1}{2\kappa}\left< \frac{\partial f}{\partial x_1},...,\frac{\partial f}{\partial x_n} \right>=\frac{1}{2\kappa} \nabla f$
Thus, we see that every extremized directional derivative is a multiple of the gradient of the function.

---
Note, I used Lagrange Multipliers. The problem might be that the prove of Lagrange Multipliers for multi-variables more than 2 is based on the fact of the max/min property of the gradient. Thus, my proof was not a proof because it uses a fact that depends on itself.
• Aug 8th 2006, 12:34 PM
AMR
thanks alot :D