steepest descent

**AMR** · August 8th 2006, 05:40 AM

how can we proof this :
direction of steepest descent of a differentiable function F of M variables is the vector

( $-cdF/dw_1,...,-cdF/dw_M$ )

where c is a positive constant of proportionality.

thanks

**ThePerfectHacker** · August 8th 2006, 07:41 AM

Originally Posted by AMR

how can we proof this :
direction of steepest descent of a differentiable function F of M variables is the vector

( $-cdF/dw_1,...,-cdF/dw_M$ )

where c is a positive constant of proportionality.

thanks

Because that is the gradient of a diffrenciable function.
---
I can prove it for $f(x,y)$ .
Let $\nabla f_0\not = 0$ at a point.
Let, $\bold{u}$ be any unit vector.
Then, directional derivative of,
$D_u f=\nabla f_0 \cdot \bold{u} = ||\nabla f_0||\cdot ||\bold{u}||\cos \theta$
The largest possible value of,
$\cos \theta =1$
Thus, maximum is,
$||\nabla f_0||(1)$
Thus, it is in the in the direction of the gradient. Thus, it is a multiple of the vector, hence the constant of proportionality.
---
I am thinking about several variables.
I guess the proof is based on finding the critical points. Since the critical points are determined by partial derivatives they appear over here. I will think more about it.

**ThePerfectHacker** · August 8th 2006, 10:15 AM

You have a function,
$f:\mathbb{R}^n\to \mathbb{R}$ .
At point, $P(x_1,x_2,...,x_n)$
You have, $\nabla f\not = \bold{0}$ .
Consider, a unit vector,
$\bold{u}=<u_1,...,u_n>$ and, $\sum_{k=1}^n u_k^2 = 1$

You need to find, what the components of $\bold{u}$ such as, $D_u f$ (directional derivative along this vector) is extremized.
But,
$D_u f =\nabla f \cdot \bold{u}$
Thus, you want to maximize,
$g(u_1,...,u_n)=\nabla f\cdot \bold{u}$
Thus,
$g(u_1,...,u_n)=\sum_{k=1}^n \frac{\partial f}{\partial x_k}u_k$
With constraint curve,
$c(u_1,...u_n)=\sum_{k=1}^n u_k^2=1$
Thus, Lagrange Multipliers,
$\nabla g=\kappa \nabla c$
Thus,
$\left< \frac{\partial f}{\partial x_1},...,\frac{\partial f}{\partial x_n} \right>=\left<2\kappa u_1,...,2\kappa u_n\right>$
Since,
$\nabla f\not = \bold{0}$ thus, $\kappa \not = 0$
Thus,
$\bold{u}=\frac{1}{2\kappa}\left< \frac{\partial f}{\partial x_1},...,\frac{\partial f}{\partial x_n} \right>=\frac{1}{2\kappa} \nabla f$
Thus, we see that every extremized directional derivative is a multiple of the gradient of the function.

---
Note, I used Lagrange Multipliers. The problem might be that the prove of Lagrange Multipliers for multi-variables more than 2 is based on the fact of the max/min property of the gradient. Thus, my proof was not a proof because it uses a fact that depends on itself.

**AMR** · August 8th 2006, 11:34 AM

thanks alot

Thread: steepest descent

LinkBack

Thread Tools

Display

steepest descent

Similar Math Help Forum Discussions

Constrained Steepest descent method - help in constraining it...

Parameter Optimization - maximise via steepest descent?

Path of steepest descent

Gradient - Steepest Descent

Directional Derivative in the direction of steepest descent.

Search Tags