I need help computing the gradient of a function.
Where y and b are vectors, and X is a matrix.
I need help to find
Is this just a standard directional derivative with respect to some vector?
If so it's going to have some dot product and hence some matrix expansion in terms of b and the normal derivative of that matrix expression.
I can't recall the identities off the top of my head, but if you do a search they should be out there.
There are severals ways to find , because there are several ways to interpret it. I'll show you a kinda weird way, but one that's often very useful:
Think of a curve such that and .
Now, although there are infinitely many, is the natural choice for g. However, we won't even need that.
So , and since that holds for EVERY , have: