Hi,

my multivariate statistics course strikes again with unproven linear algebra results!

This time, there's a calculation in which I have to calculate the derivatives of the inverse of a matrix and of the determinant of the matrix, by varying the matrix itself.

Luckily, in my course we are working only with strictly positive definite symmetric matrices.

I would like to ask if the following results are correct.

Result 1:

$\displaystyle \frac{d{\Sigma ^ {-1}}}{d \Sigma}=-{\Sigma}^{-2}$

Proof:

It can be easily proven that the derivative of a matrix product AB follows the same rule of the derivative of the product of two scalars (by calculating the limit of the product variation and neglecting infinitesimals of superior order), of course, if we assume that A and B are both differentiable.

$\displaystyle \frac{d{AB}}{d \Sigma}=\frac{d{A}}{d \Sigma}B+A\frac{d{B}}{d \Sigma}$

Now, if our matrix variable is constrained to be always invertible, we know that:

$\displaystyle \frac{ d{\left(\Sigma \Sigma^{-1}\right)}}{d \Sigma}=\Sigma^{-1}+\Sigma\frac{d{\Sigma^{-1}}}{d \Sigma}=\frac{ d{I}}{d \Sigma}=0$

Where 0 is a null square matrix. Solving the equation gives the preceding result. Notice that the first result requires the variable to be always invertible, not necessarily symmetric or positive definite.

Result 2: (this one requires the matrix to be symmetric and invertible)

$\displaystyle \frac{ d{|\Sigma|}}{d \Sigma}=|\Sigma|tr(\Sigma^{-1})$

Proof:

Let $\displaystyle \Lambda$ be the diagonal form of Sigma

$\displaystyle |\Sigma|=\prod _{i=1}^{n}{\lambda_i}$

$\displaystyle \frac{d|\Sigma|}{d\Sigma}=\left( \prod _{i=1}^{n}{\lambda_i}\right )\left(\sum _{i=1}^{n} {\left( \lambda_i^{-1}\frac{d\lambda_i}{d\Sigma} \right)}\right)=|\Sigma|tr\left(\Lambda^{-1}\frac{d\Lambda}{d\Sigma} \right )$

$\displaystyle \Lambda=R' \Sigma R$

Where R is also variable depending on Sigma, but constrained to:

$\displaystyle R'R=I$

Deriving the constraint we get:

$\displaystyle \frac{dR'}{d\Sigma}R=W=-W'$

$\displaystyle \frac{d\Lambda}{d\Sigma}=\frac{dR'}{d\Sigma}\Sigma R+R'R+R'\Sigma\frac{dR}{d\Sigma}=W\Lambda+I+\Lambd a W'$

Combining the results and remembering that the trace is linear and commutes, we finally get:

$\displaystyle tr\left(\Lambda^{-1}\frac{d\Lambda}{d\Sigma} \right )=tr(\Lambda^{-1})=tr(\Sigma^{-1})$

We assumed to vary Sigma on a set constrained to be always symmetric and invertible.

Actually, we can get the same result by using similar calculations on any Sigma constrained to be invertible, even though not diagonalizable, by using the triangular form (which is always possible if we extend to complex variables).