Higher Fréchet derivatives are a total pain, because they all belong to different spaces. If is a differentiable map from V to W then, for each v in V, Df(v) is a linear map from V to W. Using the notation L(V,W) for the space of all linear maps from V to W, Df is a map from V to L(V,W). But that means that the second derivative of f is a linear map from V to L(V,W). In other words, . And so it goes on: , ... .

The space can be identified with the space of bilinear maps from to W, and similarly for higher derivatives. So can be regarded as an n-linear map from (n factors) to W.

For the map on the space of matrices, . The second derivative is given (as a bilinear map) by . Notice that in the case of a 1×1 matrix ( , a scalar, in other words), these formulas reduce to the familiar results and .

The Fréchet derivatives of the determinant function are just as bad, if not worse. The determinant is a function from to the scalars, . So is a linear map from to , given by . The second derivative is given as a bilinear map from to by .

Having said all that, if you're trying to find the Taylor series of f(I+H) about the identity matrix I, you don't want Fréchet derivatives at all. Just use the binomial expansion (valid whenever H is small enough in some suitable metric).

Edit.You asked for the formula for the n'th derivative of the function . My guess is that (but I haven't tried to prove that).