First of all, you have calculated the matrix of minors wrong, it should be:
Now to create the adjugate (or adjoint) matrix, we multiply each ij-entry by , that is, alternate signs, starting with postive in the upper-left corner, to get:
We will want the transpose of this matrix, which is:
We need to now calculate the determinant of the original matrix, to get a suitable scalar factor:
The easiest way is to expand by minors along the second column, since it has only one non-zero entry. Since this is in the 3,2 position, we take the negative of this determinant (since 3+2 = 5 is odd), the cofactor of the other 2 subdeterminants will be 0, so we don't have to calculate the matrix of the other minors (although we've already done it, so meh...).
that is, the determinant of the original matrix is:
So our scaling factor will be 1/10.
This gives the inverse: