Can someone please help me solve the equation by using the inverse of the coefficient matrix? Thanks!
4x+y-4z=17
2x+y-z=12
-2x-4y+5z=17


If the problem specifically requires that you find the inverse of the coefficient matrix, then multiply the right side by that inverse, the coefficient matrix is
.
Now the question is, what methods for finding inverses matrices have you learned or are expected to use? There are many different ones.
The simplest, I think is to write the coefficient matrix and identity matrix side by side:
Now use "row operations" as dwsmith suggested to reduce the left matrix to the identity, at the same time applying those row operations to the matrix on the right. Once the left matrix has been reduced to the identity, the right matrix will have changed to the inverse matrix. Then multiply that inverse matrix by the right side column matrix.
Of course, what dwsmith does, row-reducing the coefficient matrix while simultaneously applying the row operations to the right hand column, combines those two things into one.
Finding the inverse matrix separately does have the advantage that then that inverse matrix can be multiplied by many different column matrices to solve Ax= b for the same "A" but many different "b"s. It typically happens in applications that result in equations of the form "Ax= b" that "A" reflects the "structure" of the problem while "b" some specific conditions. Thus, it often happens that one needs to solve "Ax= b" for the same "A" but many different "b" to observe the result of different "specific conditions" on the same "structure".