What you are calling the adjoint is what I call the adjugate. The adjoint of an endomorphism (if it exists) on an inner product space is the unique endomorphism such that for every . It has the property that if you fix an orthonormal ordered basis (assuming ) then (where here is the matrix representation with respect to that ordered basis, and ).
My linear book describes adjoint as the linear operator T* such that (T*(v), w) = (v, T(w)). * relates to linear operators in much the same way as * relates to matrices.
Of course, this isn't the first time mathematicians use the same word to describe multiple mathematical concepts.