I'm having some trouble with part 2 of this question. Any help would be appreciated. Thanks,
Let A be the matrix for part 1. Find the eigenvalues for A by solving the characteristic equation. Find 3 orthonormal eigenvectors corresponding to the eigenvalues. These are the basis vectors for B'. Form matrix P from these basis vectors. Then P^T = P^-1 from orthonormality. The representation matrix is R = P^-1 A P, which is diagonal with the eigenvalues on the diagonal.