def.: is the collection of lineair orthogonal transformations of , the elements are matrices.
I want to show that two element of are conjugated if and only if they have the same trace.
(Hint: Maby you want to use: if )
No. Two square matrices are conjugate if there exists SOME invertible matrix s.t.
One direction of your problem is almost trivial: ANY two conjugate matrices, whether orthogonal or not, have the same trace (and the same determinant, and the same eigenvalues...).
The other direction looks highly suspicious to me, but I cannot prove it isn't true.
Using the hint : by an appropriate conjugation, you can put any in the form by an appropriate base change... and in fact we can also suppose . (What this means is essentially that a rotation of three-dimensional Euclidean space is always a rotation around an axis; the axis is fixed, and its orthogonal complement is rotated like a plane. So two rotations are conjugate only if the angles of rotation are equal up to a sign.)
Can you take it from there?
Another way might be to use the isomorphism , and the fact that two Möbius transformations are conjugate if and only if they have the same multiplier. But that's more complicated.