
Matrix trace relations
The trace of a matrix is the sum of the entries on the leading diagonal. So for
a 2*2 matrix a11 a12
a21 a22
the trace is a11+a22.
Let a and b be matrices in SL(2,C) i.e. 2*2 matrices with entries that are
complex numbers and with determinant 1. Let us call the inverse matrices A,B
Preliminaries: show that trace is invariant under conjugation and inversion i.e. that trace(B*a*b) = trace(a)=trace(A)
Show that trace(a*b) + trace(a*B) = trace(a)*trace(b)
Show that if a*a = I if and only if a has trace 0 (where I is the identity matrix).
Show that a^3 = I if and only if a has trace 1 and a^6=I if and only if
a has trace 1.
Let x = trace(a),y = trace(b),z = trace(a*b),w = trace(a*B)
Show that trace(a*b*A*B) = x*x+y*y+z*zx*y*z2
= x*x+y*y+w*wx*y*w2
= x*x+y*yz*w2
(The above relations, especially the first, are very important for geometry and in the theory of Kleinian Groups)
Now the hard part:
Suppose x^2 <> 4. Show that for n>=1 (trace(a^n*b*A^N*B)2)/(trace^2(a^n)4) is constant.
I got this result from the book "Outer Circles" which says that you can prove it for n=2 and then use induction. But I could not do it by induction.
Instead I found a general expression for calculating trace(a^n*w) given knowledge of trace(a), trace(w) and trace(a*w) by using a difference equation, and then proved it by brute force.

The book suggests that for $\displaystyle \frac{\mathop{tr}(A^nBA^{n}B^{1})2}{(\mathop{tr}A^n)^24}$ we should write $\displaystyle A$ in standard form. For a $\displaystyle 2\times2$ matrix $\displaystyle A$ of determinant $\displaystyle 1$ with $\displaystyle (\mathop{tr}A)^2\neq4$ this means a matrix of the form $\displaystyle \begin{pmatrix}t&0\\0&t^{1}\end{pmatrix}$. Since $\displaystyle \mathop{tr} CD=\mathop{tr}(U^{1}CUU^{1}DU)$ for any invertible matrix $\displaystyle U$, we may assume that $\displaystyle A$ is standard and $\displaystyle B$ is any $\displaystyle \textrm{SL}(2,\mathbb C)$ matrix.
Thus $\displaystyle A^nBA^{n}B^{1}=\begin{pmatrix}t^n&0\\0&t^{n}\end{pmatrix}\begin{pmatrix}a&b\\c&d\end{pmatrix }\begin{pmatrix}t^{n}&0\\0&t^n\end{pmatrix}\begin{pmatrix}d&b\\c&a\end{pmatrix}$$\displaystyle \ =\begin{pmatrix}adt^{2n}bc&\dots\\\dots&adt^{2n}bc\end{pmatrix}$.
Thus $\displaystyle \mathop{tr}(A^nBA^{n}B^{1})2=2adbc(t^{2n}+t^{2n})2=bc(t^{2n}+t^{2n}2)$ using $\displaystyle adbc=1$.
Also $\displaystyle (\mathop{tr}A^n)^24=(t^n+t^{n})^24=t^{2n}+t^{2n}2$. Therefore $\displaystyle \frac{\mathop{tr}(A^nBA^{n}B^{1})2}{(\mathop{tr}A^n)^24}=bc$, a constant. (Of course this assumes $\displaystyle t^{2n}\neq 1$.)

That's much shorter than my proof and will teach me not to assume avoiding multiplying matrices and using trace identities instead is always is a good plan. I notice you haven't proved it by induction either. Your remark shows the result is not actually true if A has finite order.