A markov chain X has the state space transition matrix P (see attached image) where q = 1-p and 0 < p < 1
Classify the states of X and give the stationary distributions.
Show that does not converge, but and do converge as n approaches infinity.
Therefore, if converges, it must be toward 0 (there is a subsequence (the odd terms) that stays at 0).
In the same way, for any , is 0 or positive depending of the parity of , hence the limit, if any, would be 0.
It is however impossible that the whole matrix tends to 0 since the sum of the lines must stay equal to 1 at the limit. As a conclusion, does not converge.
On the other hand, if you compute , you can see that it is irreducible and aperiodic, hence (theorem...) converges. As a by-product, we get that converges as well.