A markov chain X has the state space transition matrix P (see attached image) where q = 1-p and 0 < p < 1

Classify the states of X and give the stationary distributions.

Show that does not converge, but and do converge as n approaches infinity.

Printable View

- Sep 14th 2009, 07:11 AMBrooketheChookClassifcation of Markov States, Convergence
A markov chain X has the state space transition matrix P (see attached image) where q = 1-p and 0 < p < 1

Classify the states of X and give the stationary distributions.

Show that does not converge, but and do converge as n approaches infinity. - Sep 15th 2009, 06:01 AMBrooketheChook
- Sep 23rd 2009, 04:07 PMLaurent
This relates to periodicity (the period is 2). You can see that, for any state , when is odd, and is positive when is even. Indeed, the states are "on a line", and the walk needs to perform equally many steps down and up the line to go back to the initial point, hence an even number of steps.

Therefore, if converges, it must be toward 0 (there is a subsequence (the odd terms) that stays at 0).

In the same way, for any , is 0 or positive depending of the parity of , hence the limit, if any, would be 0.

It is however impossible that the whole matrix tends to 0 since the sum of the lines must stay equal to 1 at the limit. As a conclusion, does not converge.

On the other hand, if you compute , you can see that it is irreducible and aperiodic, hence (theorem...) converges. As a by-product, we get that converges as well.