Hey ibnashraf.
Did you try drawing it in the following form? (This is a good start for markov chain problems where you need some intuition about the system)
File:Markovkate 01.svg - Wikipedia, the free encyclopedia
Hi all,
Can someone outline a method for finding the following ...
Given the transition matrix of a Markov Chain, how do I find
1. the communicating classes
2. whether or not each state is recurrent and whether periodic.
Any help would be greatly appreciated ....
Hey ibnashraf.
Did you try drawing it in the following form? (This is a good start for markov chain problems where you need some intuition about the system)
File:Markovkate 01.svg - Wikipedia, the free encyclopedia
Ok, I'm no expert at this but I think I can help with the first part anyway.
If you've got a set of states s_{i} and a sequence s[n] = s_{i} then you've got your transition matrix
Tij = Pr[ s[n+1]=s_{i} | s[n]=s_{j} ]
The labelling of the states doesn't really matter so you can reorder the rows as long as you relabel the states accordingly.
If you can reorder the rows such that T is a block diagonal matrix then the blocks represent the classes of communicating states.
for (2) I can send you here and wish you luck.