transition matrix of a Markov Chain

• Nov 29th 2013, 05:31 AM
ibnashraf
transition matrix of a Markov Chain
Hi all,

Can someone outline a method for finding the following ...

Given the transition matrix of a Markov Chain, how do I find

1. the communicating classes

2. whether or not each state is recurrent and whether periodic.

Any help would be greatly appreciated ....
• Nov 29th 2013, 02:46 PM
chiro
Re: transition matrix of a Markov Chain
Hey ibnashraf.

Did you try drawing it in the following form? (This is a good start for markov chain problems where you need some intuition about the system)

File:Markovkate 01.svg - Wikipedia, the free encyclopedia
• Nov 29th 2013, 05:07 PM
romsek
Re: transition matrix of a Markov Chain
Quote:

Originally Posted by ibnashraf
Hi all,

Can someone outline a method for finding the following ...

Given the transition matrix of a Markov Chain, how do I find

1. the communicating classes

2. whether or not each state is recurrent and whether periodic.

Any help would be greatly appreciated ....

Ok, I'm no expert at this but I think I can help with the first part anyway.
If you've got a set of states si and a sequence s[n] = si then you've got your transition matrix
Tij = Pr[ s[n+1]=si | s[n]=sj ]

The labelling of the states doesn't really matter so you can reorder the rows as long as you relabel the states accordingly.
If you can reorder the rows such that T is a block diagonal matrix then the blocks represent the classes of communicating states.

for (2) I can send you here and wish you luck.