transition matrix of a Markov Chain

Hi all,

Can someone outline a method for finding the following ...

Given the transition matrix of a Markov Chain, how do I find

1. the communicating classes

2. whether or not each state is recurrent and whether periodic.

Any help would be greatly appreciated ....

Re: transition matrix of a Markov Chain

Hey ibnashraf.

Did you try drawing it in the following form? (This is a good start for markov chain problems where you need some intuition about the system)

File:Markovkate 01.svg - Wikipedia, the free encyclopedia

Re: transition matrix of a Markov Chain

Quote:

Originally Posted by

**ibnashraf** Hi all,

Can someone outline a method for finding the following ...

Given the transition matrix of a Markov Chain, how do I find

1. the communicating classes

2. whether or not each state is recurrent and whether periodic.

Any help would be greatly appreciated ....

Ok, I'm no expert at this but I think I can help with the first part anyway.

If you've got a set of states s_{i} and a sequence s[n] = s_{i} then you've got your transition matrix

Tij = Pr[ s[n+1]=s_{i} | s[n]=s_{j} ]

The labelling of the states doesn't really matter so you can reorder the rows as long as you relabel the states accordingly.

If you can reorder the rows such that T is a block diagonal matrix then the blocks represent the classes of communicating states.

for (2) I can send you here and wish you luck.