Results 1 to 3 of 3

Math Help - transition matrix of a Markov Chain

  1. #1
    Junior Member ibnashraf's Avatar
    Joined
    Aug 2008
    From
    Caribbean
    Posts
    47

    transition matrix of a Markov Chain

    Hi all,

    Can someone outline a method for finding the following ...

    Given the transition matrix of a Markov Chain, how do I find

    1. the communicating classes

    2. whether or not each state is recurrent and whether periodic.


    Any help would be greatly appreciated ....
    Follow Math Help Forum on Facebook and Google+

  2. #2
    MHF Contributor
    Joined
    Sep 2012
    From
    Australia
    Posts
    3,667
    Thanks
    606

    Re: transition matrix of a Markov Chain

    Hey ibnashraf.

    Did you try drawing it in the following form? (This is a good start for markov chain problems where you need some intuition about the system)

    File:Markovkate 01.svg - Wikipedia, the free encyclopedia
    Follow Math Help Forum on Facebook and Google+

  3. #3
    MHF Contributor
    Joined
    Nov 2013
    From
    California
    Posts
    2,488
    Thanks
    963

    Re: transition matrix of a Markov Chain

    Quote Originally Posted by ibnashraf View Post
    Hi all,

    Can someone outline a method for finding the following ...

    Given the transition matrix of a Markov Chain, how do I find

    1. the communicating classes

    2. whether or not each state is recurrent and whether periodic.


    Any help would be greatly appreciated ....
    Ok, I'm no expert at this but I think I can help with the first part anyway.
    If you've got a set of states si and a sequence s[n] = si then you've got your transition matrix
    Tij = Pr[ s[n+1]=si | s[n]=sj ]

    The labelling of the states doesn't really matter so you can reorder the rows as long as you relabel the states accordingly.
    If you can reorder the rows such that T is a block diagonal matrix then the blocks represent the classes of communicating states.

    for (2) I can send you here and wish you luck.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Markov chain transition matrix
    Posted in the Advanced Statistics Forum
    Replies: 4
    Last Post: May 18th 2009, 10:08 PM
  2. Markov chain transition matrix
    Posted in the Advanced Statistics Forum
    Replies: 3
    Last Post: May 15th 2009, 02:52 AM
  3. Transition matrix for Markov chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: May 8th 2009, 03:04 AM
  4. Transition Matrix (markov chain help)
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: April 1st 2009, 04:32 PM
  5. Transition matrix for Markov Chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: February 17th 2009, 12:26 AM

Search Tags


/mathhelpforum @mathhelpforum