Results 1 to 3 of 3

Math Help - Classifcation of Markov States, Convergence

  1. #1
    Junior Member BrooketheChook's Avatar
    Joined
    Sep 2009
    From
    Gold Coast
    Posts
    27

    Classifcation of Markov States, Convergence

    A markov chain X has the state space transition matrix P (see attached image) where q = 1-p and 0 < p < 1

    Classify the states of X and give the stationary distributions.

    Show that  P^n does not converge, but  P^{2n} and  P^{2n+1} do converge as n approaches infinity.
    Attached Thumbnails Attached Thumbnails Classifcation of Markov States, Convergence-transition-matrix.jpg  
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Junior Member BrooketheChook's Avatar
    Joined
    Sep 2009
    From
    Gold Coast
    Posts
    27
    Quote Originally Posted by BrooketheChook View Post

    Show that  P^n does not converge, but  P^{2n} and  P^{2n+1} do converge as n approaches infinity.
    I have managed to classify the states and find the stationary distributions, but I really need help with the part above. I am really stuck
    Follow Math Help Forum on Facebook and Google+

  3. #3
    MHF Contributor

    Joined
    Aug 2008
    From
    Paris, France
    Posts
    1,174
    Quote Originally Posted by BrooketheChook View Post
    I have managed to classify the states and find the stationary distributions, but I really need help with the part above. I am really stuck
    This relates to periodicity (the period is 2). You can see that, for any state i, P^n(i,i)=0 when n is odd, and is positive when n is even. Indeed, the states are "on a line", and the walk needs to perform equally many steps down and up the line to go back to the initial point, hence an even number of steps.
    Therefore, if P^n(i,i) converges, it must be toward 0 (there is a subsequence (the odd terms) that stays at 0).
    In the same way, for any i,j, P^n(i,j) is 0 or positive depending of the parity of n, hence the limit, if any, would be 0.
    It is however impossible that the whole matrix P^n tends to 0 since the sum of the lines must stay equal to 1 at the limit. As a conclusion, P^n does not converge.

    On the other hand, if you compute P^2, you can see that it is irreducible and aperiodic, hence (theorem...) P^{2n} converges. As a by-product, we get that P^{2n+1}=P^{2n} P converges as well.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. markov process and classification of states
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: April 18th 2011, 06:56 PM
  2. Markov Chains + Transient States Proof - Help
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: November 7th 2010, 07:40 AM
  3. Markov chain convergence
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: July 20th 2009, 02:55 AM
  4. Markov chain and classifying states
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: March 17th 2009, 07:48 PM
  5. Markov Chain: Transient and aperiodic states question
    Posted in the Advanced Statistics Forum
    Replies: 5
    Last Post: March 8th 2008, 12:17 PM

Search Tags


/mathhelpforum @mathhelpforum