Results 1 to 4 of 4

Math Help - Very basic question on periodicity of markov chain

  1. #1
    Member
    Joined
    Oct 2008
    Posts
    206

    Very basic question on periodicity of markov chain

    Suppose I have the following markov chain:

    \[ \left( \begin{array}{ccc}<br />
0 & 0.5 & 0.5 \\<br />
0.5 & 0 & 0.5 \\<br />
0.5 & 0.5 & 0 \end{array} \right)\]

    it seems that in this case that a state is able to return to itself either by 2 steps or 3 steps. Then what is the period of any single state? (all three states have the same periodicity though)

    By the way, does a transient state has a period too?
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Moo
    Moo is offline
    A Cute Angle Moo's Avatar
    Joined
    Mar 2008
    From
    P(I'm here)=1/3, P(I'm there)=t+1/3
    Posts
    5,618
    Thanks
    6
    Hello,

    If it were 2 or 3 steps (or a multiple), the periodicity of a state will be the gcd of 2 and 3. This is merely the definition of a period But here, you can have 5 steps : 0 - 1 - 2 - 1 - 2 - 0. So actually, the period will be 1.

    Transience and periodicity have no relationship, there is no period associated to a transient state.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Member
    Joined
    Oct 2008
    Posts
    206
    hi,

    Thanks for your help.. I was still quite unsure though. So it's possible that a state has d=1 even though it can't reach back by one step?

    State i is said to have a period d if (p_ii^n)=0 whenever n is not divisible by d, and d is the GCD. it's so confusing
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Moo
    Moo is offline
    A Cute Angle Moo's Avatar
    Joined
    Mar 2008
    From
    P(I'm here)=1/3, P(I'm there)=t+1/3
    Posts
    5,618
    Thanks
    6
    I guess your definition of a period only holds for d>1.
    The definition I've been taught for the period of a state is that it is the gcd of the following set : \{n\in\mathbb N ~:~ p_{ii}^n>0\}.

    We can read in the English wikipedia that a state is of period 1 if it returns to the state irregularly (no pattern). Here it can return after 2,3,5,7,8,9, etc... the gcd of these first 5 numbers is 1.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Markov Chain of random variables from a primitive markov chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: October 19th 2011, 08:12 AM
  2. Markov Chain Question
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: February 26th 2011, 07:39 PM
  3. 2nd Markov chain question
    Posted in the Advanced Statistics Forum
    Replies: 2
    Last Post: August 24th 2009, 02:04 AM
  4. Replies: 2
    Last Post: October 28th 2008, 06:32 PM
  5. Markov Chain question
    Posted in the Advanced Statistics Forum
    Replies: 2
    Last Post: April 2nd 2008, 11:48 AM

Search Tags


/mathhelpforum @mathhelpforum