Results 1 to 3 of 3

Math Help - Markov Chain question

  1. #1
    Newbie
    Joined
    Mar 2008
    Posts
    2

    Markov Chain question

    how do you calculate the expected time period, the process stays in state K for example, before moving to another state?

    can anyone point me in the right direction

    thank you
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Grand Panjandrum
    Joined
    Nov 2005
    From
    someplace
    Posts
    14,972
    Thanks
    4
    Quote Originally Posted by rufusspeaks View Post
    how do you calculate the expected time period, the process stays in state K for example, before moving to another state?

    can anyone point me in the right direction

    thank you
    Let p_k be the probability that given the current state is K that the next state will be K.

    Then given that we are in state [matth]K[/tex]:

    Prob that we stay in state K for 0 epocs is (1-p_k)

    Prob that we stay in state K for 1 epocs is p_k(1-p_k)

    Prob that we stay in state K for n epocs is p_k^n(1-p_k)

    Expected number of epocs we remain in K given that we are in K is:

    E(n)=\sum_{r=0}^{\infty} r p_k^r (1-p_k)

    RonL
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Mar 2008
    Posts
    2
    Quote Originally Posted by CaptainBlack View Post
    Let p_k be the probability that given the current state is K that the next state will be K.

    Then given that we are in state [matth]K[/tex]:

    Prob that we stay in state K for 0 epocs is (1-p_k)

    Prob that we stay in state K for 1 epocs is p_k(1-p_k)

    Prob that we stay in state K for n epocs is p_k^n(1-p_k)

    Expected number of epocs we remain in K given that we are in K is:

    E(n)=\sum_{r=0}^{\infty} r p_k^r (1-p_k)

    RonL
    Would it be correct to use the limiting property of transition probabilities to give all the rs needed to evaluate the summation?

    thank you.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Markov Chain of random variables from a primitive markov chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: October 19th 2011, 09:12 AM
  2. Markov chain question
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: March 4th 2011, 11:52 AM
  3. Markov Chain Question
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: February 15th 2011, 05:42 PM
  4. Replies: 2
    Last Post: October 28th 2008, 07:32 PM
  5. Need help on markov chain question ?
    Posted in the Advanced Statistics Forum
    Replies: 3
    Last Post: April 27th 2008, 09:42 AM

Search Tags


/mathhelpforum @mathhelpforum