Results 1 to 3 of 3

Thread: Markov Chain question

  1. #1
    Newbie
    Joined
    Mar 2008
    Posts
    2

    Markov Chain question

    how do you calculate the expected time period, the process stays in state K for example, before moving to another state?

    can anyone point me in the right direction

    thank you
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Grand Panjandrum
    Joined
    Nov 2005
    From
    someplace
    Posts
    14,972
    Thanks
    5
    Quote Originally Posted by rufusspeaks View Post
    how do you calculate the expected time period, the process stays in state K for example, before moving to another state?

    can anyone point me in the right direction

    thank you
    Let $\displaystyle p_k$ be the probability that given the current state is $\displaystyle K$ that the next state will be $\displaystyle K$.

    Then given that we are in state [matth]K[/tex]:

    Prob that we stay in state $\displaystyle K$ for $\displaystyle 0$ epocs is $\displaystyle (1-p_k) $

    Prob that we stay in state $\displaystyle K$ for $\displaystyle 1$ epocs is $\displaystyle p_k(1-p_k)$

    Prob that we stay in state $\displaystyle K$ for $\displaystyle n$ epocs is $\displaystyle p_k^n(1-p_k)$

    Expected number of epocs we remain in $\displaystyle K$ given that we are in $\displaystyle K$ is:

    $\displaystyle E(n)=\sum_{r=0}^{\infty} r p_k^r (1-p_k)$

    RonL
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Mar 2008
    Posts
    2
    Quote Originally Posted by CaptainBlack View Post
    Let $\displaystyle p_k$ be the probability that given the current state is $\displaystyle K$ that the next state will be $\displaystyle K$.

    Then given that we are in state [matth]K[/tex]:

    Prob that we stay in state $\displaystyle K$ for $\displaystyle 0$ epocs is $\displaystyle (1-p_k) $

    Prob that we stay in state $\displaystyle K$ for $\displaystyle 1$ epocs is $\displaystyle p_k(1-p_k)$

    Prob that we stay in state $\displaystyle K$ for $\displaystyle n$ epocs is $\displaystyle p_k^n(1-p_k)$

    Expected number of epocs we remain in $\displaystyle K$ given that we are in $\displaystyle K$ is:

    $\displaystyle E(n)=\sum_{r=0}^{\infty} r p_k^r (1-p_k)$

    RonL
    Would it be correct to use the limiting property of transition probabilities to give all the $\displaystyle r$s needed to evaluate the summation?

    thank you.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Markov Chain of random variables from a primitive markov chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: Oct 19th 2011, 08:12 AM
  2. Markov chain question
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: Mar 4th 2011, 10:52 AM
  3. Markov Chain Question
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: Feb 15th 2011, 04:42 PM
  4. Replies: 2
    Last Post: Oct 28th 2008, 06:32 PM
  5. Need help on markov chain question ?
    Posted in the Advanced Statistics Forum
    Replies: 3
    Last Post: Apr 27th 2008, 08:42 AM

Search Tags


/mathhelpforum @mathhelpforum