# Markov Chain question

• Mar 30th 2008, 11:38 AM
rufusspeaks
Markov Chain question
how do you calculate the expected time period, the process stays in state K for example, before moving to another state?

can anyone point me in the right direction

thank you
• Mar 31st 2008, 06:13 AM
CaptainBlack
Quote:

Originally Posted by rufusspeaks
how do you calculate the expected time period, the process stays in state K for example, before moving to another state?

can anyone point me in the right direction

thank you

Let $\displaystyle p_k$ be the probability that given the current state is $\displaystyle K$ that the next state will be $\displaystyle K$.

Then given that we are in state [matth]K[/tex]:

Prob that we stay in state $\displaystyle K$ for $\displaystyle 0$ epocs is $\displaystyle (1-p_k)$

Prob that we stay in state $\displaystyle K$ for $\displaystyle 1$ epocs is $\displaystyle p_k(1-p_k)$

Prob that we stay in state $\displaystyle K$ for $\displaystyle n$ epocs is $\displaystyle p_k^n(1-p_k)$

Expected number of epocs we remain in $\displaystyle K$ given that we are in $\displaystyle K$ is:

$\displaystyle E(n)=\sum_{r=0}^{\infty} r p_k^r (1-p_k)$

RonL
• Apr 2nd 2008, 11:48 AM
rufusspeaks
Quote:

Originally Posted by CaptainBlack
Let $\displaystyle p_k$ be the probability that given the current state is $\displaystyle K$ that the next state will be $\displaystyle K$.

Then given that we are in state [matth]K[/tex]:

Prob that we stay in state $\displaystyle K$ for $\displaystyle 0$ epocs is $\displaystyle (1-p_k)$

Prob that we stay in state $\displaystyle K$ for $\displaystyle 1$ epocs is $\displaystyle p_k(1-p_k)$

Prob that we stay in state $\displaystyle K$ for $\displaystyle n$ epocs is $\displaystyle p_k^n(1-p_k)$

Expected number of epocs we remain in $\displaystyle K$ given that we are in $\displaystyle K$ is:

$\displaystyle E(n)=\sum_{r=0}^{\infty} r p_k^r (1-p_k)$

RonL

Would it be correct to use the limiting property of transition probabilities to give all the $\displaystyle r$s needed to evaluate the summation?

thank you.