1. ## Markov Chain question

how do you calculate the expected time period, the process stays in state K for example, before moving to another state?

can anyone point me in the right direction

thank you

2. Originally Posted by rufusspeaks
how do you calculate the expected time period, the process stays in state K for example, before moving to another state?

can anyone point me in the right direction

thank you
Let $\displaystyle p_k$ be the probability that given the current state is $\displaystyle K$ that the next state will be $\displaystyle K$.

Then given that we are in state [matth]K[/tex]:

Prob that we stay in state $\displaystyle K$ for $\displaystyle 0$ epocs is $\displaystyle (1-p_k)$

Prob that we stay in state $\displaystyle K$ for $\displaystyle 1$ epocs is $\displaystyle p_k(1-p_k)$

Prob that we stay in state $\displaystyle K$ for $\displaystyle n$ epocs is $\displaystyle p_k^n(1-p_k)$

Expected number of epocs we remain in $\displaystyle K$ given that we are in $\displaystyle K$ is:

$\displaystyle E(n)=\sum_{r=0}^{\infty} r p_k^r (1-p_k)$

RonL

3. Originally Posted by CaptainBlack
Let $\displaystyle p_k$ be the probability that given the current state is $\displaystyle K$ that the next state will be $\displaystyle K$.

Then given that we are in state [matth]K[/tex]:

Prob that we stay in state $\displaystyle K$ for $\displaystyle 0$ epocs is $\displaystyle (1-p_k)$

Prob that we stay in state $\displaystyle K$ for $\displaystyle 1$ epocs is $\displaystyle p_k(1-p_k)$

Prob that we stay in state $\displaystyle K$ for $\displaystyle n$ epocs is $\displaystyle p_k^n(1-p_k)$

Expected number of epocs we remain in $\displaystyle K$ given that we are in $\displaystyle K$ is:

$\displaystyle E(n)=\sum_{r=0}^{\infty} r p_k^r (1-p_k)$

RonL
Would it be correct to use the limiting property of transition probabilities to give all the $\displaystyle r$s needed to evaluate the summation?

thank you.