how do you calculate the expected time period, the process stays in state K for example, before moving to another state?

can anyone point me in the right direction

thank you

Printable View

- Mar 30th 2008, 11:38 AMrufusspeaksMarkov Chain question
how do you calculate the expected time period, the process stays in state K for example, before moving to another state?

can anyone point me in the right direction

thank you - Mar 31st 2008, 06:13 AMCaptainBlack
Let $\displaystyle p_k$ be the probability that given the current state is $\displaystyle K$ that the next state will be $\displaystyle K$.

Then given that we are in state [matth]K[/tex]:

Prob that we stay in state $\displaystyle K$ for $\displaystyle 0$ epocs is $\displaystyle (1-p_k) $

Prob that we stay in state $\displaystyle K$ for $\displaystyle 1$ epocs is $\displaystyle p_k(1-p_k)$

Prob that we stay in state $\displaystyle K$ for $\displaystyle n$ epocs is $\displaystyle p_k^n(1-p_k)$

Expected number of epocs we remain in $\displaystyle K$ given that we are in $\displaystyle K$ is:

$\displaystyle E(n)=\sum_{r=0}^{\infty} r p_k^r (1-p_k)$

RonL - Apr 2nd 2008, 11:48 AMrufusspeaks