Let be the probability that given the current state is that the next state will be .

Then given that we are in state [matth]K[/tex]:

Prob that we stay in state for epocs is

Prob that we stay in state for epocs is

Prob that we stay in state for epocs is

Expected number of epocs we remain in given that we are in is:

RonL