Hi all,

I am trying to proof that:

P^n(kj) = 0 for all n when K is recurrent, where P(kj) is the expected # of time periods that a markov chain is in state j given that it starts in state k.

I am totally lost and don't even know where to start. Please demonstrate how to tackle this proof. Thanks.

EDIT:

Ok, I have figured out the following:

P(kj) = 0 + Sum(P[1st transition is in state k) * E[# of visits to j | starts at k]

So I KNOW based on this equation why it will turn out to zero. Obviously (Sum(P[1st transition is in state k) * E[# of visits to j | starts at k]) has to = 0 for the proof to work, but how do I figure that part out?????

Can somebody show me why:

Sum(P[1st transition is in state k) * E[# of visits to j | starts at k] = 0 ? Thanks