Markov Chains + Transient States Proof - Help
Hi all,
I am trying to proof that:
P^n(kj) = 0 for all n when K is recurrent, where P(kj) is the expected # of time periods that a markov chain is in state j given that it starts in state k.
I am totally lost and don't even know where to start. Please demonstrate how to tackle this proof. Thanks.
EDIT:
Ok, I have figured out the following:
P(kj) = 0 + Sum(P[1st transition is in state k) * E[# of visits to j | starts at k]
So I KNOW based on this equation why it will turn out to zero. Obviously (Sum(P[1st transition is in state k) * E[# of visits to j | starts at k]) has to = 0 for the proof to work, but how do I figure that part out?????
Can somebody show me why:
Sum(P[1st transition is in state k) * E[# of visits to j | starts at k] = 0 ? Thanks