Is it possible to have a markov chain with an infinite number of transient states and an infinite number of positive recurrent states?
Thanks for the help.
I can see where the infinite transient states come from but how do you know it gives an infiinte number of positive recurrent states? I don't really understand the definition I have of positive recurrent.....
a recurrent state i is said to be positive recurrent if the expected time of 1st return (starting from i) is finite
(1) If i is Transient this must hold for some j
P(i-->j)>0 (its possible to get to j)
P(j-->i)=0 (its impossible to get back)
(2) i is positive recurrent if for all j
P(i-->j)>0 (its possible to get to j)
P(j-->i)>0 (its ALWAYS possible to get back)
(3) If its always possible to get back, then E(time of first return to i)< oo.
Don't get caught up with expectation, just think of it in terms of (2).
Getting back in a finite time a.s. does not guarantee that the expectation is finite. If you suppose that the first return time T is distributed something like
$\displaystyle \mathbb{P}(T=n)=c \frac{1}{n^2}$ where c is some constant, then when you take the expectation you get
$\displaystyle \mathbb{E}[T]=c\sum\frac{1}{n}$
which is not finite. However here you will get back in finite time a.s.
The intuition is that you will get there eventually, just in an arbitrarily large time.
I'm having a similar sort of problem in understanding null-recurrence. What I want is a finite, irreducible Markov chain whereby ALL states are null-recurrent.
But isn't this just a simple (symmetric) random walk with reflecting barriers?
If anyone can help I would be really grateful!