
Markov chain estimation
Hello,
I am really just hoping to know how to get started on this problem. The problem is:
Consider a Markov chain with given transition probabilities and a single recurrent class that is aperiodic. Assume that for n >=100 the transition probabilities are very close to the steady state probabilities.
(a) Find an approximate formula for P(X_{1000} = j, X_{1001} = k, X_{2000} = p  X_{0} = i)
(b) Find an approximate forumla for P(X_{1000} = i  X_{1002} = j)
I know the definitions of aperiodicity, recurrent class and steady state probabilities/transition probabilities, but I don't know what to do. Any help is appreciated!