I am really just hoping to know how to get started on this problem. The problem is:
Consider a Markov chain with given transition probabilities and a single recurrent class that is aperiodic. Assume that for n >=100 the transition probabilities are very close to the steady state probabilities.
(a) Find an approximate formula for P(X1000 = j, X1001 = k, X2000 = p | X0 = i)
(b) Find an approximate forumla for P(X1000 = i | X1002 = j)
I know the definitions of aperiodicity, recurrent class and steady state probabilities/transition probabilities, but I don't know what to do. Any help is appreciated!