Hello,

I am really just hoping to know how to get started on this problem. The problem is:

Consider a Markov chain with given transition probabilities and a single recurrent class that is aperiodic. Assume that for n >=100 the transition probabilities are very close to the steady state probabilities.

(a) Find an approximate formula for P(X_{1000}= j, X_{1001}= k, X_{2000}= p | X_{0}= i)

(b) Find an approximate forumla for P(X_{1000}= i | X_{1002}= j)

I know the definitions of aperiodicity, recurrent class and steady state probabilities/transition probabilities, but I don't know what to do. Any help is appreciated!