# Markov Chain of random variables from a primitive markov chain

• Oct 19th 2011, 08:12 AM
puggles
Markov Chain of random variables from a primitive markov chain
I'm a little unsure if my approach here is correct, which is making it difficult to find the limit. Any and all help will be appreciated!

Let $\displaystyle {X_n}$ be a primitive finite state Markov chain with transition matrix $\displaystyle P$ and stationary distribution $\displaystyle w$. Define the process $\displaystyle Y_n$ by:
$\displaystyle Y_n = (X_{n-1}, X_n)$

Show that $\displaystyle {Y_n}$ is a Markov chain, and compute:
$\displaystyle lim_{n\to\infty} P(Y_n = (i,j))$

What I've done:
So I need to show that $\displaystyle Y_{n+1}$ is only determined by $\displaystyle Y_n$, ie:
$\displaystyle P(Y_{n+1} | Y_{n}, Y_{n-1}, ... , Y_0) = P(Y_{n+1} | Y_{n})$

So, suppose $\displaystyle Y_n = (a,b)$ a & b are just some numbers. Then:
$\displaystyle P(Y_{n+1} = (i,j) | Y_{n} = (a,b))$
Plugging in the definition of $\displaystyle {Y_n}$:
= $\displaystyle P((X_{n}=i, X_{n+1}= j) | (X_{n-1} = a, X_n=b))$
Because $\displaystyle {X_n}$ are primitive Markov-Chains, can I say that
= $\displaystyle P((X_{n}=i, X_{n+1}= j)$
because $\displaystyle X_{n+1}$ doesn't depend on $\displaystyle X_{n-1}$? This is basically where I'm stuck :(