You should check the definition, i.e. compute the probability of "paths" of the process and see that it matches the same probability for some Markov chain.

For all , for all , we have:

because of the independence between and . Then you can express each factor of the right hand side using the definition of a Markov chain (like where is the distribution of the initial location). And rewrite the formula so that it looks like for some and matrix . That would answer the question.

Note that there are many different conventions of notation, and several ways to write the definitions; you may have to adapt the above, but the principle remains the same.

The idea is the following. The Markov chain is "reducible" if there are two states such that for all , i.e. if there is no "path" for from toward . On one hand, since both and are irreducible, there is both a path for from to , and a path for from to . But it may happen that there exist no two paths(b) Prove that if and are both periodic, then can be reducible.of the same lengththat satisfy these constraints, which is what it would require for to connect to .

Consider the following simple example: is a Markov chain with state space and transition matrix such that and . What does look like? If has same distribution as , is irreducible?

(You could look for a less singular example afterward)

Let . You need to prove for some .(c) Show that is irreducible if both and are aperiodic.Hint:Use the fact that if is irreducible and aperiodic, then for each there exists some such that for all

Given the assumptions, we can apply the hint to both (with the states and ) and . The expression for obtained in (a) should then make it easy to conclude.

Tell me what you tried if you experience further difficulties.

Laurent.