Let X0,X1, . . . be the Markov chain with state space {1,2,3,4} and transition matrix

$\displaystyle \begin{pmatrix}p & 0 & 0.5-p & 0.5\\ 0 & 0 & 1/{3}& 2/{3}\\ 0.5 & 0.5 & 0 & 0\\ 0.5 & 0.5 & 0 & 0\\ \end{pmatrix} $

for some 0<=p<=1/2.

i) For which values of p is this Markov chain regular?

Is that p=0? otherwise the first row will not sum to 1.

ii) Find the limiting distribution for this Markov chain when p = 1/10.

I think I will be able to work this out!

How do do the the following?

iii) Suppose that the Xi forming the Markov chain of part ii) represent the location

on day i of a study of a gorilla moving randomly between 4 regions of a forest.

Write one sentence which explains the significance of the first coordinate

in the limiting distribution. Your answer should make sense to an intelligent

non-mathematician and should not use any special mathematical notation or

terminology.

Many Thanks