# Math Help - Markov Chain

1. ## Markov Chain

Hi Guys, just need a bit of help with this question. Suppose we have P = [0 1/2 1/2; 1 0 0; 1 0 0] and let Q_0=[x y z] where x,y,z >= 0 and x+y+z = 1.

How would I find Q_(2n) and Q_(2n-1), that is the state distribution after an even or odd number of steps? And then how would we use this to explain whether or not the limit distribution exists.

2. ## Re: Markov Chain

Hello, liedora!

Did you try anything?

$P \:=\:\begin{pmatrix}0&\frac{1}{2}&\frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix}\,\text{ and let }Q_0\,=\,[x\:y\:z]$
. . $\text{where }x,y,z \ge 0\text{ and }x+y+z \,=\, 1.$

$\text{Find }Q_{2n}\text{ and }Q_{2n-1}$
. . $\text{the state distribution after an even or odd number of steps.}$

We have: . $P \:=\:\begin{pmatrix}0&\frac{1}{2}&\frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix}$

$P^2 \:=\:\begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix}\begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix} \:=\:\begin{pmatrix}1&0&0\\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$P^3\:=\:\begin{pmatrix}1&0&0\\0&0&0\\0&0&0 \end{pmatrix}\begin{pmatrix}0&\frac{1}{2}&\frac{1} {2} \\ 1&0&0 \\ 1&0&0\end{pmatrix} \:=\:\begin{pmatrix}0&\frac{1}{2} & \frac{1}{2} \\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$P^4 \:=\:\begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 0&0&0 \\ 0&0&0 \end{pmatrix} \begin{pmatrix}0 & \frac{1}{2} & \frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix} \:=\:\begin{pmatrix}1&0&0 \\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$\text{Hence: }\:P^{2n} \:=\:\begin{pmatrix}1&0&0\\0&0&0\\0&0&0 \end{pmatrix} \qquad P^{2n+1} \:=\: \begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$\text{Therefore: }\:\begin{Bmatrix}Q_{2n} &=& Q_0\!\cdot\!P^{2n} &=& [x\;\;0\;\;0] \\ \\[-3mm] Q_{2n+1} &=& Q_0\!\cdot\!P^{2n+1} &=& \left[0\;\frac{1}{2}x\;\frac{1}{2}x\right] \end{Bmatrix}$

3. ## Re: Markov Chain

Hi Soroban I think that your matrix multiplication is incorrect, for example by hand and in MATLAB I got P^(2n) = P^2 = [1 0 0;0 1/2 1/2;0 1/2 1/2] and P^(2n+1) = P^3 = [0 1/2 1/2 ; 1 0 0 ; 1 0 0]. But I now understand what it means to find the state distribution after an odd or even amount of steps. So thanks for that

Also what does it mean by the limit distribution (using the above results we have established) I can't seem to decrypt the notes but I have a feeling the limit distribution does not exist.. Not sure how to show this however.

4. ## Re: Markov Chain

Originally Posted by liedora
Hi Soroban I think that your matrix multiplication is incorrect, for example by hand and in MATLAB I got P^(2n) = P^2 = [1 0 0;0 1/2 1/2;0 1/2 1/2] and P^(2n+1) = P^3 = [0 1/2 1/2 ; 1 0 0 ; 1 0 0]. But I now understand what it means to find the state distribution after an odd or even amount of steps. So thanks for that

Also what does it mean by the limit distribution (using the above results we have established) I can't seem to decrypt the notes but I have a feeling the limit distribution does not exist.. Not sure how to show this however.