# Markov Chain

• May 5th 2012, 07:14 PM
liedora
Markov Chain
Hi Guys, just need a bit of help with this question. Suppose we have P = [0 1/2 1/2; 1 0 0; 1 0 0] and let Q_0=[x y z] where x,y,z >= 0 and x+y+z = 1.

How would I find Q_(2n) and Q_(2n-1), that is the state distribution after an even or odd number of steps? And then how would we use this to explain whether or not the limit distribution exists.

• May 6th 2012, 05:58 AM
Soroban
Re: Markov Chain
Hello, liedora!

Did you try anything?

Quote:

$\displaystyle P \:=\:\begin{pmatrix}0&\frac{1}{2}&\frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix}\,\text{ and let }Q_0\,=\,[x\:y\:z]$
. . $\displaystyle \text{where }x,y,z \ge 0\text{ and }x+y+z \,=\, 1.$

$\displaystyle \text{Find }Q_{2n}\text{ and }Q_{2n-1}$
. . $\displaystyle \text{the state distribution after an even or odd number of steps.}$

We have: .$\displaystyle P \:=\:\begin{pmatrix}0&\frac{1}{2}&\frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix}$

$\displaystyle P^2 \:=\:\begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix}\begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix} \:=\:\begin{pmatrix}1&0&0\\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$\displaystyle P^3\:=\:\begin{pmatrix}1&0&0\\0&0&0\\0&0&0 \end{pmatrix}\begin{pmatrix}0&\frac{1}{2}&\frac{1} {2} \\ 1&0&0 \\ 1&0&0\end{pmatrix} \:=\:\begin{pmatrix}0&\frac{1}{2} & \frac{1}{2} \\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$\displaystyle P^4 \:=\:\begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 0&0&0 \\ 0&0&0 \end{pmatrix} \begin{pmatrix}0 & \frac{1}{2} & \frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix} \:=\:\begin{pmatrix}1&0&0 \\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$\displaystyle \text{Hence: }\:P^{2n} \:=\:\begin{pmatrix}1&0&0\\0&0&0\\0&0&0 \end{pmatrix} \qquad P^{2n+1} \:=\: \begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$\displaystyle \text{Therefore: }\:\begin{Bmatrix}Q_{2n} &=& Q_0\!\cdot\!P^{2n} &=& [x\;\;0\;\;0] \\ \\[-3mm] Q_{2n+1} &=& Q_0\!\cdot\!P^{2n+1} &=& \left[0\;\frac{1}{2}x\;\frac{1}{2}x\right] \end{Bmatrix}$
• May 6th 2012, 02:12 PM
liedora
Re: Markov Chain
Hi Soroban I think that your matrix multiplication is incorrect, for example by hand and in MATLAB I got P^(2n) = P^2 = [1 0 0;0 1/2 1/2;0 1/2 1/2] and P^(2n+1) = P^3 = [0 1/2 1/2 ; 1 0 0 ; 1 0 0]. But I now understand what it means to find the state distribution after an odd or even amount of steps. So thanks for that :) :)

Also what does it mean by the limit distribution (using the above results we have established) I can't seem to decrypt the notes but I have a feeling the limit distribution does not exist.. Not sure how to show this however.

• May 9th 2012, 11:33 PM
Moo
Re: Markov Chain
Quote:

Originally Posted by liedora
Hi Soroban I think that your matrix multiplication is incorrect, for example by hand and in MATLAB I got P^(2n) = P^2 = [1 0 0;0 1/2 1/2;0 1/2 1/2] and P^(2n+1) = P^3 = [0 1/2 1/2 ; 1 0 0 ; 1 0 0]. But I now understand what it means to find the state distribution after an odd or even amount of steps. So thanks for that :) :)

Also what does it mean by the limit distribution (using the above results we have established) I can't seem to decrypt the notes but I have a feeling the limit distribution does not exist.. Not sure how to show this however.