Markov Chain

**liedora** · May 5th 2012, 07:14 PM

Hi Guys, just need a bit of help with this question. Suppose we have P = [0 1/2 1/2; 1 0 0; 1 0 0] and let Q_0=[x y z] where x,y,z >= 0 and x+y+z = 1.

How would I find Q_(2n) and Q_(2n-1), that is the state distribution after an even or odd number of steps? And then how would we use this to explain whether or not the limit distribution exists.

Thanks in advance

**Soroban** · May 6th 2012, 05:58 AM

Hello, liedora!

Did you try anything?

$P \:=\:\begin{pmatrix}0&\frac{1}{2}&\frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix}\,\text{ and let }Q_0\,=\,[x\:y\:z]$
. . $\text{where }x,y,z \ge 0\text{ and }x+y+z \,=\, 1.$

$\text{Find }Q_{2n}\text{ and }Q_{2n-1}$
. . $\text{the state distribution after an even or odd number of steps.}$

We have: . $P \:=\:\begin{pmatrix}0&\frac{1}{2}&\frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix}$

$P^2 \:=\:\begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix}\begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix} \:=\:\begin{pmatrix}1&0&0\\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$P^3\:=\:\begin{pmatrix}1&0&0\\0&0&0\\0&0&0 \end{pmatrix}\begin{pmatrix}0&\frac{1}{2}&\frac{1} {2} \\ 1&0&0 \\ 1&0&0\end{pmatrix} \:=\:\begin{pmatrix}0&\frac{1}{2} & \frac{1}{2} \\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$P^4 \:=\:\begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 0&0&0 \\ 0&0&0 \end{pmatrix} \begin{pmatrix}0 & \frac{1}{2} & \frac{1}{2} \\ 1&0&0 \\ 1&0&0\end{pmatrix} \:=\:\begin{pmatrix}1&0&0 \\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$\text{Hence: }\:P^{2n} \:=\:\begin{pmatrix}1&0&0\\0&0&0\\0&0&0 \end{pmatrix} \qquad P^{2n+1} \:=\: \begin{pmatrix}0&\frac{1}{2}& \frac{1}{2} \\ 0&0&0 \\ 0&0&0 \end{pmatrix}$

$\text{Therefore: }\:\begin{Bmatrix}Q_{2n} &=& Q_0\!\cdot\!P^{2n} &=& [x\;\;0\;\;0] \\ \\[-3mm] Q_{2n+1} &=& Q_0\!\cdot\!P^{2n+1} &=& \left[0\;\frac{1}{2}x\;\frac{1}{2}x\right] \end{Bmatrix}$

**liedora** · May 6th 2012, 02:12 PM

Hi Soroban I think that your matrix multiplication is incorrect, for example by hand and in MATLAB I got P^(2n) = P^2 = [1 0 0;0 1/2 1/2;0 1/2 1/2] and P^(2n+1) = P^3 = [0 1/2 1/2 ; 1 0 0 ; 1 0 0]. But I now understand what it means to find the state distribution after an odd or even amount of steps. So thanks for that

Also what does it mean by the limit distribution (using the above results we have established) I can't seem to decrypt the notes but I have a feeling the limit distribution does not exist.. Not sure how to show this however.

Thanks for your fast reply Soroban.

**Moo** · May 9th 2012, 11:33 PM

Originally Posted by liedora

Hi Soroban I think that your matrix multiplication is incorrect, for example by hand and in MATLAB I got P^(2n) = P^2 = [1 0 0;0 1/2 1/2;0 1/2 1/2] and P^(2n+1) = P^3 = [0 1/2 1/2 ; 1 0 0 ; 1 0 0]. But I now understand what it means to find the state distribution after an odd or even amount of steps. So thanks for that

Also what does it mean by the limit distribution (using the above results we have established) I can't seem to decrypt the notes but I have a feeling the limit distribution does not exist.. Not sure how to show this however.

Thanks for your fast reply Soroban.

Yes, your calculations are correct.
For the limit distribution, just think of it as a limit in normal times. After a certain index N, you should have something approaching the limiting distribution. But here, it's oscillating, you'll never get a unique limit.
If you want to do it formally, just look for how one defines the limiting distribution of a Markov chain.

Thread: Markov Chain

LinkBack

Thread Tools

Display

Markov Chain

Re: Markov Chain

Re: Markov Chain

Re: Markov Chain

Similar Math Help Forum Discussions

Markov Chain of random variables from a primitive markov chain

Markov Chain

Markov chain

Markov Chain

how to show that the X_kn is a markov chain if X_n is a markov chain

Search Tags