# Markov Chain HELP!!!!

• Apr 9th 2009, 08:22 PM
stephy7878
Markov Chain HELP!!!!
Hello,
Not sure where to put this question and let me know if it needs to be put somewhere else... I'm learning about Markov Chain Matrices and I am a little confused on how to take a transition matrix and finding the fixed probability vector??? Here is the problem.. let me know if you can help me out..

This is a transition matrix

0.375 0.625 0
0.375 0.375 0.25
0.375 0.5 0.125

Find the fixed probability vector. I'm not really sure how to start this I'm really confused. Any ideas?

The result should be a 1x3 Martix
• Apr 9th 2009, 09:28 PM
Soroban
Hello, stephy7878!

Quote:

Given the transition matrix: .$\displaystyle A \;=\;\begin{pmatrix}\frac{3}{8} & \frac{5}{8} & 0 \\ \\[-4mm] \frac{3}{8} & \frac{3}{8} & \frac{1}{4} \\ \\[-4mm] \frac{3}{8} & \frac{1}{2} & \frac{1}{8} \end{pmatrix}$

Find the fixed probability vector.

We want a row vector: .$\displaystyle v \:=\:(x,y,z)\:\text{ so that }\:v\!\cdot\! A \:=\:v$

We have: .$\displaystyle (x,y,z)\,\begin{pmatrix}\frac{3}{8}&\frac{5}{8}&0 \\ \\[-4mm] \frac{3}{8} & \frac{3}{8} & \frac{1}{4} \\ \\[-4mm] \frac{3}{8} & \frac{1}{2} & \frac{1}{8}\end{pmatrix} \;=\;(x,y,z)$

Multiply: .$\displaystyle \begin{array}{ccc}\frac{3}{8}x + \frac{3}{8}y + \frac{3}{8}z &=& x \\ \\[-4mm] \frac{5}{8}x + \frac{3}{8}y + \frac{1}{2}z &=& y \\ \\[-4mm] \qquad \frac{1}{4}y + \frac{1}{8}z &=& z \end{array}$

The equations simplify to: .$\displaystyle \begin{array}{cccc}\text{-}5x + 3y + 3z &=& 0 & {\color{blue}[1]} \\ 5x - 5y + 4z &=& 0 & {\color{blue}[2]} \\ \qquad 2y - 7z &=& 0 & {\color{blue}[3]} \end{array}$

Add [1] and [2]: .$\displaystyle -2y + 7z \:=\:0$ . . . whch is equivalent to [3].

So we need another equaton.
Here it is! . . . $\displaystyle x + y + z \;=\;1\;\;{\color{blue}[4]}$ .
. . . This is true for all fixed probability vectors.

From [3], we have: .$\displaystyle y \:=\:\tfrac{7}{2}z$

Substitute into [2]: .$\displaystyle 5x - 5\left(\tfrac{7}{2}z\right) + 4z \:=\:0 \quad\Rightarrow\quad x \:=\:\tfrac{27}{10}z$

Substitute into [4]: .$\displaystyle \tfrac{27}{10}z + \tfrac{7}{2}z + z \:=\:1 \quad\Rightarrow\quad z \:=\:\frac{5}{36}$
. . . Then: .$\displaystyle x \:=\:\tfrac{27}{10}\left(\tfrac{5}{36}\right) \quad\Rightarrow\quad x \:=\:\frac{3}{8}$
. . And: .$\displaystyle y \:=\:\tfrac{7}{2}\left(\tfrac{5}{36}\right) \quad\Rightarrow\quad y \:=\:\frac{35}{72}$

Therefore, the fixed probabiity vector is: .$\displaystyle v \;=\;\left(\frac{3}{8},\:\frac{35}{72},\:\frac{5}{ 36}\right)$