Markov Chain HELP!!!!

**stephy7878** · April 9th 2009, 08:22 PM

Hello,
Not sure where to put this question and let me know if it needs to be put somewhere else... I'm learning about Markov Chain Matrices and I am a little confused on how to take a transition matrix and finding the fixed probability vector??? Here is the problem.. let me know if you can help me out..

This is a transition matrix

0.375 0.625 0
0.375 0.375 0.25
0.375 0.5 0.125

Find the fixed probability vector. I'm not really sure how to start this I'm really confused. Any ideas?

The result should be a 1x3 Martix

**Soroban** · April 9th 2009, 09:28 PM

Hello, stephy7878!

Given the transition matrix: . $A \;=\;\begin{pmatrix}\frac{3}{8} & \frac{5}{8} & 0 \\ \\[-4mm] \frac{3}{8} & \frac{3}{8} & \frac{1}{4} \\ \\[-4mm] \frac{3}{8} & \frac{1}{2} & \frac{1}{8} \end{pmatrix}$

Find the fixed probability vector.

We want a row vector: . $v \:=\:(x,y,z)\:\text{ so that }\:v\!\cdot\! A \:=\:v$

We have: . $(x,y,z)\,\begin{pmatrix}\frac{3}{8}&\frac{5}{8}&0 \\ \\[-4mm] \frac{3}{8} & \frac{3}{8} & \frac{1}{4} \\ \\[-4mm] \frac{3}{8} & \frac{1}{2} & \frac{1}{8}\end{pmatrix} \;=\;(x,y,z)$

Multiply: . $\begin{array}{ccc}\frac{3}{8}x + \frac{3}{8}y + \frac{3}{8}z &=& x \\ \\[-4mm] \frac{5}{8}x + \frac{3}{8}y + \frac{1}{2}z &=& y \\ \\[-4mm] \qquad \frac{1}{4}y + \frac{1}{8}z &=& z \end{array}$

The equations simplify to: . $\begin{array}{cccc}\text{-}5x + 3y + 3z &=& 0 & {\color{blue}[1]} \\ 5x - 5y + 4z &=& 0 & {\color{blue}[2]} \\ \qquad 2y - 7z &=& 0 & {\color{blue}[3]} \end{array}$

Add [1] and [2]: . $-2y + 7z \:=\:0$ . . . whch is equivalent to [3].

So we need another equaton.
Here it is! . . . $x + y + z \;=\;1\;\;{\color{blue}[4]}$ . . . . This is true for all fixed probability vectors.

From [3], we have: . $y \:=\:\tfrac{7}{2}z$

Substitute into [2]: . $5x - 5\left(\tfrac{7}{2}z\right) + 4z \:=\:0 \quad\Rightarrow\quad x \:=\:\tfrac{27}{10}z$

Substitute into [4]: . $\tfrac{27}{10}z + \tfrac{7}{2}z + z \:=\:1 \quad\Rightarrow\quad z \:=\:\frac{5}{36} $
. . . Then: . $x \:=\:\tfrac{27}{10}\left(\tfrac{5}{36}\right) \quad\Rightarrow\quad x \:=\:\frac{3}{8}$
. . And: . $y \:=\:\tfrac{7}{2}\left(\tfrac{5}{36}\right) \quad\Rightarrow\quad y \:=\:\frac{35}{72}$

Therefore, the fixed probabiity vector is: . $v \;=\;\left(\frac{3}{8},\:\frac{35}{72},\:\frac{5}{ 36}\right)$

Thread: Markov Chain HELP!!!!

LinkBack

Thread Tools

Display

Markov Chain HELP!!!!

Similar Math Help Forum Discussions

Markov Chain of random variables from a primitive markov chain

Markov Chain Help

Markov Chain

Markov Chain

how to show that the X_kn is a markov chain if X_n is a markov chain

Search Tags