Expected Value Markov Chain

**joestevens** · February 13th 2011, 02:30 PM

A Markov chain $\{X_n , n\geq 0\}$ with states $0, 1, 2$ , has the transition probability matrix
$\left[ \begin{array}{ccc} \frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ 0 & \frac{1}{3} & \frac{2}{3} \\ \frac{1}{2} & 0 & \frac{1}{2} \end{array} \right] $
If $P(X_0 = 0)=P(X_0 = 1)=\frac{1}{4}$ , find $E(X_3)$ .

Is this just as simple as taking $\left[ \begin{array}{ccc} \frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ 0 & \frac{1}{3} & \frac{2}{3} \\ \frac{1}{2} & 0 & \frac{1}{2} \end{array} \right]^3 \left[ \begin{array}{c} \frac{1}{4} \\ \frac{1}{4} \\ \frac{1}{2} \end{array} \right] $ ? If not, how should I go about this?

**Moo** · February 13th 2011, 11:57 PM

Hello,

To get the probabilities of state 3, it's rather

$\left[ \begin{array}{c} \frac{1}{4} & \frac{1}{4} & \frac{1}{2} \end{array} \right] \left[ \begin{array}{ccc} \frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ 0 & \frac{1}{3} & \frac{2}{3} \\ \frac{1}{2} & 0 & \frac{1}{2} \end{array} \right]^3$

And in order to get $\displaystyle E[X_3]=\sum_{j=0}^2 jP(X_3=j)$ , multiply the result on the right by $\begin{bmatrix} 0\\1\\2\end{bmatrix}$ (see the code to have a simpler way to write matrices)

**joestevens** · February 14th 2011, 06:09 AM

Do you mean $\left[ \begin{array}{c} \frac{1}{4} & \frac{1}{4} & \frac{1}{2} \end{array} \right]^T \left[ \begin{array}{ccc} \frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ 0 & \frac{1}{3} & \frac{2}{3} \\ \frac{1}{2} & 0 & \frac{1}{2} \end{array} \right]^3\begin{bmatrix} 0\\1\\2\end{bmatrix}$ ? (Notice transpose)

**Moo** · February 14th 2011, 10:07 AM

Yep sorry, I forgot the transpose, but that's what you have to compute =)

Thread: Expected Value Markov Chain

LinkBack

Thread Tools

Display

Expected Value Markov Chain

Similar Math Help Forum Discussions

Markov Chain of random variables from a primitive markov chain

Expected value of markov chain URGENT!

Expected Value and markov conditional distribution

markov chain

how to show that the X_kn is a markov chain if X_n is a markov chain

Search Tags