# Thread: Expected Value Markov Chain

1. ## Expected Value Markov Chain

A Markov chain $\displaystyle \{X_n , n\geq 0\}$ with states $\displaystyle 0, 1, 2$, has the transition probability matrix
$\displaystyle \left[ \begin{array}{ccc} \frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ 0 & \frac{1}{3} & \frac{2}{3} \\ \frac{1}{2} & 0 & \frac{1}{2} \end{array} \right]$
If $\displaystyle P(X_0 = 0)=P(X_0 = 1)=\frac{1}{4}$, find $\displaystyle E(X_3)$.

Is this just as simple as taking $\displaystyle \left[ \begin{array}{ccc} \frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ 0 & \frac{1}{3} & \frac{2}{3} \\ \frac{1}{2} & 0 & \frac{1}{2} \end{array} \right]^3 \left[ \begin{array}{c} \frac{1}{4} \\ \frac{1}{4} \\ \frac{1}{2} \end{array} \right]$? If not, how should I go about this?

2. Hello,

To get the probabilities of state 3, it's rather

$\displaystyle \left[ \begin{array}{c} \frac{1}{4} & \frac{1}{4} & \frac{1}{2} \end{array} \right] \left[ \begin{array}{ccc} \frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ 0 & \frac{1}{3} & \frac{2}{3} \\ \frac{1}{2} & 0 & \frac{1}{2} \end{array} \right]^3$

And in order to get $\displaystyle \displaystyle E[X_3]=\sum_{j=0}^2 jP(X_3=j)$, multiply the result on the right by $\displaystyle \begin{bmatrix} 0\\1\\2\end{bmatrix}$ (see the code to have a simpler way to write matrices)

3. Do you mean $\displaystyle \left[ \begin{array}{c} \frac{1}{4} & \frac{1}{4} & \frac{1}{2} \end{array} \right]^T \left[ \begin{array}{ccc} \frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ 0 & \frac{1}{3} & \frac{2}{3} \\ \frac{1}{2} & 0 & \frac{1}{2} \end{array} \right]^3\begin{bmatrix} 0\\1\\2\end{bmatrix}$? (Notice transpose)

4. Yep sorry, I forgot the transpose, but that's what you have to compute =)