# Conditional expectation problem

• Jan 5th 2011, 09:27 AM
LHS
Conditional expectation problem
Hi,
I'd be rather grateful if anyone could help me with this? Thanks :)

http://img253.imageshack.us/img253/5672/91409074.png
• Jan 6th 2011, 05:42 AM
Opalg
I don't know how to do this by using conditional expectations. My method would be to use the theory of Markov processes.

The transition matrix for this process is $\displaystyle \begin{bmatrix}0&\scriptstyle 1/3&0&1&0&0\\ \scriptstyle 1/2&0&1&0&\scriptstyle 1/2&0\\ 0&\scriptstyle 1/3&0&0&0&0\\ \scriptstyle 1/2&0&0&0&0&0\\ 0&\scriptstyle 1/3&0&0&0&0\\ 0&0&0&0&\scriptstyle 1/2&1\end{bmatrix}$, where the (i,j)-element gives the transition probability from room j to room i. If $\displaystyle R$ is the 5x5 submatrix obtained by deleting the final row and column, then $\displaystyle I-R = \begin{bmatrix}1&\scriptstyle -1/3&0&-1&0\\ \scriptstyle -1/2&1&-1&0&\scriptstyle -1/2\\ 0&\scriptstyle -1/3&1&0&0\\ \scriptstyle -1/2&0&0&1&0\\ 0&\scriptstyle -1/3&0&0&1\end{bmatrix}$. The inverse of $\displaystyle I-R$ is the so-called fundamental matrix of the process, $\displaystyle (I-R)^{-1} = \begin{bmatrix}6&4&4&6&2\\ 6&6&6&6&3\\ 2&2&3&2&1\\ 3&2&2&4&1\\ 2&2&2&2&2\end{bmatrix}$.

According to the theory of absorbing Markov systems (see here, for example), the (i,j)-entry in the fundamental matrix gives the expected duration of being in state i, having started in state j. So if the mouse starts in room 1, then (reading off the numbers in the first column of the fundamental matrix) it can expect to spend a total of 6 minutes in each of rooms 1 and 2, 2 minutes in each of rooms 3 and 5, and 3 minutes in room 4, before disappearing into the fatal room 6. That gives it a total expected life of 6+6+2+3+2 = 19 minutes. (That is much longer than I would have guessed – I hope I haven't got the arithmetic wrong.)
• Jan 7th 2011, 05:24 AM
LHS
Wow! thank you for your detailed answer, I do believe 19 is correct. Surprising isn't it!