Help on Markov chains to calculate the mean time to reach a particular state

• Mar 26th 2013, 07:53 PM
Koustubh
Help on Markov chains to calculate the mean time to reach a particular state
http://mathhelpforum.com/image/png;b...AAAElFTkSuQmCC
Hello guys,

I have a 2D markov chain as shown above with 4 states. I need to calculate the mean time to reach the state '11' from state '00'. Could please someone help me with this.

Thanks and Regards,
Koustubh
• Mar 27th 2013, 10:48 AM
Soroban
Re: Help on Markov chains to calculate the mean time to reach a particular state
Hello, Koustubh!

Without numbers, the problem is virtually impossible.
Even with numbers, the solution would fill volumes.

Quote:

I have a 2D markov chain with 4 states.
I need to calculate the mean time to reach the state '11' from state '00'.
Could please someone help me with this?

According to your diagram, we seem to have this transition chart.

. . $\begin{array}{c|cccc|} & 00 & 01 & 10 & 11 \\ \hline 00 & i & a & g & 0 \\ 01 & b & j & 0 & c \\ 10 & h & 0 & l & e \\ 11 & 0 & d & f & k \\ \hline\end{array}$

Let $A \:=\:\begin{pmatrix}i&a&g&0 \\ b&j&0&c \\ h&0&l&e \\ 0&d&f&k \end{pmatrix}$

We want the probability of going from $00$ to $11$ in $n$ steps.

$\begin{array}{ccccc}n=2\!: & \text{ Calculate }A^2. & \text{ Examine the term }a_{14} \\ n=3\!: & \text{ Calculate }A^3. & \text{ Examine the term }a_{14} \\ n=4\!: & \text{ Calculate }A^4. & \text{ Examine the term }a_{14} \\ n=5\!: & \text{ Calculate }A^5. & \text{ Examine the term }a_{14} \\ \vdots & \vdots & \vdots \end{array}$

Do you see the difficulty?