# Markov Chain probabilities

• January 27th 2010, 10:40 AM
charikaar
Markov Chain probabilities
A Markov chain on state space {1,2,3,4,5} has transition matrix

$
\begin{pmatrix}
0 & 0 & 1 & 0 & 0\\
0 & 0 & 4/{5} & 1/{5} & 0\\
0 & 1/{6} & 2/{3} & 0 & 1/{6}\\
0 & 0 & 0 & 1 & 0\\
0 & 0 & 0 & 0 &1\\
\end{pmatrix}
$

The process starts in state 1.
a) Which states are absorbing?
b) Calculate the probability that the process is absorbed at state 4 (ends up at state 4).
c) Calculate the expectation of the time of absorption.

I am missed my lectures due illnes so I don't have much understanding of this topic. I would be grateful for some help with this revision question.

Thanks
• January 27th 2010, 11:31 AM
Moo
Hello,

A state is said to be absorbing if once you're in this state, you don't leave it anymore. x is absorbing if $P(X_{n+1}=x|X_n=x)=1$
When you're given the transition matrix, the absorbing states are the ones where there's a 1 in the diagonal.

For question 2), find all the possible paths that go from 1 to 4 (in any given number of steps) and add their probabilities. Maybe doing a sketch will help.

For question 3), which formula are you given ? You must have some lecture notes !
• January 27th 2010, 11:36 AM
charikaar
so state 4 and 5 are absorbing...
• January 27th 2010, 11:38 AM
Moo
Yes they are :)
• January 27th 2010, 12:30 PM
charikaar
for part (c). I think, I have to use $w_{i}=E(T|X_{0}=i)$

i know $w_{4}=w_{5}=0$

for part (b) $u_{i}=P(X_{A}=1|X_{0}=i)$

Can you help me to answer part b,c fully?

thanks