Thread: Markov Chain Transition Probabilities Help.

Hi. For a project I am using a Markov Chain model with 17 states. I have used data to estimate transition probabilities. From these transition probabilities I can get n-step transition probabilities, i.e. the probability of going from state i to state j in exactly n steps. I would like to calculate the probability of going from state i to state j within n steps. I have searched through my literature but can't find a way to do this. Can anybody offer any help?

Thanks in advance,
Chloe.

Hi

First you form a matrix P (dimensions k x k where k is the number of possible states) with entries $\displaystyle p_{i,j}=$ the probability of going from state i to state j in one step. Then if you raise the matrix to the power of n, the entry of the final matrix in the ith row and jth column will be the probability of going from state i to state j in exactly n steps. If you want to get the probability of going from state i to step j in at most n steps, then you need to get the entry from the ith row and the jth column of the matrix:

$\displaystyle S=P+P^2+\cdots+P^n=\left ( P^{n+1}-P \right )\cdot\left ( P-I \right )^{-1}$

That is exactly what I was looking for! Thanks so much.

You are welcome.