# Markov Chain Transition Probabilities Help.

• Sep 4th 2012, 04:37 AM
chloe08276
Markov Chain Transition Probabilities Help.
Hi. For a project I am using a Markov Chain model with 17 states. I have used data to estimate transition probabilities. From these transition probabilities I can get n-step transition probabilities, i.e. the probability of going from state i to state j in exactly n steps. I would like to calculate the probability of going from state i to state j within n steps. I have searched through my literature but can't find a way to do this. Can anybody offer any help?

Chloe.
• Sep 4th 2012, 08:16 AM
anonimnystefy
Re: Markov Chain Transition Probabilities Help.
Hi

First you form a matrix P (dimensions k x k where k is the number of possible states) with entries $p_{i,j}=$ the probability of going from state i to state j in one step. Then if you raise the matrix to the power of n, the entry of the final matrix in the ith row and jth column will be the probability of going from state i to state j in exactly n steps. If you want to get the probability of going from state i to step j in at most n steps, then you need to get the entry from the ith row and the jth column of the matrix:

$S=P+P^2+\cdots+P^n=\left ( P^{n+1}-P \right )\cdot\left ( P-I \right )^{-1}$
• Sep 4th 2012, 08:20 AM
chloe08276
Re: Markov Chain Transition Probabilities Help.
That is exactly what I was looking for! Thanks so much.
• Sep 4th 2012, 08:32 AM
anonimnystefy
Re: Markov Chain Transition Probabilities Help.
You are welcome. :)