Markov Chain Transition Probabilities Help.

**chloe08276** · Sep 4th 2012, 04:37 AM

Hi. For a project I am using a Markov Chain model with 17 states. I have used data to estimate transition probabilities. From these transition probabilities I can get n-step transition probabilities, i.e. the probability of going from state i to state j in exactly n steps. I would like to calculate the probability of going from state i to state j within n steps. I have searched through my literature but can't find a way to do this. Can anybody offer any help?

Thanks in advance,
Chloe.

**anonimnystefy** · Sep 4th 2012, 08:16 AM

Hi

First you form a matrix P (dimensions k x k where k is the number of possible states) with entries $p_{i,j}=$ the probability of going from state i to state j in one step. Then if you raise the matrix to the power of n, the entry of the final matrix in the ith row and jth column will be the probability of going from state i to state j in exactly n steps. If you want to get the probability of going from state i to step j in at most n steps, then you need to get the entry from the ith row and the jth column of the matrix:

$S=P+P^2+\cdots+P^n=\left ( P^{n+1}-P \right )\cdot\left ( P-I \right )^{-1}$

**chloe08276** · Sep 4th 2012, 08:20 AM

That is exactly what I was looking for! Thanks so much.

**anonimnystefy** · Sep 4th 2012, 08:32 AM

You are welcome.

Thread: Markov Chain Transition Probabilities Help.

LinkBack

Thread Tools

Display

Markov Chain Transition Probabilities Help.

Re: Markov Chain Transition Probabilities Help.

Re: Markov Chain Transition Probabilities Help.

Re: Markov Chain Transition Probabilities Help.

Similar Math Help Forum Discussions

Markov chain transition matrix

Markov chain transition matrix

Transition matrix for Markov chain

Markov Chains & Transition Probabilities

Markov Chain (transition matrix)

Search Tags