steady state vector markov chain and in the Wikipedia entry you'll find two ways to calculate it, as a limit and as an eigenvector.
I have the following transition matrix for a Markov chain.
This represents the problem:
A child is standing in a square on the grid (namely x).
Once time per second, he jumps to another square that is adjacent to x.
The grid looks like this:[/size][size=14]Code:* - * - * - * | 1 | 2 | 3 | * - * - * - * | 4 | 5 | 6 | * - * - * - * | 7 | 8 | 9 | * - * - * - *
My question is, how do I find the steady state vector for the Markov chain?
There are two ways of looking at this that are essentially mathematically equivalent.
1) A steady state is an eigenvector x from x = xP. The iterative method x(n+1) = x(n)P will converge to x if P is regular.
2) A steady state is the limit of x(n+1) = x(n)P. If P is regular, the limit exists and is an eigenvector x from x = xP.
The second way is what is presented in the Wikipedia page. I favor this way because the iteration can be implemented simply on a computer without introducing the machinery of eigenvectors.