Results 1 to 4 of 4

Math Help - Markov chain

  1. #1
    Junior Member
    Joined
    Dec 2006
    Posts
    43

    Markov chain

    I have the following transition matrix for a Markov chain.

    \L A\;=\;\begin{pmatrix}0 \,&\, \frac{1}{3} \,&\, 0 \,&\, \frac{1}{3} \,&\, \frac{1}{3} \,&\, 0 \,&\, 0 \,&\, 0 \,&\, 0 \\<br />
\frac{1}{5} & 0 & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & 0 & 0 & 0 \\<br />
0 & \frac{1}{3} & 0 & 0 & \frac{1}{3} & \frac{1}{3} & 0 & 0 & 0 \\<br /> <br />
\frac{1}{5} & \frac{1}{5} & 0 & 0 & \frac{1}{5} & 0 & \frac{1}{5} & \frac{1}{5} & 0 \\<br />
\frac{1}{8} & \frac{1}{8} & \frac{1}{8} & \frac{1}{8} & 0 & \frac{1}{8} & \frac{1}{8} & \frac{1}{8} & \frac{1}{8} \\<br />
0 & \frac{1}{5} & \frac{1}{5} & 0 & \frac{1}{5} & 0 & 0 & \frac{1}{5} & \frac{1}{5} \\<br /> <br />
0 & 0 & 0 & \frac{1}{3} & \frac{1}{3} & 0 & 0 & \frac{1}{3} & 0 \\<br />
0 & 0 & 0 & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & 0 & \frac{1}{5} \\<br />
0 & 0 & 0 & 0 & \frac{1}{3} & \frac{1}{3} & 0 & \frac{1}{3} & 0<br />
\end{pmatrix} [/size]

    This represents the problem:
    A child is standing in a square on the grid (namely x).
    Once time per second, he jumps to another square that is adjacent to x.

    The grid looks like this:[/size]
    Code:
        * - * - * - *
        | 1 | 2 | 3 |
        * - * - * - *
        | 4 | 5 | 6 |
        * - * - * - *
        | 7 | 8 | 9 |
        * - * - * - *
    [size=14]

    My question is, how do I find the steady state vector for the Markov chain?
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Senior Member
    Joined
    Apr 2006
    Posts
    399
    Awards
    1
    Quote Originally Posted by buckaroobill View Post
    I have the following transition matrix for a Markov chain.

    \L A\;=\;\begin{pmatrix}0 \,&\, \frac{1}{3} \,&\, 0 \,&\, \frac{1}{3} \,&\, \frac{1}{3} \,&\, 0 \,&\, 0 \,&\, 0 \,&\, 0 \\<br />
\frac{1}{5} & 0 & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & 0 & 0 & 0 \\<br />
0 & \frac{1}{3} & 0 & 0 & \frac{1}{3} & \frac{1}{3} & 0 & 0 & 0 \\<br /> <br />
\frac{1}{5} & \frac{1}{5} & 0 & 0 & \frac{1}{5} & 0 & \frac{1}{5} & \frac{1}{5} & 0 \\<br />
\frac{1}{8} & \frac{1}{8} & \frac{1}{8} & \frac{1}{8} & 0 & \frac{1}{8} & \frac{1}{8} & \frac{1}{8} & \frac{1}{8} \\<br />
0 & \frac{1}{5} & \frac{1}{5} & 0 & \frac{1}{5} & 0 & 0 & \frac{1}{5} & \frac{1}{5} \\<br /> <br />
0 & 0 & 0 & \frac{1}{3} & \frac{1}{3} & 0 & 0 & \frac{1}{3} & 0 \\<br />
0 & 0 & 0 & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & 0 & \frac{1}{5} \\<br />
0 & 0 & 0 & 0 & \frac{1}{3} & \frac{1}{3} & 0 & \frac{1}{3} & 0<br />
\end{pmatrix} [/size]

    This represents the problem:
    A child is standing in a square on the grid (namely x).
    Once time per second, he jumps to another square that is adjacent to x.

    The grid looks like this:[/size]
    Code:
        * - * - * - *
        | 1 | 2 | 3 |
        * - * - * - *
        | 4 | 5 | 6 |
        * - * - * - *
        | 7 | 8 | 9 |
        * - * - * - *
    [size=14]

    My question is, how do I find the steady state vector for the Markov chain?
    Do a google on steady state vector markov chain and in the Wikipedia entry you'll find two ways to calculate it, as a limit and as an eigenvector.
    Last edited by JakeD; March 13th 2007 at 11:39 PM.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Grand Panjandrum
    Joined
    Nov 2005
    From
    someplace
    Posts
    14,972
    Thanks
    4
    Quote Originally Posted by JakeD View Post
    Do a google on steady state vector markov chain and in the Wikipedia entry you'll find two ways to calculate it, as a limit and as an eigenvector.
    Isn't that limit just another way of calculating that eigen vector?

    (Rhetorical question I know the answer, and I know that you know the answer)

    RonL
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Senior Member
    Joined
    Apr 2006
    Posts
    399
    Awards
    1
    Quote Originally Posted by CaptainBlack View Post
    Isn't that limit just another way of calculating that eigen vector?

    (Rhetorical question I know the answer, and I know that you know the answer)

    RonL
    I don't disagree, but ...

    There are two ways of looking at this that are essentially mathematically equivalent.

    1) A steady state is an eigenvector x from x = xP. The iterative method x(n+1) = x(n)P will converge to x if P is regular.

    2) A steady state is the limit of x(n+1) = x(n)P. If P is regular, the limit exists and is an eigenvector x from x = xP.

    The second way is what is presented in the Wikipedia page. I favor this way because the iteration can be implemented simply on a computer without introducing the machinery of eigenvectors.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Markov Chain of random variables from a primitive markov chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: October 19th 2011, 08:12 AM
  2. Markov Chain Help
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: June 28th 2010, 07:37 AM
  3. Markov Chain
    Posted in the Discrete Math Forum
    Replies: 0
    Last Post: December 12th 2009, 04:52 PM
  4. Markov Chain HELP!!!!
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: April 9th 2009, 09:28 PM
  5. Replies: 2
    Last Post: October 28th 2008, 06:32 PM

Search Tags


/mathhelpforum @mathhelpforum