Results 1 to 4 of 4

Math Help - Need help on markov chain question ?

  1. #1
    Newbie
    Joined
    Apr 2008
    Posts
    11

    Need help on markov chain question ?

    The question i am stuck on is on this exercise sheet
    http://www.maths.qmul.ac.uk/~ig/MAS338/ex2-07.pdf

    Its question 1.
    Basically i have done the transition graph, but i dont know how to calculate the probabilities.

    I dont just want the answer, i wanna know the method of how to calculate them
    Thankyou
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Grand Panjandrum
    Joined
    Nov 2005
    From
    someplace
    Posts
    14,972
    Thanks
    4
    Quote Originally Posted by Rakesh View Post
    The question i am stuck on is on this exercise sheet
    http://www.maths.qmul.ac.uk/~ig/MAS338/ex2-07.pdf

    Its question 1.
    Basically i have done the transition graph, but i dont know how to calculate the probabilities.

    I dont just want the answer, i wanna know the method of how to calculate them
    Thankyou
    Let T be the transition matrix. Then if the state probability vector at epoc i is P_i, then the state probability vector at epoc i+k is T^kP_i

    That is P_i is a column vector the j-th element of which is the probability that the state is the j-th at epoc i.

    RonL
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Apr 2008
    Posts
    11
    yh but what are the value of i,j, and P, could u give me an example ?
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Grand Panjandrum
    Joined
    Nov 2005
    From
    someplace
    Posts
    14,972
    Thanks
    4
    Quote Originally Posted by Rakesh View Post
    yh but what are the value of i,j, and P, could u give me an example ?
    Looking at the question again I note that the transition matrix given is the transpose of what I would expect for the transition matrix so my T is the transpose of the given matrix. This is a result of differing conventions on the state probability vectors, if you use row vectors you post multiply them by the transition matrix to get the probability vector at the next epoc and you have a right stochastic matrix as in your question. I normall use column vector and so use the other convention.

    Look at (i) you are asked for the Probability that X_1=2 given that X_0=1.

    If X_0=1 then:

    <br />
P_0  = \left[ {\begin{array}{*{20}c}<br />
   1  \\<br />
   0  \\<br />
   0  \\<br />
   0  \\<br />
\end{array}} \right]<br />


    <br />
P_1  = \left[ {\begin{array}{*{20}c}<br />
   0 & {1/2} & {1/2} & 0  \\<br />
   {1/3} & 0 & {1/3} & {1/3}  \\<br />
   {1/3} & {1/3} & 0 & {1/3}  \\<br />
   0 & {1/2} & {1/2} & 0  \\<br />
\end{array}} \right]^t \left[ {\begin{array}{*{20}c}<br />
   1  \\<br />
   0  \\<br />
   0  \\<br />
   0  \\<br />
\end{array}} \right] = \left[ {\begin{array}{*{20}c}<br />
   0  \\<br />
   {1/2}  \\<br />
   {1/2}  \\<br />
   0  \\<br />
\end{array}} \right]<br />

    So P(X_1=2|X_0=1) is the second element of this vector of probabilities so is 1/2.

    RonL
    Last edited by CaptainBlack; April 27th 2008 at 08:58 AM.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Markov Chain of random variables from a primitive markov chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: October 19th 2011, 08:12 AM
  2. Markov chain question
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: March 4th 2011, 10:52 AM
  3. Markov Chain Question
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: February 26th 2011, 07:39 PM
  4. Markov Chain Question
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: February 15th 2011, 04:42 PM
  5. Replies: 2
    Last Post: October 28th 2008, 06:32 PM

Search Tags


/mathhelpforum @mathhelpforum