Need help on markov chain question ?

**Rakesh** · April 27th 2008, 04:07 AM

The question i am stuck on is on this exercise sheet
http://www.maths.qmul.ac.uk/~ig/MAS338/ex2-07.pdf

Its question 1.
Basically i have done the transition graph, but i dont know how to calculate the probabilities.

I dont just want the answer, i wanna know the method of how to calculate them
Thankyou

**CaptainBlack** · April 27th 2008, 08:01 AM

Originally Posted by Rakesh

The question i am stuck on is on this exercise sheet
http://www.maths.qmul.ac.uk/~ig/MAS338/ex2-07.pdf

Its question 1.
Basically i have done the transition graph, but i dont know how to calculate the probabilities.

I dont just want the answer, i wanna know the method of how to calculate them
Thankyou

Let $T$ be the transition matrix. Then if the state probability vector at epoc $i$ is $P_i$ , then the state probability vector at epoc $i+k$ is $T^kP_i$

That is $P_i$ is a column vector the $j$ -th element of which is the probability that the state is the $j$ -th at epoc $i$ .

RonL

**Rakesh** · April 27th 2008, 08:28 AM

yh but what are the value of i,j, and P, could u give me an example ?

**CaptainBlack** · April 27th 2008, 08:42 AM

Originally Posted by Rakesh

yh but what are the value of i,j, and P, could u give me an example ?

Looking at the question again I note that the transition matrix given is the transpose of what I would expect for the transition matrix so my $T$ is the transpose of the given matrix. This is a result of differing conventions on the state probability vectors, if you use row vectors you post multiply them by the transition matrix to get the probability vector at the next epoc and you have a right stochastic matrix as in your question. I normall use column vector and so use the other convention.

Look at (i) you are asked for the Probability that $X_1=2$ given that $X_0=1$ .

If $X_0=1$ then:

$ P_0 = \left[ {\begin{array}{*{20}c} 1 \\ 0 \\ 0 \\ 0 \\ \end{array}} \right] $

$ P_1 = \left[ {\begin{array}{*{20}c} 0 & {1/2} & {1/2} & 0 \\ {1/3} & 0 & {1/3} & {1/3} \\ {1/3} & {1/3} & 0 & {1/3} \\ 0 & {1/2} & {1/2} & 0 \\ \end{array}} \right]^t \left[ {\begin{array}{*{20}c} 1 \\ 0 \\ 0 \\ 0 \\ \end{array}} \right] = \left[ {\begin{array}{*{20}c} 0 \\ {1/2} \\ {1/2} \\ 0 \\ \end{array}} \right] $

So $P(X_1=2|X_0=1)$ is the second element of this vector of probabilities so is $1/2$ .

RonL

Thread: Need help on markov chain question ?

LinkBack

Thread Tools

Display

Need help on markov chain question ?

Similar Math Help Forum Discussions

Markov Chain of random variables from a primitive markov chain

Markov chain question

Markov Chain Question

Markov Chain Question

how to show that the X_kn is a markov chain if X_n is a markov chain

Search Tags