# Thread: Need help on markov chain question ?

1. ## Need help on markov chain question ?

The question i am stuck on is on this exercise sheet
http://www.maths.qmul.ac.uk/~ig/MAS338/ex2-07.pdf

Its question 1.
Basically i have done the transition graph, but i dont know how to calculate the probabilities.

I dont just want the answer, i wanna know the method of how to calculate them
Thankyou

2. Originally Posted by Rakesh
The question i am stuck on is on this exercise sheet
http://www.maths.qmul.ac.uk/~ig/MAS338/ex2-07.pdf

Its question 1.
Basically i have done the transition graph, but i dont know how to calculate the probabilities.

I dont just want the answer, i wanna know the method of how to calculate them
Thankyou
Let $T$ be the transition matrix. Then if the state probability vector at epoc $i$ is $P_i$, then the state probability vector at epoc $i+k$ is $T^kP_i$

That is $P_i$ is a column vector the $j$-th element of which is the probability that the state is the $j$-th at epoc $i$.

RonL

3. yh but what are the value of i,j, and P, could u give me an example ?

4. Originally Posted by Rakesh
yh but what are the value of i,j, and P, could u give me an example ?
Looking at the question again I note that the transition matrix given is the transpose of what I would expect for the transition matrix so my $T$ is the transpose of the given matrix. This is a result of differing conventions on the state probability vectors, if you use row vectors you post multiply them by the transition matrix to get the probability vector at the next epoc and you have a right stochastic matrix as in your question. I normall use column vector and so use the other convention.

Look at (i) you are asked for the Probability that $X_1=2$ given that $X_0=1$.

If $X_0=1$ then:

$
P_0 = \left[ {\begin{array}{*{20}c}
1 \\
0 \\
0 \\
0 \\
\end{array}} \right]
$

$
P_1 = \left[ {\begin{array}{*{20}c}
0 & {1/2} & {1/2} & 0 \\
{1/3} & 0 & {1/3} & {1/3} \\
{1/3} & {1/3} & 0 & {1/3} \\
0 & {1/2} & {1/2} & 0 \\
\end{array}} \right]^t \left[ {\begin{array}{*{20}c}
1 \\
0 \\
0 \\
0 \\
\end{array}} \right] = \left[ {\begin{array}{*{20}c}
0 \\
{1/2} \\
{1/2} \\
0 \\
\end{array}} \right]
$

So $P(X_1=2|X_0=1)$ is the second element of this vector of probabilities so is $1/2$.

RonL