The question i am stuck on is on this exercise sheet
http://www.maths.qmul.ac.uk/~ig/MAS338/ex2-07.pdf
Its question 1.
Basically i have done the transition graph, but i dont know how to calculate the probabilities.
I dont just want the answer, i wanna know the method of how to calculate them
Thankyou
Looking at the question again I note that the transition matrix given is the transpose of what I would expect for the transition matrix so my is the transpose of the given matrix. This is a result of differing conventions on the state probability vectors, if you use row vectors you post multiply them by the transition matrix to get the probability vector at the next epoc and you have a right stochastic matrix as in your question. I normall use column vector and so use the other convention.
Look at (i) you are asked for the Probability that given that .
If then:
So is the second element of this vector of probabilities so is .
RonL