Hello everyone
I have this problem which is not very difficult
A Markov process with state space S = {1,2,3,4,5,6} and the transition matrix P
[ 0 2/7 1/7 1/7 1/7 2/7 |
| 0 0 1/3 2/3 0 0 |
| 1 0 0 0 0 0 |
P = | 0 0 0 1 0 0 |
| 0 0 0 0 0 1 |
| 0 0 0 0 1 0 ]
Find the absorption probabilities associated with each of the closed classes.
Ok , The classes are:
{1,2,3}: non-closed, transient
{4}: closed, positive recurrent (absorbing state)
{5,6}: closed, recurrent
For each i, let y_i = P(process X(.) absorbed in state 4| X(0) = i)
Need to solve y = P.y ( invariant distribution)
We have y_5 = y_6 = 0
After a bunch of calculation, I got: y = (7/16, 2/16, 7/16, 0, 0, 0)
I am not sure about this answer because the sum of y_i must be equal to 1 for i = 1,2,3,4,5,6.
BUT, if you work out the probability for each individual state then P(process X(.) absorbed in state 4| X(0) = 4) = 1. So, the sum is more than 1. That's why I am confused because y_4 = 0 and 1 at the same time.
And P(process X(.) absorbed in class {5,6}| X(0) = i) = 1 - y_i for i = 1,2,3,4,5,6. (This is easy because there are 2 closed classes)
Thank you so much