# Math Help - Discrete time Markov Chain - Long-term frequency

1. ## Discrete time Markov Chain - Long-term frequency

I just want to confirm something regarding this question. Since self transitions are allowed does that mean, for example, state 1 has 1/2 probability of self transitioning and 1/2 probability of transitioning to state 4? Similarly for state 4, it has 1/4 probability of transitioning to itself, state 1, state 5, and state 7, respectively, correct?

2. ## Re: Discrete time Markov Chain - Long-term frequency

Hello, usagi_killer!

Code:
      *-------*-------*-------*
|       |       |       |
|   1   |   2   |   3   |
|       |       |       |
*--   --*--   --*--   --*
|       |       |       |
|   4       5       6   |
|       |       |       |
*--   --*--   --*--   --*
|       |       |       |
|   7   |   8   |   9   |
|       |       |       |
*-------*-------*-------*
I just want to confirm something regarding this question.
Since self-transitions are allowed does that mean, for example,
state 1 has 1/2 probability of self transitioning and 1/2 probability of transitioning to state 4?

Similarly for state 4, it has 1/4 probability of transitioning to itself,
state 1, state 5, and state 7, respectively, correct? .Yes!

I assume that we are to find the transition matrix.

. . . $\begin{bmatrix}\frac{1}{2} &0&0& \frac{1}{2}&0&0&0&0&0\\ 0& \frac{1}{2} &0&0& \frac{1}{2} &0&0&0&0 \\ 0&0&\frac{1}{2}&0&0&\frac{1}{2} &0&0&0 \\ \frac{1}{4} &0&0& \frac{1}{4} & \frac{1}{4} &0& \frac{1}{4} &0&0 \\ 0&\frac{1}{5} &0&\frac{1}{5}&\frac{1}{5}&\frac{1}{5} &0&\frac{1}{5} & 0 \\ 0&0&\frac{1}{4} &0& \frac{1}{4}&\frac{1}{4} &0&0&\frac{1}{4} \\ 0&0&0&\frac{1}{2} &0&0&0&\frac{1}{2}&0\\ 0&0&0&0&\frac{1}{2}&0&0&\frac{1}{2}&0 \\ 0&0&0&0&0&\frac{1}{2}&0&0&\frac{1}{2} \end{bmatrix}$

Do we agree?

3. ## Re: Discrete time Markov Chain - Long-term frequency

it's not stated that way but it's a legitimate assumption and in the absence of a further specification of the probabilities involved it's the one you should go with.

But in general "at random" does not equate to "with equal probability"