Markov Chain - Transition Matrix
Describe the state space and the one-step transition probability matrix for the homogeneous Markov chain
below.
Two black balls and two white balls are placed in 2 urns so that each urn contains 2 balls. At each step 1 ball is selected at random from each urn. The 2 balls selected are interchanged. Let
denote the number of white balls initially in the first urn. For
, let
denote the number of white balls in the first urn after
interchanges have taken place.
Now just as i was halfway through typing this i had an idea on how to do it! So heres my answer. Is this correct?
)
And how i think this is explained is that say if there is 0 or 2 white balls in urn 1 on step n (rows 1 & 3).
Then on the n+1'th step a white ball and a black ball will swap urns and each urn will contain 1 black and 1 white ball so the amount of white balls will in urn 1 will be 1 with probability 1. This is shown in row 1 and 2.
For 1 white ball in urn 1 there is 4 different swaps that can happen. {WW, WB, BW, BB} and these will give probability 0.25 that there will be 0 or 2 white balls in urn 1 and 1 white ball in urn 1 with probability 0.5.
Is this making sense..?