Thread: game theory,transition matrix

1.a gambler with 2 dollar makes a series of one-dollar bets. His probability of winning one dollar is 2/5. He either wins one dollar or loses one dollar, he decides to quit playing as soon as he gains 4 dollars or loses 2 dollars.
Let Xn be the amoun the Gambler is having at time n.
Write down the one-step transition matrix for this process.

2. Suppose 2 black and 2 white balls are placed in 2 boxes so that each box contains 2 balls. the number of the black balls in the first box is the state of the system. From each of the boxes, one ball is selected at random and the 2 balls are put back in the opposite boxes. Treating this process as a markov chain, find the transition matrix P.

Originally Posted by abiola

1.a gambler with 2 dollar makes a series of one-dollar bets. His probability of winning one dollar is 2/5. He either wins one dollar or loses one dollar, he decides to quit playing as soon as he gains 4 dollars or loses 2 dollars.
Let Xn be the amoun the Gambler is having at time n.
Write down the one-step transition matrix for this process.

The possible state of the game is that the player has 0, 1, 2, ..., 6 dollars.
The 0 and 6 dollar states are absorbing.

So lets represent the game after plays as a column vector the th element being the probability that the game is in sate . Then after the th play the probabilities satisfy::




which give the 1-step transitions form which you should be able to calculate the 1-step transition matrix.

CB