# game theory,transition matrix

• Feb 25th 2009, 04:21 PM
abiola
game theory,transition matrix
1.a gambler with 2 dollar makes a series of one-dollar bets. His probability of winning one dollar is 2/5. He either wins one dollar or loses one dollar, he decides to quit playing as soon as he gains 4 dollars or loses 2 dollars.
Let Xn be the amoun the Gambler is having at time n.
Write down the one-step transition matrix for this process.

2. Suppose 2 black and 2 white balls are placed in 2 boxes so that each box contains 2 balls. the number of the black balls in the first box is the state of the system. From each of the boxes, one ball is selected at random and the 2 balls are put back in the opposite boxes. Treating this process as a markov chain, find the transition matrix P.
• Feb 26th 2009, 11:28 PM
CaptainBlack
Quote:

Originally Posted by abiola
1.a gambler with 2 dollar makes a series of one-dollar bets. His probability of winning one dollar is 2/5. He either wins one dollar or loses one dollar, he decides to quit playing as soon as he gains 4 dollars or loses 2 dollars.
Let Xn be the amoun the Gambler is having at time n.
Write down the one-step transition matrix for this process.

The possible state of the game is that the player has 0, 1, 2, ..., 6 dollars.
The 0 and 6 dollar states are absorbing.

So lets represent the game after $\displaystyle n$ plays as a column vector the $\displaystyle i$ th element being the probability that the game is in sate $\displaystyle i$. Then after the $\displaystyle n$ th play the probabilities satisfy::

$\displaystyle p(i,n)= \begin{cases} p(0,n-1)+(3/5)p(1,n-1), & i=0\\ p(6,n-1)+(2/5)p(5,n-1), & i=6\\ (2/5)p(i-1,n-1)+(3/5)p(i+1,n-1),& \text{otherwise} \end{cases}$

which give the 1-step transitions form which you should be able to calculate the 1-step transition matrix.

CB