# Math Help - game theory,transition matrix

1. ## game theory,transition matrix

1.a gambler with 2 dollar makes a series of one-dollar bets. His probability of winning one dollar is 2/5. He either wins one dollar or loses one dollar, he decides to quit playing as soon as he gains 4 dollars or loses 2 dollars.
Let Xn be the amoun the Gambler is having at time n.
Write down the one-step transition matrix for this process.

2. Suppose 2 black and 2 white balls are placed in 2 boxes so that each box contains 2 balls. the number of the black balls in the first box is the state of the system. From each of the boxes, one ball is selected at random and the 2 balls are put back in the opposite boxes. Treating this process as a markov chain, find the transition matrix P.

2. Originally Posted by abiola
1.a gambler with 2 dollar makes a series of one-dollar bets. His probability of winning one dollar is 2/5. He either wins one dollar or loses one dollar, he decides to quit playing as soon as he gains 4 dollars or loses 2 dollars.
Let Xn be the amoun the Gambler is having at time n.
Write down the one-step transition matrix for this process.
The possible state of the game is that the player has 0, 1, 2, ..., 6 dollars.
The 0 and 6 dollar states are absorbing.

So lets represent the game after $n$ plays as a column vector the $i$ th element being the probability that the game is in sate $i$. Then after the $n$ th play the probabilities satisfy::

$
p(i,n)=
\begin{cases}
p(0,n-1)+(3/5)p(1,n-1), & i=0\\
p(6,n-1)+(2/5)p(5,n-1), & i=6\\
(2/5)p(i-1,n-1)+(3/5)p(i+1,n-1),& \text{otherwise}
\end{cases}
$

which give the 1-step transitions form which you should be able to calculate the 1-step transition matrix.

CB