game theory,transition matrix
1.a gambler with 2 dollar makes a series of one-dollar bets. His probability of winning one dollar is 2/5. He either wins one dollar or loses one dollar, he decides to quit playing as soon as he gains 4 dollars or loses 2 dollars.
Let Xn be the amoun the Gambler is having at time n.
Write down the one-step transition matrix for this process.
2. Suppose 2 black and 2 white balls are placed in 2 boxes so that each box contains 2 balls. the number of the black balls in the first box is the state of the system. From each of the boxes, one ball is selected at random and the 2 balls are put back in the opposite boxes. Treating this process as a markov chain, find the transition matrix P.