
Markov Chains Help
hello i would like u to help me with this markov chains problem thanks for ur help
The bestselling college statistics text, The Thrill of Statistics, sells 5 million copies every fall. Some users keep the book, and some sell it back to the bookstore. Suppose that 90% of all students who buy a new book sell it back, 80% of all students who buy a onceused book sell it back, and 60% of all students who buy a twiceused book sell it back. If a book has been used four or more times, the cover falls off, and it cannot be sold back.
a In the steady state, how many new copies of the book will the publisher be able to sell each year?
b Suppose that a bookstore’s proﬁt on each type of book is as follows:
New book: $6
Onceused book: $3
Twiceused book: $2
Thriceused book: $1
If the steadystate census is representative of the bookstore’s sales, what will be its average proﬁt per book?

Re: Markov Chains Help
So if they sell it back they do so before next fall?
So, let a_{n}, b_{n}, c_{n} and d_{n} be the number of new, secondhand, thirdhand and fourthhand books sold during fall in year n.
a_{0} = 5 million; b_{0} = c_{0} = d_{0} = 0.
a_{1} = 5 million  b_{1}; b_{1} = 0.9 a_{0}; c_{1} = d_{1} = 0.
a_{2} = 5 million  b_{2}  c_{2}; b_{2} = 0.9 a_{1}; c_{2} = 0.8 b_{1}; d_{2} = 0.
a_{3} = 5 million  b_{3}  c_{3}  d_{3}; b_{3} = 0.9 a_{2}; c_{3} = 0.8 b_{2}; d_{3} = 0.6 c_{2}.
Or eliminate b, c and d to get a_{n} in terms of a_{n1}, a_{n2} and a_{n3}.
Make a spreadsheet from either to see it stabilise at a_{60}.
Homogenise the second as directed here Recurrence relation  Wikipedia, the free encyclopedia, and you can make a transition matrix,
$\displaystyle \left( \begin{array}{cccc}0 & 1 & 0 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1 \\0.432 & 0.238 & 0.18 & 0.1 \end{array} \right)$