## Discrete Time Markov Chain

Hi everyone, this is my first post on the math help forum so I hope you can help. Here is the problem I am having trouble on:

Suppose we model a communications channel as a discrete time queue. Time is divided into ﬁxed lengths called slots. The packets are of ﬁxed length, and the time needed to transmit one packet is exactly the length of one slot. If there are any packets awaiting transmission at the beginning of a time slot, exactly one packet is transmitted during the slot. During the slot, a random number of new packets may arrive that need transmission. The number of arrivals in the time slots are i.i.d. Poisson random variables with mean λ > 0. Packets awaiting or being transmitted wait in a buﬀer. The buﬀer can hold at most 4 packets including any being transmitted. Packets that overﬂow are lost. Model the above process as a Markov chain X0 , X1 , . . . . (By model, I mean give the state space, describe what Xn = i means for all states i and times n, and give the transition matrix P . It might be convenient to deﬁne quantities like pk = e−λ λk /k! and qk = ∑ pl (sum from k= l to infinity)

Please let me know if you have any suggestions, it would be greatly appreciated because I'm stumped!