is a discrete time markov chain with Transition matrix P
show that is a markov chain with transition matrix
Are you asking for a complete course in Markov chains?
BTW: Markov was real mathematician so please capitalize his name.
You can easily do a web-search on this topic.