This is the one step ahead property.. I think you just have to show an inductive proof.
For a Markov chain , show that
I know what the Markov property is, but I am worthless at manipulating contidional probabilities like this one. Any place to start would be helpful.
I write down the definition of conditional probability, and no matter which way I try to manipulate it I end up with a gigantic mess. Any help would be appreciated.