Looking for ideas regarding Markov Chains question.
Hi, i have a problem regarding the calculation of the limit of pij(n) goin from a certain state 4 to a state 1 as n tends to infinity. I observed that state 1 is both recurrent and aperiodic and therefore i am able to use the theorem which states: if j is recurrent and aperiodic then lim Pij(n) tends to fij/mean as n tends to infinity.
While calculation of the mean is straightforward, calculation of fij is difficult. Having drawn the transition diagram i have trouble calculating a series which describes the possible ways of getting from state 4 to 1. This is because there are too many combinations for ways of getting from state 4 to 1 and therefore it is hard to see any pattern as you calculate the probability of getting from state 4 to 1 in an increasing number of transitions. Hopefully i can get some ideas as to how i can compute fij. Your help would be much appreciated.