Originally Posted by

**akbar** Took me a little while to get used to your (unorthodox?) notation of conditional probabilities. But it kind of makes sense now.

My understanding is that $\displaystyle P_\pi\equiv\mathbb{P}$ and

$\displaystyle P_\pi(\cdot)=\sum_x \pi(x)P_x(\cdot)$ should be understood as $\displaystyle \mathbb{P}(.)=\sum_x \pi(x)\mathbb{P}(\cdot|X_0=x)$,

$\displaystyle P_1(X_{n}=x)$ as $\displaystyle \mathbb{P}(X_{n}=x|X_{n-1}=1)$. Please let me know if I didn't decipher it properly.

Sorry I didn't define my notation; but I really don't think it's unorthodox since I've been using it over and over and I've seen it used in many places. By the way it is extremely convenient and I couldn't even think of studying Markov chains without it ...

First of all you can consider the X as a typo in place of $\displaystyle \omega$. (Markov chains are "always" called X...)

And the index of the probability in my notation is always the *initial* state or distribution (well, it is always the initial distribution but we write $\displaystyle P_x:=P_{\delta_x}$ for short when $\displaystyle x$ is a state). In the present situation you can think of it as a conditioned distribution indeed: $\displaystyle P_x(\cdot)=\mathbb{P}(\cdot|\omega_0=x)$ (makes sense since $\displaystyle \pi(x)>0$). For an intensive use one usually defines $\displaystyle P_x$ to be the distribution of a Markov chain starting at $\displaystyle x$ (with the currently considered transition probabilities), and $\displaystyle (X_n)_{n\geq 0}$ to be the "identity process" on $\displaystyle E^{\mathbb{N}}$ (state space $\displaystyle E$): $\displaystyle (X_n)_{n\geq 0}(\omega):=(\omega_n)_{n\geq 0}$ (the idea is to have one canonical process and several probability measures). It makes it easier to state and use Markov property.

Thus the convolution formula is, with your notation (i.e. without further notation):

$\displaystyle \mathbb{P}(\omega_n=1)=\sum_{k=1}^n \mathbb{P}(\tau=k)\mathbb{P}(\omega_{n-k}=1|\omega_0=1)$.

(note that I would have been very embarassed to write this equation if you had $\displaystyle \omega_0=2$ a.s., for instance ; while the meaning is preserved and the version with the identity process and $\displaystyle P_x$ notation would still make sense)

If my understanding of equation $\displaystyle (1)$ is correct, I understand now how my interpretation of the convolution relation was wrong, but makes me still wonder how to understand the approach of Feller's book, which is supposed to be applicable to the case of Markov chains: in short, what is the equivalent of $\displaystyle u_p$ and $\displaystyle f_p$ in this problem?

I've just had a look at Feller. If you're considering equation (3.1) and Theorem XIII.3.1 thereafter, the problem you're experiencing is typical with Feller... Even though Feller is a great reference (I own both volumes myself), it is not really easy to use, partly because it dates back to the sixties and introduces lots of notation no one else uses (any more?), partly because you constantly have to look several pages back in order to find the currently active hypotheses. In this case, remember that the "attribute" $\displaystyle \mathcal{E}$ is what he calls a "recurrent event", and if you look up the definition, you can see that "$\displaystyle \omega_n=1$" is not a recurrent event because condition (b) fails (I should say: you can see that the attribute $\displaystyle \mathcal{E}$ such that the finite sequence of states $\displaystyle (E_{j_1},\ldots,E_{j_n})$ has the attibute $\displaystyle \mathcal{E}$ iff $\displaystyle E_{j_n}=1$, is not a recurrent event...). It is a recurrent event only under the probability $\displaystyle P_1$, not under $\displaystyle P_\pi$.

I bet you'll never ever find this definition of "recurrent event" or "attribute" in any other place...

Have a nice new year's eve; it's time for me to go cooking now!