Hi, Moo,

You can indeed define conditional expectation of positive random variables even if they're not integrable: have a look p.147 of this wonderful reference

In your case, the reason why $\displaystyle E[f(X_n)]$ is finite is obtained by induction: $\displaystyle E[f(X_0)]=f(x)<\infty$ and

$\displaystyle E[f(X_{n+1})]=\sum_y f(y)P(X_{n+1}=y)=\sum_y \sum_z f(y)P(X_n=z)Q(z,y) $ $\displaystyle = \sum_z P(X_n=z)\sum_y f(y) Q(z,y)$ $\displaystyle =\sum_z P(X_n=z) f(z)=E[f(X_n)]$.

But because of the first remark, you don't really *have *to do this beforehand, it may come as a consequence of the martingale property; it may depend on what you're supposed to know...