Thanks for your answer. Given a stochastic matrix

, the original problem deals proving the existence of a vector

with non-negative elements such that

.

My initial answer was to start as you did, e.g. to restrict to a subset of (persistent) states

such that the matrix

would be irreductible. For this, you use the decomposition theorem of Markov chains (see for example Grimmett & Stirzaker p.224) and the existence of at least one persistent state, immediate in the case of a finite statespace.

Once in this subset, you end up in the case of an ergodic matrix for which you can apply the ergodicity theorem, e.g. the existence of

with positive elements such that

.

Complete the vector with zeros to obtain a solution to the problem.