Show that the following Markov chain is irreducible:

$\displaystyle S=\{0, 1, 2, ..., N\} $

$\displaystyle p(X_{0}=i)=\frac{1}{(N+1)}$ for i=0,...,N

for all state (except 0 and N)

$\displaystyle P_{ij}= \frac{1}{3}$, $\displaystyle j=i$

$\displaystyle P_{ij}= \frac{1}{3}$, $\displaystyle j=i+1$

$\displaystyle P_{ij}= \frac{1}{3}$, $\displaystyle j=i-1$

for state 0

$\displaystyle P_{0j}= \frac{2}{3}$, $\displaystyle j=i=0$

$\displaystyle P_{0j}= \frac{1}{3}$, $\displaystyle j=i+1=1$

for state N

$\displaystyle P_{Nj}= \frac{2}{3}$, $\displaystyle j=i=N$

$\displaystyle P_{Nj}= \frac{1}{3}$, $\displaystyle j=i-1=N-1$