Thread: Prove Bernoulli Process is Markov Chain

1. Prove Bernoulli Process is Markov Chain

Let $\displaystyle Z_1, Z_2, ...$ be a Bernoulli Process with success parameter p. Prove if

$\displaystyle Y_0, Y_1,...$, where $\displaystyle Y_{k+1} = max(0, Y_k + 2Z_{k+1} - 1)$ for $\displaystyle k = 0,1,...$ and $\displaystyle Y_0 = 0$

and

$\displaystyle X_0, X_1,..$ where $\displaystyle X_{k+1} = Z_{k+1} + Z_k$ for $\displaystyle k = 0,1,...,$ and $\displaystyle X_0 = 0$

are Markov chains.

2. Does anyone have any idea how to go about these?

3. $\displaystyle X_2 = Z_2 + Z_1$

Let us suppose $\displaystyle X_2 = 1$ then, either $\displaystyle Z_2=1 \mbox{ and } Z_1=0 \mbox{or} Z_1=0 \mbox{ and } Z_2=1$ with 0.5 probability of each case.

Given this, and knowing that $\displaystyle X_3 = Z_3 + Z_2$

$\displaystyle P(X_3 = 1|X_2 = 1) = p*0.5 + (1-p)*0.5 = 0.5$

However, if we know that $\displaystyle X_1 = Z_1 + Z_0 = 2$ we know that $\displaystyle Z_1 = 1$

Taking this into account, we get that,$\displaystyle X_1 = 2$ and $\displaystyle X_2 = 1$, imply $\displaystyle Z_2 = 0$.

Therefore,

$\displaystyle P(X_3 = 1 | X_2 = 1, X_1 = 2) = p$

It doesn't seem to be a Markov Chain