# Thread: Discrete Time Markov Chain

1. ## Discrete Time Markov Chain

Hey Guys,

A quick question I was wondering if you could help me or help me try and understand the way about approaching this question...

Let {Xt}t≥0 be a two-state Markov chain with state space S = {0, 1}, transition matrix:

$\displaystyle P = \left(\begin{array}{cc}1-p&p\\q&1-q\end{array}\right)$

and initial distribution $\displaystyle \pi_0 = ({\pi_0(0),\pi_0(1)})$

Define the New Stochastic Processes {Yt}t≥1 and {Zt}t≥1 as:

$\displaystyle Y_t = X_t + X_{t-1}$ and,

$\displaystyle Z_t = 10X_t + X_{t-1}$

a) What are the State Spaces for these New Stochastic Processes?

I have tried using the transition matrix and inputting the values of S = {0,1} into {Yt} and {Zt} but I'm not sure if I am on the right path.. Any Suggestions??

Thanks

2. Please somebody help!!!

So I don't feel so alone with Markov Chains?!

3. The state space is the set of possible values which the random variable can take on. In this example if $\displaystyle p,q$ are both non zero then there is some chance of going from any state to any other state in {$\displaystyle 0,1$}.

Therefore:
$\displaystyle Y_t = X_t + X_{t-1}$

can be $\displaystyle 0$ when $\displaystyle X_t = X_{t-1} = 0$
or $\displaystyle 1$ when $\displaystyle X_t = 0 \text{ and } X_{t-1} = 1 \text{ or } X_t = 1 \text{ and } X_{t-1} = 0$
or $\displaystyle 2$ when $\displaystyle X_t = 1 \text{ and } X_{t-1} = 1$

Use the same reasoning to determine what states $\displaystyle Z_t$ can be in. Can it achieve the value 11?

4. Thanks for helping to clear up my interpretation of the State Space!

I'm assuming that with $\displaystyle Z_t = 10X_t + X_{t-1}$

$\displaystyle 0$ when $\displaystyle 10X_t = X_{t-1} = 0$
$\displaystyle 1$ when $\displaystyle 10X_t = 0$ and $\displaystyle X_{t-1} = 1$
$\displaystyle 10$ when $\displaystyle 10X_t = 1$ and $\displaystyle X_{t-1} = 0$
$\displaystyle 11$ when $\displaystyle 10X_t = 1$ and $\displaystyle X_{t-1} = 1$

So would I be right in saying that $\displaystyle S_z = \{0,1,10,11\}$