Discrete Time Markov Chain

**ninmaster** · March 9th 2008, 05:25 PM

Hey Guys,

A quick question I was wondering if you could help me or help me try and understand the way about approaching this question...

Let {Xt}t≥0 be a two-state Markov chain with state space S = {0, 1}, transition matrix:

$P = \left(\begin{array}{cc}1-p&p\\q&1-q\end{array}\right)$

and initial distribution $\pi_0 = ({\pi_0(0),\pi_0(1)})$

Define the New Stochastic Processes {Yt}t≥1 and {Zt}t≥1 as:

$Y_t = X_t + X_{t-1}$ and,

$Z_t = 10X_t + X_{t-1}$

a) What are the State Spaces for these New Stochastic Processes?

I have tried using the transition matrix and inputting the values of S = {0,1} into {Yt} and {Zt} but I'm not sure if I am on the right path.. Any Suggestions??

Thanks

**ninmaster** · March 10th 2008, 01:52 PM

Please somebody help!!!

So I don't feel so alone with Markov Chains?!

**iknowone** · March 10th 2008, 03:11 PM

The state space is the set of possible values which the random variable can take on. In this example if $p,q$ are both non zero then there is some chance of going from any state to any other state in { $0,1$ }.

Therefore:
$<br /> Y_t = X_t + X_{t-1}<br />$

can be $0$ when $X_t = X_{t-1} = 0$
or $1$ when $X_t = 0 \text{ and } X_{t-1} = 1 \text{ or } X_t = 1 \text{ and } X_{t-1} = 0$
or $2$ when $X_t = 1 \text{ and } X_{t-1} = 1$

Use the same reasoning to determine what states $Z_t$ can be in. Can it achieve the value 11?

**ninmaster** · March 10th 2008, 10:54 PM

Thanks for helping to clear up my interpretation of the State Space!

I'm assuming that with $Z_t = 10X_t + X_{t-1}$

$0$ when $10X_t = X_{t-1} = 0$
$1$ when $10X_t = 0$ and $X_{t-1} = 1$
$10$ when $10X_t = 1$ and $X_{t-1} = 0$
$11$ when $10X_t = 1$ and $X_{t-1} = 1$

So would I be right in saying that $S_z = \{0,1,10,11\}$

Thread: Discrete Time Markov Chain

LinkBack

Thread Tools

Display