Results 1 to 4 of 4

Thread: Discrete Time Markov Chain

  1. #1
    Newbie
    Joined
    Mar 2008
    From
    America
    Posts
    8

    Discrete Time Markov Chain

    Hey Guys,

    A quick question I was wondering if you could help me or help me try and understand the way about approaching this question...

    Let {Xt}t≥0 be a two-state Markov chain with state space S = {0, 1}, transition matrix:

    $\displaystyle P = \left(\begin{array}{cc}1-p&p\\q&1-q\end{array}\right)$

    and initial distribution $\displaystyle \pi_0 = ({\pi_0(0),\pi_0(1)})$

    Define the New Stochastic Processes {Yt}t≥1 and {Zt}t≥1 as:

    $\displaystyle Y_t = X_t + X_{t-1}$ and,

    $\displaystyle Z_t = 10X_t + X_{t-1}$

    a) What are the State Spaces for these New Stochastic Processes?

    I have tried using the transition matrix and inputting the values of S = {0,1} into {Yt} and {Zt} but I'm not sure if I am on the right path.. Any Suggestions??

    Thanks
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Newbie
    Joined
    Mar 2008
    From
    America
    Posts
    8
    Please somebody help!!!

    So I don't feel so alone with Markov Chains?!
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Member
    Joined
    Mar 2008
    Posts
    148
    The state space is the set of possible values which the random variable can take on. In this example if $\displaystyle p,q$ are both non zero then there is some chance of going from any state to any other state in {$\displaystyle 0,1$}.

    Therefore:
    $\displaystyle
    Y_t = X_t + X_{t-1}
    $

    can be $\displaystyle 0$ when $\displaystyle X_t = X_{t-1} = 0$
    or $\displaystyle 1$ when $\displaystyle X_t = 0 \text{ and } X_{t-1} = 1 \text{ or } X_t = 1 \text{ and } X_{t-1} = 0 $
    or $\displaystyle 2$ when $\displaystyle X_t = 1 \text{ and } X_{t-1} = 1 $

    Use the same reasoning to determine what states $\displaystyle Z_t$ can be in. Can it achieve the value 11?
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Newbie
    Joined
    Mar 2008
    From
    America
    Posts
    8
    Thanks for helping to clear up my interpretation of the State Space!

    I'm assuming that with $\displaystyle Z_t = 10X_t + X_{t-1} $

    $\displaystyle 0 $ when $\displaystyle 10X_t = X_{t-1} = 0 $
    $\displaystyle 1 $ when $\displaystyle 10X_t = 0 $ and $\displaystyle X_{t-1} = 1 $
    $\displaystyle 10 $ when $\displaystyle 10X_t = 1 $ and $\displaystyle X_{t-1} = 0 $
    $\displaystyle 11 $ when $\displaystyle 10X_t = 1 $ and $\displaystyle X_{t-1} = 1 $

    So would I be right in saying that $\displaystyle S_z = \{0,1,10,11\} $
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Discrete Time Markov Chain
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: Mar 22nd 2011, 07:56 AM
  2. Markov Chain mixing time
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: Apr 18th 2010, 09:32 AM
  3. Discrete Time Markov Chain
    Posted in the Advanced Statistics Forum
    Replies: 3
    Last Post: Apr 2nd 2010, 03:21 PM
  4. Discrete Time Markov Chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: Sep 29th 2009, 01:44 PM
  5. Replies: 0
    Last Post: Mar 10th 2009, 11:29 AM

Search Tags


/mathhelpforum @mathhelpforum