Results 1 to 6 of 6

Math Help - Definition of Stationary markov chain

  1. #1
    Member mabruka's Avatar
    Joined
    Jan 2010
    From
    Mexico City
    Posts
    150

    Definition of Stationary markov chain

    hello

    im a little confused about this definition.


    Is a staionary markov chain a chain with a stationary distribution (not necessarily unique)?

    or the stationary distribution needs to be unique (so the chain is irreducible and positive recurrent) to be a stationary markov chain?

    or is something totally different?

    Ive read something about being P-invariant under "time translations" but i can not find this definition formally.


    any help would be great

    thank you!
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Member Focus's Avatar
    Joined
    Aug 2009
    Posts
    228
    Quote Originally Posted by mabruka View Post
    hello

    im a little confused about this definition.


    Is a staionary markov chain a chain with a stationary distribution (not necessarily unique)?

    or the stationary distribution needs to be unique (so the chain is irreducible and positive recurrent) to be a stationary markov chain?

    or is something totally different?

    Ive read something about being P-invariant under "time translations" but i can not find this definition formally.


    any help would be great

    thank you!
    From my knowledge, stationary refers to time homogeneity which is \mathbb{P}(X_n | X_{n-1}=x)=\mathbb{P}(X_{n-1}|X_{n-2}=x).

    A context may help in determining if this is what you are after.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Super Member Anonymous1's Avatar
    Joined
    Nov 2009
    From
    Big Red, NY
    Posts
    517
    Thanks
    1
    Quote Originally Posted by mabruka View Post
    hello

    im a little confused about this definition.


    Is a staionary markov chain a chain with a stationary distribution (not necessarily unique)?

    or the stationary distribution needs to be unique (so the chain is irreducible and positive recurrent) to be a stationary markov chain?

    or is something totally different?

    Ive read something about being P-invariant under "time translations" but i can not find this definition formally.


    any help would be great

    thank you!
    In general:
    Stationary Markov chain = Markov chain with transition probabilities that represent the "long run" fraction in state i.

    The stationary fraction in state i is given by the solution \pi (i) \Rightarrow uniqueness.

    For definiteness:
    Consider a Markov chain whose stationary distribution is given by \vec\pi = [1/4 \ \ 2/4 \ \ 1/4]<br />

    \Rightarrow Stationary Markov chain = \lim_{n\rightarrow \infty} p^n (i, j)= \left( \begin{array}{ccc} <br />
1/4 & 2/4 & 1/4 \\ 1/4 & 2/4 & 1/4 \\ 1/4 & 2/4 & 1/4 \end{array} \right)

    In plain english:
    If we have a Markov chain whose stationary distribution exists, then the stationary Markov chain has the solutions \pi (i) written in every entry of it's i^{th} column.

    Say in the above example we had sample space S=\{1 \ \ 2 \ \ 3\} where 1 = lower class, 2 = middle class 3 = upper class. Then the stationary Markov chain represents the fraction of people who will be in the lower, upper and middle classes as t\rightarrow \infty. e.g. in the long run 1/4 of the population will be in the lower class.

    Nahwatimsayin?
    Last edited by Anonymous1; April 9th 2010 at 03:59 PM.
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Member mabruka's Avatar
    Joined
    Jan 2010
    From
    Mexico City
    Posts
    150
    Thank you both for your answers.

    Actually what Focus said is what im interested in.

    Context:

    We have \{\xi_t\}_{t\in \mathbb R} an irreducible, recurrent and stationary Markov process with finite space state S and invariant distribution \Pi=\{\pi_i\}_{i\in S} on (\Omega,\mathbb F, \mathbb P). We consider \Omega to be the canonical space given by Kolmogorov's Extension theorem for \xi.

    My question about stationarity comes after introducing the concept of reversibility:

    \xi is reversible if  (\xi_{t_1},\ldots,\xi_{t_k}) has the same distribution as (\xi_{T-t_1},\ldots,\xi_{T-t_k}) for all k\geq1, t_1<\ldots,<t_k and T\in \mathbb R.


    A measurable transformation is introduced (time shift) :

    \theta_t:\Omega\longrightarrow \Omega

    \theta_t \omega(s) =\omega(s+t)

    Then it is stated:

    Since \xi is stationary, \mathbb P is \theta_t -invariant, i.e. \theta_t \mathbb P =\mathbb P.


    My question: What role does stationarity play so it makes our \mathbb P    \theta_t - invariant and how does this follows from the definition of stationarity?? That is why i first asked about the formal definition of stationarity.
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Super Member Anonymous1's Avatar
    Joined
    Nov 2009
    From
    Big Red, NY
    Posts
    517
    Thanks
    1
    Quote Originally Posted by mabruka View Post
    Thank you both for your answers.

    Actually what Focus said is what im interested in.
    Okay...

    I assume you realize that the definition of stationary Markov chain that I gave directly implies time homogeneity, or \mathbb{P}(X_n | X_{n-1}=x)=\mathbb{P}(X_{n-1}|X_{n-2}=x).

    I was simply trying to paint a vivid picture...
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Member mabruka's Avatar
    Joined
    Jan 2010
    From
    Mexico City
    Posts
    150
    So stationarity and time homogeneiety are two different definitions, but


    stationarity implies time homogeneiety right?
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Markov Chain of random variables from a primitive markov chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: October 19th 2011, 09:12 AM
  2. chain rule and stationary points question
    Posted in the Calculus Forum
    Replies: 1
    Last Post: October 9th 2011, 01:34 PM
  3. Stationary distribution - Continuous Markov Chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: December 31st 2010, 06:28 AM
  4. Markov chain
    Posted in the Advanced Statistics Forum
    Replies: 8
    Last Post: April 14th 2010, 07:19 AM
  5. Replies: 2
    Last Post: October 28th 2008, 07:32 PM

Search Tags


/mathhelpforum @mathhelpforum