Results 1 to 2 of 2

Math Help - Markov Chains

  1. #1
    Junior Member
    Joined
    Nov 2009
    Posts
    34

    Markov Chains

    Hello, I have two problems I'm stuck on. Any hints would be appreciated.

    \{X_n : n\geq0\} \mbox{ is a Markov Chain and suppose } P_{ii} > 0

    \eta_i \mbox{ is the exit time from state i}

    \eta_i = min\{n \geq 1 : X_n \neq i\}

    How would I derive the distribution of \eta_i?


    Secondly, suppose that whether not a team wins the Super Bowl depends on the number of Super Bowl wins for that team in the last three seasons. How would I transform this into a Markov Chain? I'm having trouble identifying the state space. Thank you.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Member Focus's Avatar
    Joined
    Aug 2009
    Posts
    228
    Quote Originally Posted by BERRY View Post
    Hello, I have two problems I'm stuck on. Any hints would be appreciated.

    \{X_n : n\geq0\} \mbox{ is a Markov Chain and suppose } P_{ii} > 0

    \eta_i \mbox{ is the exit time from state i}

    \eta_i = min\{n \geq 1 : X_n \neq i\}

    How would I derive the distribution of \eta_i?
    First, \mathbb{P}(\eta_i =1)=1-P_{X_0,i}.

    Now for k greater than 1, \{\eta_i=k\} tells you that you have stayed at i for k-2 times and then moved off to an other state. So first you have to move into state i, with probability P_{X_0,i}, now you have to stay there until k-1, and at k move along.

    If that is not clear, do let me know.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Markov Chains
    Posted in the Advanced Statistics Forum
    Replies: 5
    Last Post: August 26th 2009, 08:22 AM
  2. Markov Chains
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: April 26th 2009, 08:17 PM
  3. Markov Chains
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: March 18th 2009, 10:53 AM
  4. Markov Chains
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: March 6th 2009, 09:34 AM
  5. Markov Chains
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: November 17th 2008, 09:33 AM

Search Tags


/mathhelpforum @mathhelpforum