Results 1 to 10 of 10

Math Help - Markov Chain Question

  1. #1
    Junior Member
    Joined
    Nov 2007
    Posts
    68

    Markov Chain Question

    Am i right in thinking that it is impossible to have a Markov chain with an infinite number of transient states and an infinite number of positive recurent states??????

    where a positive-recurrent state, is a recurrent state where the expected no. of transition to return is finite....

    If anyone could let me know whether I am correct and this is impossible I would be very grateful...
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Oli
    Oli is offline
    Member
    Joined
    Apr 2008
    Posts
    82
    False:

    Start at n=0.

    If n even let P(n->(n+1))=0.5 P(n->(n+2))=0.5

    In n odd let P(n->n)=1

    Then odd numbers are trivially recurrant, even numbers trivially transient.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Junior Member
    Joined
    Nov 2007
    Posts
    68
    thanks for that, what is the notation ->

    and are the recurrent states here positive-recurrent??


    Thanks...
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Oli
    Oli is offline
    Member
    Joined
    Apr 2008
    Posts
    82
    P(a->b) = probability that Markov chain goes from state a to state b.

    The expected number of steps before an odd number returns to itself is one, which is less than infinity, and so the odd numbers are positive recurrent.

    According to my understanding of the definitions, this is a counter example.
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Junior Member
    Joined
    Nov 2007
    Posts
    68
    thanks alot for that, really appriciate it..
    Im just stuck on one last small part of one question asking for an example of a markov chain with 5 states and more than 1 stationary distribution... any ideas..
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Oli
    Oli is offline
    Member
    Joined
    Apr 2008
    Posts
    82
    There are more trivial examples, but here is one from a question paper I did:

    Continuous time Markov Chain with Q matrix:

    -3, 2, 0, 0, 1
    0, -3, 3, 0, 0
    0, 5, -5, 0, 0
    0, 0, 0, -2, 2
    0, 0, 0, 1, -1
    Follow Math Help Forum on Facebook and Google+

  7. #7
    Junior Member
    Joined
    Nov 2007
    Posts
    68
    thanks, the work which im doing is all about disrete time markov chains, and i have to represent the chain wither with the transition matrix or diagram.. any thoughts???
    Follow Math Help Forum on Facebook and Google+

  8. #8
    Oli
    Oli is offline
    Member
    Joined
    Apr 2008
    Posts
    82
    Matrix:
    0 0.5 0 0.5 0
    0 0.5 0.5 0 0
    0 0.5 0.5 0 0
    0 0 0 0.5 0.5
    0 0 0 0.5 0.5

    should do it... its basically a similar set up to the one I posted earlier.
    Hope you can read that.

    Not quite sure why you need 5 states though.
    Is the 2x2 identity matrix a markov chain?
    If it is, does that not have 2 stationary distributions?
    Follow Math Help Forum on Facebook and Google+

  9. #9
    Junior Member
    Joined
    Nov 2007
    Posts
    68
    Thanks alot Oli, you obviously know your markov chains,, are you a lecturer or something??
    Follow Math Help Forum on Facebook and Google+

  10. #10
    Oli
    Oli is offline
    Member
    Joined
    Apr 2008
    Posts
    82
    Nah, am a bored finalist trying to do revision. Came here to get a question answered for me, and kind of got addicted to doing other peoples problems.

    Oops! Should really get on with some of my own work! Its too hard though.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Markov Chain of random variables from a primitive markov chain
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: October 19th 2011, 08:12 AM
  2. Markov chain question
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: March 4th 2011, 10:52 AM
  3. Markov Chain Question
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: February 15th 2011, 04:42 PM
  4. Replies: 2
    Last Post: October 28th 2008, 06:32 PM
  5. Need help on markov chain question ?
    Posted in the Advanced Statistics Forum
    Replies: 3
    Last Post: April 27th 2008, 08:42 AM

Search Tags


/mathhelpforum @mathhelpforum