# Markov Chain Steady State Probablity

• Jan 20th 2010, 08:39 AM
wxyj
Hi, all

Just a quick question about the Markov Chain, if we have (x,y,z)*A=(x,y,z), where A is the transmission matrix, and find a solution to (x,y,z), which is the probablities for the steady state. Does it mean...if n appoaches to infinity.the steady state probability is the final equilibrium? and not some factor or fraction of the steady state probability???

Thanks
• Jan 20th 2010, 08:58 AM
Moo
Hello,
Quote:

Originally Posted by wxyj
Hi, all

Just a quick question about the Markov Chain, if we have (x,y,z)*A=(x,y,z), where A is the transmission matrix, and find a solution to (x,y,z), which is the probablities for the steady state. Does it mean...if n appoaches to infinity.the steady state probability is the final equilibrium?

For the steady state probability to be the final equilibrium (as you call it), you need the chain to be irreducible, aperiodic and recurrent positive.

It's written here : Markov chain - Wikipedia, the free encyclopedia (the 2nd & 3rd formulas)

Quote:

and not some factor or fraction of the steady state probability???
You're working with stationary probabilities. So the sum of the components has to be 1. If you were working with stationary measures, then there is a unique solution, up to a factor.

NB : for some vocabulary problems, stationary = steady
• Jan 20th 2010, 09:08 AM
Drexel28
Quote:

Originally Posted by Moo
Hello,

For the steady state probability to be the final equilibrium (as you call it), you need the chain to be irreducible, aperiodic and recurrent positive.

It's written here : Markov chain - Wikipedia, the free encyclopedia (the 2nd & 3rd formulas)

Your're working with stationary probabilities. So the sum of the components has to be 1. If you were working with stationary measures, then there is a unique solution, up to a factor.

NB : for some vocabulary problems, stationary = steady

I'm sorry. What does "NB" mean? I see it everywhere!
• Jan 20th 2010, 09:17 AM
Moo
Quote:

Originally Posted by Drexel28
I'm sorry. What does "NB" mean? I see it everywhere!

Nota bene. No wonder you guys don't know it, because you usually don't have the Latin touch ! :D

Nota bene - Wikipedia, the free encyclopedia

PS (post scriptum) : when I saw you replied this thread, I was surprised lol! Thought you had done these probabilities you like so much (Rofl)
• Jan 20th 2010, 09:38 AM
Drexel28
Quote:

Originally Posted by Moo
Nota bene. No wonder you guys don't know it, because you usually don't have the Latin touch ! :D

Nota bene - Wikipedia, the free encyclopedia

PS (post scriptum) : when I saw you replied this thread, I was surprised lol! Thought you had done these probabilities you like so much (Rofl)

Haha! Thanks. Actually, my ignorant unfounded dislike of probability was dispelled by you and Laurent with your measure-theoretic probability theory! So, while I still am as ignorant as ever about the subject I really appreciate it now!
• Jan 20th 2010, 09:49 AM
Moo
Quote:

Originally Posted by Drexel28
Haha! Thanks. Actually, my ignorant unfounded dislike of probability was dispelled by you and Laurent with your measure-theoretic probability theory! So, while I still am as ignorant as ever about the subject I really appreciate it now!

'Serious' probabilities use & overuse measure theory :D
Probability is not reduced to combinatorics (very fortunately !)

Glad to see you say such a thing :p