Thread: help in markov chain

Hi i have an exercise and have a trouble with it if anyone can help me i would appreciate it plz help if you can.

This is the exercise:

Suppose the weather in a city in the (n) day is described from a Markov chain and it depend on the (n-1) day as follows:

1. if in (n-1) day rains then in (n) day rains with probability 60% or will be sunshine with probability 40%

2. if in (n-1) day has sunshine then in (n) day will have sunshine with probability 60% or will have clouds with probability 40%.

3. if in (n-1) day have clouds then in (n) day will rain with probability 20% , will have sunshine with probability 20% , will have clouds with probability 55% or will snow with probability 5%.

4. if in (n-1) day snows then in (n) day will have clouds with probability 90% or will snow with probability 10%


The exercise asks for the following:

1. Markov chain transition matrix
2. Solve the balanced equations and calculate the steady-state probabilities .

Thanks in advance

the transition matrix can be written down, without calculation, from the information in the question. exactly where are you stuck?

in the Markov chain transition matrix can you help i cant make it .

suppose the states are (in order): rain, sun, cloud, snow.

The first row of the transition matrix gives the probability of transitioning from RAIN to each of the other states, which is given in the question as (0.6,0.4,0,0)
The first row of the transition matrix gives the probability of transitioning from SUN to each of the other states, which is given in the question as (0,0.6,0.4,0)

can you finish?

i see yeah i can finish it now one more thing it says also in the exercise that we have to make the situation diagram it's a diagram that takes us from one situation to another . can u help with that?

And something else :


We have to simulate the Markov chain that we created i will explain:

You can start the chain from whichever situation you want.
1. In every step of the simulation we must create a random number .
2. So when we finish we use the transition matrix to go to the next situation
and so on
The simulation must have time equal to 1080 steps of the previous (1. and 2.)