1. ## Markov Chain

Hi, I am having trouble with this problem:

There are two transmission lines from a generating station to a nearby city. Normally,
both are operating (“up”). On any day on which line A is operating, there
is probability
p that it will go down at the end of the day. On any day on which
line B is operating, there is probability
q that it will go down at the end of the day.
It takes the repair crew a day to repair a broken line. Only one line can be repaired
at a time; if both are down, they repair line A first.

Model this situation as a Markov chain. (Hint: there are four states.)

I have the answer but I can't figure it out past probabilities p, q, & 1.

The answer is according to the transition diagram:

P= 1 p(1-q) q(1-p) pq
1-q 0 q 0
1-p p 0 0
0 0 1 0

I have drawn the state transition diagram, as well. I can easily see: p (prob line A will go down), q (prob line B will go down), and 1 (prob A repair) but I am lost after that. Do I have the states mixed up?

If p is the prob A will go down (state 3 to 2), I thought 1-p would be the prob A will go up (which would be transiton from state 2 to 1) but according to the solution the transition from 2 to 1 is 1-q.....

I'm soooo lost - can anyone put this in plain english for me.

The prof also said if we were having difficulty to set p=.01 and q =.02 and solve numerically but I have no idea how to do that.

Thanks very much

2. Originally Posted by rbursell
Hi, I am having trouble with this problem:

There are two transmission lines from a generating station to a nearby city. Normally,
both are operating (“up”). On any day on which line A is operating, there
is probability
p that it will go down at the end of the day. On any day on which
line B is operating, there is probability
q that it will go down at the end of the day.
It takes the repair crew a day to repair a broken line. Only one line can be repaired
at a time; if both are down, they repair line A first.

Model this situation as a Markov chain. (Hint: there are four states.)

I have the answer but I can't figure it out past probabilities p, q, & 1.

The answer is according to the transition diagram:

P= 1 p(1-q) q(1-p) pq
1-q 0 q 0
1-p p 0 0
0 0 1 0

I have drawn the state transition diagram, as well. I can easily see: p (prob line A will go down), q (prob line B will go down), and 1 (prob A repair) but I am lost after that. Do I have the states mixed up?

If p is the prob A will go down (state 3 to 2), I thought 1-p would be the prob A will go up (which would be transiton from state 2 to 1) but according to the solution the transition from 2 to 1 is 1-q.....

I'm soooo lost - can anyone put this in plain english for me.

The prof also said if we were having difficulty to set p=.01 and q =.02 and solve numerically but I have no idea how to do that.

Thanks very much

This can't be quite right. The probabilities along a row should sum to 1. First we have to understand the states.
State 1: both working
State 2: B is working, but not A
State 3: A is working, but not B
State 4: Neither working

First row. This is the row of probabilities of going to state 1, 2, 3, 4 given that you are starting in state 1 (both working). Probability of staying in state 1? (1-p)(1-q) - this is the probability that neither A nor B go down. Probability of going to state 2: p(1-q) - A goes down but B stays up. 1->3: q(1-p) - B goes down A stays up. 1->4: pq (Both go down)

If p is the prob A will go down (state 3 to 2), I thought 1-p would be the prob A will go up (which would be transiton from state 2 to 1) but according to the solution the transition from 2 to 1 is 1-q.....

Don't think of p as the probability of going from 3 to 2 (even though it happens to be true). You are told that p is the probability that A goes down. If you are in State 3, then you start the day with B down and A up. So at the end of the next day, B will be up (it was fixed), and A may or may not be down. So state 3 can go to either 1 or 2. What is the probability that it goes to 1? well (1-p) which is the probability that A stays up. What is the probability that it goes to 2? Well that is p - the probability that A goes down.

It is important to remember what p and q are. They represent the probabilities of A and B (respectively) going down. You use these to build up the probabilities of going from state to state. The probability of going from 2 to 1? State 2: B is up, A is down. At the end of the next day, A will be up (it was fixed), but B may or may not be up (States 1 or 3 respectively). What is the probability that B is still up? 1-q! What is the probability that B goes down? q!

Do you see?

So here is the final matrix:

P= (1-p)(1-q) p(1-q) q(1-p) pq
1-q 0 q 0
1-p p 0 0
0 0 1 0