Let X be a Markov chain with state space (a,b,c,d) and transition probabilities given by

P= [0.3 0.1 0.0 0.6

0.0 0.3 0.5 0.2

0.0 0.5 0.5 0.0

0.0 0.0 1.0 0.0]

each time the chain is in state a, a profit of 10 is made; each visit to state b yields a 30 profit;each visit to state c yields a 5 profit; and each visit to state d cost 10.

Find the following:

a) E[f(X1)|X0=b]

b) E[f(X2)|X0=c]

c) let the initial probabilities be given by the vector mu=(0.4, 0.2, 0.3 ,0.1).

What is the expected profit at the first step, i.e E of mu[f(X1)]?

can anyone help with ne of the above questions please. im really stuck. thank you.