Consider the markov chain with the transition matrix:
[1/2 1/3 1/6]
[3/4 0 1/4]
[ 0 1 0 ]
the process is started in state 1; find the probability that it is in state 3 after two steps
Hello, morganfor!
We want $\displaystyle A^2.$Consider the transition matrix: .$\displaystyle A \;=\; \begin{pmatrix}\frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ \\[-4mm]
\frac{3}{4} & 0 & \frac{1}{4} \\ \\[-4mm]
0 & 1 & 0 \end{pmatrix}$
The process is started in state 1.
Find the probability that it is in state 3 after two steps.
. . $\displaystyle A^2 \;=\;\begin{pmatrix}\frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ \\[-4mm] \frac{3}{4} & 0 & \frac{1}{4} \\ \\[-4mm] 0 & 1 & 0 \end{pmatrix} \begin{pmatrix}\frac{1}{2} & \frac{1}{3} & \frac{1}{6} \\ \\[-4mm] \frac{3}{4} & 0 & \frac{1}{4} \\ \\[-4mm] 0 & 1 & 0 \end{pmatrix}$ .$\displaystyle =\; \begin{pmatrix}\frac{1}{2} & \frac{1}{3} & {\color{red}\frac{1}{6}} \\ \\[-4mm]
\frac{3}{8} & \frac{1}{2} & \frac{1}{8} \\ \\[-4mm]
\frac{3}{4} & 1 & \frac{1}{4} \end{pmatrix}$
Therefore: .$\displaystyle P(a_1\to a_3\text{, 2 steps}) \:=\:\frac{1}{6} $
No need to calculate A^2, if you want to calculate $\displaystyle f^{2}_{1,3}$ which is the probability to get to state 3 first time after two steps from state 1.
This probability is just: P1,1*P1,3+P1,2*P2,3, no need to calculate the whole matrix.
Got an exam tomorrow and just going over past papers and stuck on these questions would be greatful for any help
Question 1.
A random walk starts at k, where k is a positive integer, and takes place on the integers. It is assumed that Xn+1, the location of the walk after (n+ 1) steps,
satisfies
Xn+1 =
Xn + 1 with probability p
Xn - 1 with probability 1 - p
where p + (1 - p) = 1.
(a) Find the mean and variance of Xn, and comment on your result when p = 1
2 .
(b) Determine the value of p which maximizes the expression for the variance obtained in (a) above and comment.
(c) Derive the probability distribution of Xn when k = 0.
Question 2.
(b) A Markov chain has the following transition matrix P, where:
P =
0 0 0 0 1
1/8 1/2 3/8 0 0
0 0 1/4 3/4 0
0 0 1/3 0 2/3
1 0 0 0 0
(i) Draw the Chain Diagram showing the transitions between the states.
(ii) Identify communicating classes and indicate whether they are closed or not.
(iii) State which of the properties in (a) each state possesses.
(iv) Determine whether this Markov chain has a unique stationary distribution and,
if so,find it.