# Thread: Markov Chain coin toss

1. ## Markov Chain coin toss

Coin 1 has probability of coming up heads with probability 0.7.
Coin 2 has probability of coming up heads with probability 0.6.

If we flip a coin today and it comes up heads, we will be flipping coin 1 tomorrow.

If we flip a coin today and it comes up tails ,we will be flipping coin 2 tomorrow.

if the coin initially flipped is equally likely to be coin 1 or coin 2, what is probability that the coin flipped on the third day after the initial flip is coin 1?

I'm having trouble setting my matrix, I was pretty sure that my matrix would look like:

$\displaystyle \begin{bmatrix}0.7 & 0.3\\0.6& 0.4\end{bmatrix}$

but apparently this is wrong, since I have to consider the fact that both coins are equally likely to be flipped originally. Therefore you would have:

$\displaystyle 0.5 \times 0.7 =0.35$ of obtaining a head using coin one initially.
$\displaystyle 0.5 \times 0.3 =0.15$ of obtaining tails using coin one initially.
$\displaystyle 0.5 \times 0.6=0.3$ of obtaining a head using coin two initially.
$\displaystyle 0.5 \times 0.4=0.2$ of obtaining tails using coin two initially.

The problem is that if I make a matrix using these values, they won't sum up to one; so I'm a little lost on what to do.

2. Originally Posted by lllll
Coin 1 has probability of coming up heads with probability 0.7.
Coin 2 has probability of coming up heads with probability 0.6.

If we flip a coin today and it comes up heads, we will be flipping coin 1 tomorrow.

If we flip a coin today and it comes up tails ,we will be flipping coin 2 tomorrow.

if the coin initially flipped is equally likely to be coin 1 or coin 2, what is probability that the coin flipped on the third day after the initial flip is coin 1?

I'm having trouble setting my matrix, I was pretty sure that my matrix would look like:

$\displaystyle \begin{bmatrix}0.7 & 0.3\\0.6& 0.4\end{bmatrix}$

but apparently this is wrong, since I have to consider the fact that both coins are equally likely to be flipped originally. Mr F says: The transition matrix is $\displaystyle {\color{red}\begin{bmatrix}0.6 & 0.7\\0.4& 0.3\end{bmatrix}}$. The initial state matrix is $\displaystyle {\color{red}\begin{bmatrix}0.5 \\ 0.5 \end{bmatrix}}$. The final state matrix is $\displaystyle {\color{red}\begin{bmatrix}Pr(C1) \\ Pr(C1') \end{bmatrix}}$.

[snip]
..

3. Originally Posted by lllll
I'm having trouble setting my matrix, I was pretty sure that my matrix would look like:

$\displaystyle \begin{bmatrix}0.7 & 0.3\\0.6& 0.4\end{bmatrix}$
It seems there are different conventions to define transition matrices. I am used to defining the transition matrix as $\displaystyle P=(p(i,j))_{i,j}$ (the probability $\displaystyle p(i,j)$ to go from state $\displaystyle i$ to state $\displaystyle j$ lies on row $\displaystyle i$ and column $\displaystyle j$). In this case, your transition matrix is correct. If the initial probability distribution is $\displaystyle \begin{bmatrix}0.5 & 0.5\end{bmatrix}$, then after one step it becomes: $\displaystyle \begin{bmatrix}0.5 & 0.5\end{bmatrix}\times \begin{bmatrix}0.7 & 0.3\\0.6& 0.4\end{bmatrix}$. And, on the third day, the distribution is $\displaystyle \begin{bmatrix}0.5 & 0.5\end{bmatrix}\times \begin{bmatrix}0.7 & 0.3\\0.6& 0.4\end{bmatrix}^2$ (the first component of the vector you get after computation is the probability that coin 1 is flipped on the third day).

4. Originally Posted by Laurent
It seems there are different conventions to define transition matrices.
I am so glad that this has been noted by someone else. It has bothered me in at least one other post. I do not know of a textbook commonly used in North America that gives the transition matrix as ‘column’ driven. Can someone point me to a text that uses the column convention?

5. Originally Posted by Plato
I am so glad that this has been noted by someone else. It has bothered me in at least one other post. I do not know of a textbook commonly used in North America that gives the transition matrix as ‘column’ driven. Can someone point me to a text that uses the column convention?
I can't off-hand but I can say that in some parts of Australia this convention is taught at secondary and tertiary levels.

6. Originally Posted by Plato
I am so glad that this has been noted by someone else. It has bothered me in at least one other post. I do not know of a textbook commonly used in North America that gives the transition matrix as ‘column’ driven. Can someone point me to a text that uses the column convention?
You are no doubt referring to my previous random walk post. My professor instructed us in class to use a column driven transitional matrix. Oddly enough, the textbook for the class uses rows.