discrete markov stochastic process

Jan 2009
hi guys

I have a simple problem driving me crazy :)

the topic is stochastic process and markov process, this is the problem:

assuming every day's weather has only two options: rainy or sunny.
the probability that in certain day the weather is the same as the day before is \(\displaystyle p\). the probability of change in weather is \(\displaystyle q=1-p\).

now i will define \(\displaystyle (x_{[n]})_{n>=0}\) as a discrete time stochastic process as: \(\displaystyle x_{[n]}=1\) if the n'th day was rainy and \(\displaystyle x_{[n]}=0\) if the n'th day was sunny.

the question is: given that today, n=0, is a rainy day, find the probability to rain in the n'th day. clue: you can organize the data in a matrix.
i've tried few approaches but nothing comes out, i know its simple, will be happy for assistance. thanks :)


MHF Hall of Honor
Mar 2008
P(I'm here)=1/3, P(I'm there)=t+1/3

Let A be the transition matrix. It's a 2x2 matrix because there are only two states. Since the probability of staying in the same state the next day is p, you will have p in the diagonal. And q otherwise.

\(\displaystyle A=\begin{pmatrix} p&q \\ q&p\end{pmatrix}\)

it can be diagonalized : its eigenvalues are 1 (eigenvector (1,1)) and p-q (eigenvector (1,-1)).
So \(\displaystyle A=PDP^{-1}\), where \(\displaystyle P=\begin{pmatrix}1&1\\1&-1\end{pmatrix}\) (matrix of the eigenvectors) and \(\displaystyle D=\begin{pmatrix} 1&0 \\ 0&p-q\end{pmatrix}\)
Also note that \(\displaystyle P^{-1}=\frac 12\begin{pmatrix} 1&1 \\ 1&-1\end{pmatrix}\)

Hence \(\displaystyle A^n=P\begin{pmatrix} 1^n & 0 \\ 0&(p-q)^n\end{pmatrix}\)
But the element \(\displaystyle (i,j)\) in \(\displaystyle A^n\) denotes \(\displaystyle P(X_n=j\mid X_0=i)\) (more or less)
And you're looking for \(\displaystyle P(X_n=0 \mid X_0=0)\), so you'll be looking at the (1,1) element of the matrix \(\displaystyle A^n\)