So, last class, we introduced a simple model for weather in the land of Oz.
In Oz, the weather switches among 3 states every day... Sunny, Rainy, or Snowy. And given the state of the weather today, we know how to predict the state of the weather tomorrow.
Prob( x > y )  Sunny today  Rainy today  Snowy today 

Sunny tomorrow  1/2  1/2  1/4 
Rainy tomorrow  1/4 

1/4 
Snowy tomorrow  1/4  1/2  1/2 
Show how learn composition can be rewritten as \(p(t+1) = A p(t)\) where
\[A := \frac{1}{4} \begin{bmatrix} 2 & 2 & 1 \\ 1 & 0 & 1\\ 1 & 2 & 2\end{bmatrix}\]
We can simulate this process with a very simple python algorithm
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 

One example of the weather we might observer is then ...
0 Sunny
1 Sunny
2 Sunny
3 Sunny
4 Rainy
5 Snowy
6 Snowy
7 Rainy
8 Snowy
9 Sunny
This is not a very general method (though it is fast). Better methods like aliasing exist.
If it is sunny today, what is the chance that it will slow in 3 days? (we don't care about the weather for the next 2 days)
Do an example calculation of weather prediction out the long way.
\[ p(t)^T A^t p(0)\]
Some Markov processes, when run long enough into the future, forget their initial state and approach a stationary distribution. These kinds of markov processes are called ergodic. The simplest test for ergoticity is to see if the process can every get trapped in a subset of statespace. If you can always move from one state to another by some path, then the process is ergodic.
Since the columns some to \(1\), there must be an eigenvalue equal to \(1\).
(has simple eigenvalues and steadystate) The characteristic polynomial \(z^{3}  z^{2} {z}/{16} + {1}/{16} = 0\) factors to \[\left(z  1\right) \left(4 z  1\right) \left(4 z + 1\right) = 0\]
Calculate equilibrium distribution using reduced roweschelon form. (\(IA\) will be singular! But we also want \(p_1 + p_2 + p _ 2 = 1\).)
\[p(\infty) = \frac{1}{5} \begin{bmatrix}2\\1\\2\end{bmatrix}\] So, after forgetting the initial condition, we expect it to be sunny \(2/5\)th of the time, snowy \(2/5\)th of the time, and rainy only \(1/5\)th of the time.