Lecture 22 - Introduction to Markov chains (continued)

(previous, next)

Weather in the land of OZ -- never rains 2 days in a row

So, last class, we introduced a simple model for weather in the land of Oz.

In Oz, the weather switches among 3 states every day... Sunny, Rainy, or Snowy. And given the state of the weather today, we know how to predict the state of the weather tomorrow.

Prob( x -> y ) Sunny today Rainy today Snowy today
Sunny tomorrow 1/2 1/2 1/4
Rainy tomorrow 1/4
0
1/4
Snowy tomorrow 1/4 1/2 1/2

Show how learn composition can be rewritten as \(p(t+1) = A p(t)\) where

\[A := \frac{1}{4} \begin{bmatrix} 2 & 2 & 1 \\ 1 & 0 & 1\\ 1 & 2 & 2\end{bmatrix}\]

We can simulate this process with a very simple python algorithm

[Show code]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#!/usr/bin/env python2

from scipy import rand

s = 'Sunny'
r = 'Rainy'
w = 'Snowy'

transitions = {s: (s,s,r,w), r:(s,s,w,w), w:(s,r,w,w)}
states = transitions.keys()

def rndStep(state):
    return transitions[state][int(rand()*4)]

def rndPath(t, x, tmax):
    # x is the initial state
    # t is the initial time
    # tmax is the time to stop at
    while t < tmax:
        yield t, x
        t, x = t + 1, rndStep(x)

def main():
    for t, x in rndPath(0,s,10):
        print t, x

main()

One example of the weather we might observer is then ...

0 Sunny
1 Sunny
2 Sunny
3 Sunny
4 Rainy
5 Snowy
6 Snowy
7 Rainy
8 Snowy
9 Sunny

This is not a very general method (though it is fast). Better methods like aliasing exist.

Path integrals

If it is sunny today, what is the chance that it will slow in 3 days? (we don't care about the weather for the next 2 days)

Do an example calculation of weather prediction out the long way.

\[ p(t)^T A^t p(0)\]

Steady-state calculations

Some Markov processes, when run long enough into the future, forget their initial state and approach a stationary distribution. These kinds of markov processes are called ergodic. The simplest test for ergoticity is to see if the process can every get trapped in a subset of state-space. If you can always move from one state to another by some path, then the process is ergodic.

Since the columns some to \(1\), there must be an eigenvalue equal to \(1\).

(has simple eigenvalues and steady-state) The characteristic polynomial \(z^{3} - z^{2} -{z}/{16} + {1}/{16} = 0\) factors to \[\left(z - 1\right) \left(4 z - 1\right) \left(4 z + 1\right) = 0\]

Calculate equilibrium distribution using reduced row-eschelon form. (\(I-A\) will be singular! But we also want \(p_1 + p_2 + p _ 2 = 1\).)

\[p(\infty) = \frac{1}{5} \begin{bmatrix}2\\1\\2\end{bmatrix}\] So, after forgetting the initial condition, we expect it to be sunny \(2/5\)th of the time, snowy \(2/5\)th of the time, and rainy only \(1/5\)th of the time.