Lecture 21 - Introduction to Markov chains

(previous, next)

(class was delayed because of a fire-alarm, so this lecture was much sorter than usual).

Background

Motivating examples

Define a Markov process

Matrix analysis of simple Markov processes

Weather in the land of OZ -- never rains 2 days in a row

Introduce digraph representation of probabilities of changing from 1 state to the next.

In Oz, the weather switches among 3 states every day... Sunny, Rainy, or Snowy. And given the state of the weather today, we know how to predict the state of the weather tomorrow.

Prob( x -> y ) Sunny today Rainy today Snowy today
Sunny tomorrow 1/2 1/2 1/4
Rainy tomorrow 1/4
0
1/4
Snowy tomorrow 1/4 1/2 1/2

We can also represent the transitions between states with a labelled directed graph.

Show how learn composition can be rewritten as \(p(t+1) = A p(t)\) where

\[A := \frac{1}{4} \begin{bmatrix} 2 & 2 & 1 \\ 1 & 0 & 1\\ 1 & 2 & 2\end{bmatrix}\]