Markov Chains
Model probalistic state transitions
Markov property
Transition probabilities depend
only
on the current state
Example
State
Action
Successor
p
BUMPED
FORWARD
BUMPED
0.99
BUMPED
FORWARD
CLEAR
0.01
BUMPED
BACK
BUMPED
0.01
BUMPED
BACK
CLEAR
0.99
CLEAR
FORWARD
BUMPED
0.3
CLEAR
FORWARD
CLEAR
0.7
CLEAR
BACK
BUMPED
0.01
CLEAR
BACK
CLEAR
0.99
(next)