Games:Expectimax

Andrea DanylukSeptember 25, 2013

Announcements

• Programming Assignment 2: Games– Anyone still looking for a partner?– Partner can’t be the same as for Assignment 1

Today’s Lecture

• Just a bit more on α-β pruning• Games with more than 2 players• Expectimax• Pacman

8 47 381911

2

619 5

α = 8Β = +inf

8

8 9

2α = 8Β = +inf

8

Is there a problem here???

function αβSEARCH(state) returns an action a v = MAX-VALUE(state, -infinity, +infinity)

return action a in ACTIONS(state) with value v

function MAX-VALUE(state, α, β) returns a utility value v if TERMINAL-TEST(state) then return UTILITY(state) v = -infinity for each a in ACTIONS(state) do v = MAX(v, MIN-VALUE(RESULT(state, a), α, β)) if v ≥ β then return v α = MAX(α, v) return v

function MIN-VALUE(state, α, β) returns a utility value v if TERMINAL-TEST(state) then return UTILITY(state) v = infinity for each a in ACTIONS(state) do v = MIN(v, MAX-VALUE(RESULT(state, a), α, β)) if v ≤ α then return v β = MIN(β, v) return v

Node Order Matters

5724810873

B DC

A MAX

min

Best ordering formaximum pruning?

Move Ordering

• Can we order moves in such a way that α-β will prune more rather than less?

• Chess?• Connect 4?• Don’t worry about this for Pacman assignment

Properties of α-β

• Pruning does not affect the final result (but be careful)

• Good move ordering improves effectiveness of pruning

• If search depth is d, what is the time complexity of minimax?– O(bd)

• With perfect pruning, get down to O(b d/2)– Doubles solvable depth

[Adapted from Russell]

Games with > 2 players

• Up to 4 players• Player tries to place all

21 of his/her pieces• Hopes to block

opponents from placing their pieces

A Four-Player Game

Blue to move

Yellow to move

Red

Green

…

… …

… … … …

(5,7,2,9) (4,3,1,1) (2,6,10,2)(6,4,19,8)

(5,7,2,9) (3,5,1,8) (8,7,6,5) (9,7,1,3) (4,3,2,1) (4,4,4,4) (6,4,19,8) (3,5,7,9)

(5,7,2,9) (8,7,6,5) (4,4,4,4) (6,4,19,8)

(5,7,2,9) (4,4,4,4)

(5,7,2,9)

Multi-Player Games

• Evaluation function returns a vector of utilities• Each player chooses the move that maximizes

its utility.

• Is Pacman with 2 ghosts a multi-player game?

Stochastic Games

Expectimax

• Introduce chance nodes into a minimax tree

2 4 7 4 6 0 5 -2

2 4 0 -2

0.5 0.5 0.5 0.5

3 -1

Expectimax

• Introduce chance nodes into a minimax tree

Expectimax

• Introduce chance nodes into a minimax tree

0.5 0.5

Expectimax

• Introduce chance nodes into a minimax tree

2 4

2

0.5 0.5

Expectimax

• Introduce chance nodes into a minimax tree

2 4

2

0.5 0.5

1+

Expectimax

• Introduce chance nodes into a minimax tree

2 4 7 4

2 4

0.5 0.5

1+

Expectimax

• Introduce chance nodes into a minimax tree

2 4 7 4

2 4

0.5 0.5

3

Expectimax

• Introduce chance nodes into a minimax tree

2 4 7 4

2 4

0.5 0.5 0.5 0.5

3

Expectimax

• Introduce chance nodes into a minimax tree

2 4 7 4 6 0

2 4 0

0.5 0.5 0.5 0.5

3

Expectimax

• Introduce chance nodes into a minimax tree

2 4 7 4 6 0

2 4 0

0.5 0.5 0.5 0.5

3 0+

Expectimax

• Introduce chance nodes into a minimax tree

2 4 7 4 6 0 5 -2

2 4 0 -2

0.5 0.5 0.5 0.5

3 0+

Expectimax

• Introduce chance nodes into a minimax tree

2 4 7 4 6 0 5 -2

2 4 0 -2

0.5 0.5 0.5 0.5

3 -1

Expectimax

• Introduce chance nodes into a minimax tree

2 4 7 4 6 0 5 -2

2 4 0 -2

0.5 0.5 0.5 0.5

3 -1

[From Russell]

Expectimax

…

Evaluation Functions Revisited

1 20 20 4001 2 2 4

Behavior is preserved under any monotonic transformation of EVAL

[From Russell]

Evaluation Functions Revisited

Behavior is preserved only under positive linear transformation of EVAL

[From Russell]

20 20 30 30 1 1 400 4002 2 3 3 1 1 4 4

2 3 1 4 20 30 1 400

.9 .1 .9 .1.9 .1 .9 .1

2.1 1.3 21 40.9

Expectimax Pacman

def value(s)if s is a max node return maxValue(s)if s is an exp node return expValue(s)if s is a terminal node return eval(s)

def maxValue(s)values = [value(s1) for s1 in succ(s)]return max(values)

def expValue(s)values = [value(s1) for s1 in succ(s)]weights = [prob(s,s1) for s1 in succ(s)]return expectation(values, weights)

[Verbatim from CS 188]

Depth-limited Expectimax

1 7 3 4 6 2 5 1

1 ply

Don’t forget: Magnitudes of the utilities / heuristic evaluations need to be meaningful.

Why Pacman Can Starve

He knows his score can go up by eating now.He knows his score can go up by eating later.Within this search window there are no other eating opportunities.