Benchmarking Parity GamesFSEN 2014

Jeroen KeirenOpen University of the Netherlands & VU University Amsterdam

24 April 2015

1 / 19

Where are parity games used?

I Model Checking

I Equivalence Checking

I Satisfiability/Validity of modal logic

I Synthesis

2 / 19

Parity Games

V A set of vertices.→ ⊆ V × V An edge relation.Ω :V → N A priority mapping., Two players (even, odd).

(V ,V) A partition of V .

0

s0

t0

u

0

w

1

v

=even=odd

I Winner?

I Optimal strategies?

3 / 19

Parity Games

V A set of vertices.→ ⊆ V × V An edge relation.Ω :V → N A priority mapping., Two players (even, odd).

(V ,V) A partition of V .

0

s0

t0

u

0

w

1

v

=even=odd

I Winner?

I Optimal strategies?

3 / 19

Parity Games

V A set of vertices.→ ⊆ V × V An edge relation.Ω :V → N A priority mapping., Two players (even, odd).

(V ,V) A partition of V .

0

s0

t0

u

0

w

1

v

=even=odd

I Winner?

I Optimal strategies?

3 / 19

Parity Games

V A set of vertices.→ ⊆ V × V An edge relation.Ω :V → N A priority mapping., Two players (even, odd).

(V ,V) A partition of V .

0

s0

t0

u

0

w

1

v

=even=odd

I Winner?

I Optimal strategies?

3 / 19

Parity Games

V A set of vertices.→ ⊆ V × V An edge relation.Ω :V → N A priority mapping., Two players (even, odd).

(V ,V) A partition of V .

0

s0

t0

u

0

w

1

v

=even=odd

I Winner?

I Optimal strategies?

3 / 19

Parity Games

V A set of vertices.→ ⊆ V × V An edge relation.Ω :V → N A priority mapping., Two players (even, odd).

(V ,V) A partition of V .

0

s0

t0

u

0

w

1

v

=even=odd

I Winner?

I Optimal strategies?

3 / 19

Parity Games

V A set of vertices.→ ⊆ V × V An edge relation.Ω :V → N A priority mapping., Two players (even, odd).

(V ,V) A partition of V .

0

s0

t0

u

0

w

1

v

=even=odd

I Winner?

I Optimal strategies?

3 / 19

Parity Games

V A set of vertices.→ ⊆ V × V An edge relation.Ω :V → N A priority mapping., Two players (even, odd).

(V ,V) A partition of V .

0

s0

t0

u

0

w

1

v

=even=odd

I Winner?

I Optimal strategies?

3 / 19

Parity Games

V A set of vertices.→ ⊆ V × V An edge relation.Ω :V → N A priority mapping., Two players (even, odd).

(V ,V) A partition of V .

0

s0

t0

u

0

w

1

v

=even=odd

I Winner?

I Optimal strategies?

3 / 19

Parity Games

V A set of vertices.→ ⊆ V × V An edge relation.Ω :V → N A priority mapping., Two players (even, odd).

(V ,V) A partition of V .

0

s0

t0

u

0

w

1

v

=even=odd

I Winner?

I Optimal strategies?

3 / 19

Winning Parity Games

Memoryless determinacy

I Partition (W ,W) of V

I Player © has memoryless winning strategy from W©, for© ∈ ,

4 / 19

Solving Parity Games

Solving a parity game:

I Determine partition (W ,W)

Complexity:

I Problem is in NP ∩ co-NP

I Is it in P?

Open!

5 / 19

Solving Parity Games

Solving a parity game:

I Determine partition (W ,W)

Complexity:

I Problem is in NP ∩ co-NP

I Is it in P? Open!

5 / 19

Why benchmark?

Complexity + applications ⇒ active research

Algorithms for:

I solving

I simplifying

I reducing

parity games

How to compare new algorithms to existing ones?

6 / 19

Existing practice

I Only theoretical analysis (big-O)

I Class of games that meets upper bound

I Random games

I (Very) small set of games

Results from different papers not comparable

7 / 19

Requirements on Benchmarks

I Cover broad range of games:I Different problemsI Different structural properties

I Games from the literature

8 / 19

Contributions

I Set of parity games

I List of structural properties

I Analysis of games w.r.t. these properties

9 / 19

Set of parity games

I Model checking:I Communication protocols (C)ABP, BRP, SWPI Cache coherence protocolI Two-player board gamesI Industrial IEEE-1394 link-layer, truck liftI Elevator, Hanoi towers

I Equivalence checking: strong-, weak-, branchingbisimulation of communication protocols

I Validity/satisfiability of LTL, CTL, CTL*, PDL andµ-calculus (using MLSolver)

I Random games (using PGSolver)

I Hard cases (using PGSolver)

10 / 19

Structural properties

I Some properties known to affect complexity of solving:I Number of vertices and edges (“size”)I Number of prioritiesI Width measures (tree-width, DAG-width, etc.)I SCCs

I New: alternation depth (inspired by modal equationsystems)

I And some more. . .

11 / 19

Alternation depth

I Describe complexity more accurately

I Similar to ideas in [Emerson & Lee 1986] for µ-calculus

Three steps (let C ∈ sccs(G ))

1. Nesting depth of v in C is #alternations between evenand odd priorities on paths of descending priorities in C

2. Nesting depth of C is maxnestingdepth(v) | v ∈ C3. Alternation depth of of a parity game is the maximal

nesting depth of its SCCs

12 / 19

Analysis of games w.r.t. structural propertiesVertices vs. edges

101 102 103 104 105 106 107

101

102

103

104

105

106

107

108

Vertices

Edg

es

modelcheckingequivalencemlsolverspecialcasesrandom

13 / 19

Analysis of games w.r.t. structural propertiesAlternation depth

101 102 103 104 105 106 107100

101

102

103

104

Vertices

Alt

erna

tion

dept

h

modelcheckingequivalencemlsolverspecialcasesrandom

14 / 19

Analysis of games w.r.t. structural propertiesDiameter

101 102 103 104100

101

102

103

104

BFS Height

Dia

met

er

modelcheckingequivalencemlsolverspecialcasesrandom

15 / 19

Applications

I Used to assess parity game reductions in [Cranen, K &Willemse 2011,2012]

I Subset of generation process used for benchmarks in [K,Wesselink & Willemse, 2014]

I Confirmed observation from [Friedmann & Lange 2009]:recursive algorithm beats sophisticated algorithms(unpublished)

16 / 19

Summary

I presented:

I A set of parity games

I Structural properties of parity games

I An analysis of the games w.r.t. these properties

17 / 19

Open issues

I Use structural properties to optimise/design algorithms

I Perform large-scale comparison of different algorithms

I Extend set of games with other encodings/more examples

I Design algorithms for computing more complex structuralproperties

18 / 19

Please contribute your own games!

jeroenkeiren.nl

github.com/jkeiren/paritygame-generator

19 / 19