UC Berkeley GuidedSampler: EECS Department Rafael Dutra...
Transcript of UC Berkeley GuidedSampler: EECS Department Rafael Dutra...
![Page 1: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/1.jpg)
GuidedSampler:Coverage-guided Samplingof SMT SolutionsRafael Dutra, Jonathan Bachrach, Koushik SenEECS DepartmentUC Berkeley
Formal Methods in Computer-Aided DesignOctober 25, 2019
![Page 2: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/2.jpg)
A SMT solver can generate one solution:
Constraint Sampling
2
mem[0] mem[1]
σ0 1 0 0 0 1 0 0 0
∧(x + y = 4 ∧ x ≥ 0 ∧ x < 4)∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4),wherex = mem[0],y = mem[1],mem’ = store(mem, mem[0], -1 * mem[mem[0]])
Input: SMT formula φ
![Page 3: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/3.jpg)
Goal: Generate many solutions to φ
Constraint Sampling
3
mem[0] mem[1]
σ0
σ1
σ2
σ3
σ4
σ5
1 0 0 0 1 0 0 0
0 0 0 0 1 0 1 0
1 1 0 0 1 0 0 1
1 0 0 0 0 1 0 0
0 1 0 0 1 0 1 1
0 0 0 0 0 1 1 0
∧(x + y = 4 ∧ x ≥ 0 ∧ x < 4)∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4),wherex = mem[0],y = mem[1],mem’ = store(mem, mem[0], -1 * mem[mem[0]])
Input: SMT formula φ
![Page 4: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/4.jpg)
● Synthesis● Symbolic execution
Motivation: Sampling Solutions
● Thoroughly exercising some target functionality● Constrained-Random Verification
4
int4 x, y, z, w;int4 mem[4] = {x, y, z, w};for (int4 i = 0; i < 4; ++i) {
mem[mem[i]] *= -1;}
i < 4
mem[0] < 0∨ mem[0] ≥ 4
![Page 5: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/5.jpg)
SMT: Satisfiability Modulo Theories
5
SMT formula φ
∧(mem[0] ≥ 0 ∧ mem[0] < 4)∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4),wheremem’ = store(mem, mem[0], -1 * mem[mem[0]])
mem ∈ Array(BV[4], BV[4])
![Page 6: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/6.jpg)
mem ∈ Array(BV[4], BV[4])
SMT: Satisfiability Modulo Theories
6
SMT formula φBit-vector
∧(mem[0] ≥ 0 ∧ mem[0] < 4)∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4),wheremem’ = store(mem, mem[0], -1 * mem[mem[0]])
![Page 7: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/7.jpg)
SMT: Satisfiability Modulo Theories
7
SMT formula φBit-vector
Array
mem ∈ Array(BV[4], BV[4])
∧(mem[0] ≥ 0 ∧ mem[0] < 4)∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4),wheremem’ = store(mem, mem[0], -1 * mem[mem[0]])
![Page 8: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/8.jpg)
State of the art
● SMTSampler (our prior work)○ Efficient generation of solutions for SMT formulas
● Markov Chain Monte Carlo (MCMC)○ Works for linear constraints and can generate biased solutions
● Constraint solver heuristics○ Can be expensive, requiring one solver call per solution
● Universal hashing○ Expensive, but can guarantee exact distribution of solution
● Weighted Sampling○ Literal-weighted distributions: WAPS
8
![Page 9: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/9.jpg)
Goal: Generate solutions to φ
SMTSampler
9
mem[0] mem[1]
σ0
σ1
σ2
σ3
σ4
σ5
1 0 0 0 1 0 0 0
0 0 0 0 1 0 1 0
1 1 0 0 1 0 0 1
1 0 0 0 0 1 0 0
0 1 0 0 1 0 1 1
0 0 0 0 0 1 1 0
∧(x + y = 4 ∧ x ≥ 0 ∧ x < 4)∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4),wherex = mem[0],y = mem[1],mem’ = store(mem, mem[0], -1 * mem[mem[0]])
Input: SMT formula φ
![Page 10: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/10.jpg)
Goal: Generate solutions to φ
Coverage-guided Sampling
10
mem[0] mem[1]
σ0
σ1
σ2
σ3
σ4
σ5
1 0 0 0 1 0 0 0
0 0 0 0 1 0 1 0
1 1 0 0 1 0 0 1
1 0 0 0 0 1 0 0
0 1 0 0 1 0 1 1
0 0 0 0 0 1 1 0
∧(x + y = 4 ∧ x ≥ 0 ∧ x < 4)∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4),wherex = mem[0],y = mem[1],mem’ = store(mem, mem[0], -1 * mem[mem[0]])
mem’[1] < 0
Input: SMT formula φ
Input: Coverage predicates
mem’[1] ≥ 4 mem’[0] < 0
ψ1 ψ2 ψ3
![Page 11: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/11.jpg)
Goal: Generate solutions to φ such that the predicates ψ1, ψ2, …, ψn are covered uniformly
Coverage-guided Sampling
11
mem[0] mem[1] ψ1 ψ2 ψ3
σ0
σ1
σ2
σ3
σ4
σ5
1 0 0 0 1 0 0 0
0 0 0 0 1 0 1 0
1 1 0 0 1 0 0 1
1 0 0 0 0 1 0 0
0 1 0 0 1 0 1 1
0 0 0 0 0 1 1 0
∧(x + y = 4 ∧ x ≥ 0 ∧ x < 4)∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4),wherex = mem[0],y = mem[1],mem’ = store(mem, mem[0], -1 * mem[mem[0]])
mem’[1] < 0
Input: SMT formula φ
0 1 0
0 1 0
0 0 1
1 1 0
0 1 0
1 1 0
mem’[1] ≥ 4 mem’[0] < 0
Input: Coverage predicates
ψ1 ψ2 ψ3
![Page 12: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/12.jpg)
Goal: Generate solutions to φ such that the predicates ψ1, ψ2, …, ψn are covered uniformly
Coverage-guided Sampling
12
mem[0] mem[1] ψ1 ψ2 ψ3
σ0
σ1
σ2
σ3
σ4
σ5
1 0 0 0 1 0 0 0
0 0 0 0 1 0 1 0
1 1 0 0 1 0 0 1
1 0 0 0 0 1 0 0
0 1 0 0 1 0 1 1
0 0 0 0 0 1 1 0
∧(x + y = 4 ∧ x ≥ 0 ∧ x < 4)∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4),wherex = mem[0],y = mem[1],mem’ = store(mem, mem[0], -1 * mem[mem[0]])
mem’[1] < 0
Input: SMT formula φ
0 1 0
0 1 0
0 0 1
1 1 0
0 1 0
1 1 0
mem’[1] ≥ 4 mem’[0] < 0
Input: Coverage predicates
ψ1 ψ2 ψ3
![Page 13: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/13.jpg)
Uniformity over Coverage Classes
13
![Page 14: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/14.jpg)
Uniformity over Coverage Classes
14
![Page 15: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/15.jpg)
Uniformity over Coverage Classes
15
![Page 16: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/16.jpg)
Challenges
● Coverage of the formula might still not be ideal even using state-of-the-art approaches, such as SMTSampler
● User might be interested in a specific notion of coverage for the produced solutions
16
![Page 17: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/17.jpg)
GuidedSampler
17
![Page 18: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/18.jpg)
GuidedSampler
Our goals:
● Sample solutions from a formula φ, but have the distribution determined by the coverage predicates ψ1, ψ2, …, ψn
● Uniformly sample solutions from the different coverage classes
● Uniformly sample within each coverage class
Our approach:
● Compute simple mutations that can be applied to one solution to generate another solution from a different class
● Combine those mutations together to generate a large number of new solutions
18
![Page 19: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/19.jpg)
19
Formula φ
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
Coverage Predicates
...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
ψ1
ψ2
ψ3
![Page 20: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/20.jpg)
20
Formula φx = mem[0]y = mem[1]
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
![Page 21: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/21.jpg)
21
Formula φx = mem[0]y = mem[1]
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])ψ1 ψ2 ψ3
0 1 1RandomClass
...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
![Page 22: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/22.jpg)
22
Solution σ 1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])ψ1 ψ2 ψ3
0 1 1
ψ1 ψ2 ψ3
0 1 0...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
RandomClass
![Page 23: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/23.jpg)
23
Solution σ 1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
ψ1 ψ2 ψ3
0 1 0
MAX-SMT ...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
![Page 24: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/24.jpg)
24
Solution σ 1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
MAX-SMT
Hard constraints● φ● ψ1 ≠ 0
Soft constraints● ψ2 = 1● ψ3 = 0● x1 = 0● x2 = 0● x3 = 0● y0 = 1● y1 = 0● y2 = 0● y3 = 0
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
ψ1 ψ2 ψ3
0 1 0
MAX-SMT ...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
![Page 25: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/25.jpg)
σ1
25
Solution σ
0 0 0 0 1 0 1 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
MAX-SMT
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
ψ1 ψ2 ψ3
0 1 0
1 1 0
...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
![Page 26: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/26.jpg)
σ1
26
Solution σ
0 0 0 0 1 0 1 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
ψ1 ψ2 ψ3
0 1 0
MAX-SMT ...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
![Page 27: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/27.jpg)
σ1
27
Solution σ
0 0 0 0 1 0 1 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
![Page 28: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/28.jpg)
σ1
28
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
σ2
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0
0 0 0
...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
![Page 29: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/29.jpg)
σ1
29
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
σ2
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
![Page 30: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/30.jpg)
σ1
30
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
σ2
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
Coverage Predicates
ψ1
ψ2
ψ3
![Page 31: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/31.jpg)
σ1
31
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
σ2 σ3
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
0 0 1
Coverage Predicates
ψ1
ψ2
ψ3
![Page 32: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/32.jpg)
σ1
32
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
σ2 σ3
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 33: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/33.jpg)
σ1
33
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0
σ2 σ3
δ31 0 0 0 0 0 1 0 δ2δ1
MAX-SMT
= σ ⊕ σ1 = σ ⊕ σ2 = σ ⊕ σ3
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 34: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/34.jpg)
δ12
σ1
34
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0
σ2 σ3
δ31 0 0 0 0 0 1 0 δ2δ1
1 1 0 0 0 0 1 1
MAX-SMT
= δ1 ∨ δ2
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 35: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/35.jpg)
σ12
δ12
σ1
35
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0
σ2 σ3
δ31 0 0 0 0 0 1 0 δ2δ1
1 1 0 0 0 0 1 1
0 1 0 0 1 0 1 1
MAX-SMT
= σ ⊕ δ12
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
ψ1 ψ2 ψ3
1 0 0
...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 36: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/36.jpg)
σ12
δ12
σ1
36
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0
σ2 σ3
δ31 0 0 0 0 0 1 0 δ2δ1
1 1 0 0 0 0 1 1
0 1 0 0 1 0 1 1
MAX-SMT
= σ ⊕ δ12
Why does it work?● δ1 and δ2 are a minimal set of bits that can be
flipped and preserve the satisfiability of the formula● It’s likely that the formula has some clauses
establishing a relation between those bits● Those clauses will likely still be satisfied when
flipping both the bits in δ1 and δ2
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
ψ1 ψ2 ψ3
1 0 0
...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 37: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/37.jpg)
σ12
δ12
σ1
37
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0
σ2 σ3
δ31 0 0 0 0 0 1 0 δ2δ1
1 1 0 0 0 0 1 1
0 1 0 0 1 0 1 1
MAX-SMT
= σ ⊕ δ12
⇐ And new sample σ12 is likelyfrom a new coverage class
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
ψ1 ψ2 ψ3
1 0 0
...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 38: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/38.jpg)
σ12
δ12
σ1
38
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0
σ2 σ3
δ31 0 0 0 0 0 1 0 δ2δ1
1 1 0 0 0 0 1 1
0 1 0 0 1 0 1 1
MAX-SMT
= σ ⊕ δ12
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
ψ1 ψ2 ψ3
1 0 0
...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 39: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/39.jpg)
σ12
δ12
σ1
39
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0
σ2 σ3
δ31 0 0 0 0 0 1 0 δ2δ1
1 1 0 0 0 0 1 1
0 1 0 0 1 0 1 1
MAX-SMT
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
δ131 0 0 0 1 1 1 0
= δ1 ∨ δ3
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 40: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/40.jpg)
σ12
δ12 δ13
σ13
σ1
40
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0
1 0 0 0 1 1 1 0
0 0 0 0 0 1 1 0
σ2 σ3
δ31 0 0 0 0 0 1 0 δ2δ1
1 1 0 0 0 0 1 1
0 1 0 0 1 0 1 1
MAX-SMT
= σ ⊕ δ13
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
ψ1 ψ2 ψ3
1 0 1
...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 41: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/41.jpg)
σ12
δ12 δ13 δ23
σ13
σ1
41
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0
1 0 0 0 1 1 1 0
0 0 0 0 0 1 1 0
σ2 σ3
δ31 0 0 0 0 0 1 0 δ2δ1
1 1 0 0 0 0 1 1
0 1 0 0 1 0 1 1
0 1 0 0 1 1 0 1
MAX-SMT
= δ2 ∨ δ3
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 42: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/42.jpg)
σ12
δ12 δ13 δ23
σ13
σ1
42
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0
1 0 0 0 1 1 1 0
0 0 0 0 0 1 1 0
σ2 σ3
δ31 0 0 0 0 0 1 0 δ2δ1
1 1 0 0 0 0 1 1
0 1 0 0 1 0 1 1
0 1 0 0 1 1 0 1
MAX-SMT
σ231 1 0 0 0 1 0 1
= σ ⊕ δ23
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
ψ1 ψ2 ψ3
0 0 1
...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 43: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/43.jpg)
σ12 σ13
σ1
43
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 0 0 0 0 1 1 0
σ2 σ3
0 1 0 0 1 0 1 1
MAX-SMT
σ231 1 0 0 0 1 0 1
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 44: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/44.jpg)
σ12 σ13
σ1
44
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 0 0 0 0 1 1 0
σ2 σ3
0 1 0 0 1 0 1 1
MAX-SMT
σ231 1 0 0 0 1 0 1
17 / 18 valid solutions
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 45: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/45.jpg)
σ12 σ13
σ1
45
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 0 0 0 0 1 1 0
σ2 σ3
0 1 0 0 1 0 1 1
MAX-SMT
σ231 1 0 0 0 1 0 1
σ1230 1 0 0 0 1 1 1
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
ψ1 ψ2 ψ3
1 0 1
...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 46: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/46.jpg)
σ12 σ13
σ1
46
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 0 0 0 0 1 1 0
σ2 σ3
0 1 0 0 1 0 1 1
MAX-SMT
σ231 1 0 0 0 1 0 1
σ1230 1 0 0 0 1 1 1
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
ψ1 ψ2 ψ3
1 0 1 Repeated class
...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 47: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/47.jpg)
σ12 σ13
σ1
47
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 0 0 0 0 1 1 0
σ2 σ3
0 1 0 0 1 0 1 1
MAX-SMT
σ231 1 0 0 0 1 0 1
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 48: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/48.jpg)
σ12 σ13
σ1
48
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 0 0 0 0 1 1 0
σ2 σ3
0 1 0 0 1 0 1 1
MAX-SMT
σ231 1 0 0 0 1 0 1
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
At most ngenerate atomic mutations
=O(n6)mutations: NO MAX-SMT ( ) n
6 samples by combining
MAX-SMT calls to
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 49: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/49.jpg)
σ12 σ13
σ1
49
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 0 0 0 0 1 1 0
σ2 σ3
0 1 0 0 1 0 1 1
MAX-SMT
σ231 1 0 0 0 1 0 1
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
At most 50generate atomic mutations
15 890 700mutations: NO MAX-SMT
samples by combining
MAX-SMT calls to
MAX-SMT
ψ1 ψ2 ψ3
0 1 0......
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 50: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/50.jpg)
σ12 σ13
σ1
50
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 0 0 0 0 1 1 0
σ2 σ3
0 1 0 0 1 0 1 1
MAX-SMT
σ231 1 0 0 0 1 0 1
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 51: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/51.jpg)
...
mem’[1] < 0
mem’[1] ≥ 4
mem’[0] < 0
σ12 σ13
σ1
51
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 0 0
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0Random assignment σ’ x0 x1 x2 x3 y0 y1 y2 y3
0 0 0 0 0 1 1 0
σ2 σ3
0 1 0 0 1 0 1 1
MAX-SMT
σ231 1 0 0 0 1 0 1
Formula φx = mem[0]y = mem[1]
(mem[0] ≥ 0 ∧ mem[0] < 4) ∧ (mem’[1] < 0 ∨ mem’[1] ≥ 4), where mem’ = store(mem, mem[0], -1* mem[mem[0]])
MAX-SMT
ψ1 ψ2 ψ3
0 1 0...
Coverage Predicates
ψ1
ψ2
ψ3
![Page 52: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/52.jpg)
Random assignment
σ′
52
Key Ideas
![Page 53: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/53.jpg)
σ
Random assignment
Base solution
σ′
53
Key Ideas
● M3: In the MAX-SMT query to generate σ, set coverage predicates to random values
![Page 54: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/54.jpg)
σ2
σ1
σ
Random assignment
Base solution
Closest solutions
σ′
54
σ3σ4
Key Ideas
● M3: In the MAX-SMT query to generate σ, set coverage predicates to random values
● M1: Find neighboring solutions that flip coverage predicates
![Page 55: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/55.jpg)
Random assignment
Base solution
Closest solutions
Generated samples
σ2 σ12
σ1
σσ′
55
σ3σ4
Key Ideas
● M3: In the MAX-SMT query to generate σ, set coverage predicates to random values
● M1: Find neighboring solutions that flip coverage predicates
● M2: Whenever generating a new sample, discard it if it’s from a repeated coverage class
![Page 56: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/56.jpg)
σ2 σ12
σ1
σ
Random assignment
Base solution
Closest solutions
Generated samples
σ′
56
σ3σ4
Key Ideas
● M3: In the MAX-SMT query to generate σ, set coverage predicates to random values
● M1: Find neighboring solutions that flip coverage predicates
● M2: Whenever generating a new sample, discard it if it’s from a repeated coverage class
![Page 57: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/57.jpg)
σ2 σ12
σ1
σ
Random assignment
Base solution
Closest solutions
Generated samples
σ′
57
σ3σ4
Key Ideas
● M3: In the MAX-SMT query to generate σ, set coverage predicates to random values
● M1: Find neighboring solutions that flip coverage predicates
● M2: Whenever generating a new sample, discard it if it’s from a repeated coverage class
![Page 58: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/58.jpg)
Implementation
● Implemented in C++ using Z3 as the constraint solver● https://github.com/RafaelTupynamba/GuidedSampler
58
![Page 59: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/59.jpg)
Experiments on SMT-LIB
We evaluated GuidedSampler on 213 industrial benchmarks from 22 classes.
59
Benchmark Class Average # Nodes Average # Bits
QF_AUFBV/ecc 179 1931
QF_ABV/bmc-arrays 855 53
QF_ABV/stp_samples 1139 192
QF_BV/bmc-bv-svcomp14 7518 7607
QF_BV/tacas07 8812 16620
QF_BV/sage/app8 978 1047
![Page 60: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/60.jpg)
Experiments
We compared 6 approaches for SMT sampling:● BH: Baseline with hard constraints● BS: Baseline with soft constraints● S0: SMTSampler [1]● S1 = S0 + M1 (flipping coverage predicates to generate neighboring solutions)● S2 = S0 + M1 + M2 (discarding solutions from repeated classes)● S3 = S0 + M1 + M2 + M3: GuidedSampler (randomize class of base solution)
[1] Rafael Dutra, Jonathan Bachrach and Koushik Sen. 2018. SMTSampler: Efficient Stimulus Generation from Complex SMT Constraints. In ICCAD 2018. 60
![Page 61: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/61.jpg)
Coverage Predicates
● Internal Predicates○ Look at values of internal nodes○ Analogous to internal wires in a circuit○ General notion of coverage from the
formula itself
● Random Predicates○ Random formulas generated from a
grammar including variables of φ○ Problem-specific notion of coverage
61
![Page 62: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/62.jpg)
Experiments: Unique Coverage Classes
62
Higher is better
GuidedSampler vs. BH Baseline
Number of uniquecoverage classesper time
![Page 63: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/63.jpg)
Experiments: Unique Coverage Classes
63
Higher is better
GuidedSampler vs. BS Baseline
Number of uniquecoverage classesper time
![Page 64: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/64.jpg)
Experiments: Unique Coverage Classes
64
Higher is better
GuidedSampler vs. SMTSampler
Number of uniquecoverage classesper time
![Page 65: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/65.jpg)
Experiments: Unique Coverage Classes
65
Higher is better
S3 = GuidedSamplerS0 = SMTSamplerBS, BH: baselines
Number of uniquecoverage classesper time
![Page 66: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/66.jpg)
Experiments: Uniformity over Coverage Classes
66
S3 = GuidedSamplerS0 = SMTSamplerBS, BH: baselines
→ GuidedSampler generated > 100 000 classes
![Page 67: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/67.jpg)
Discussion
● The most important modification is M1, which allows covering 3.1 times more classes in average
● M1 and M2 are also essential for producing a more uniform distribution over coverage classes.
● Similar results for internal predicates and random predicates
67
Modifications:
● M1: Flipping coverage predicates to compute neighboring solutions
● M2: Discarding new solutions that repeat a previously seen coverage class
● M3: Randomizing coverage class of initial base solution
![Page 68: UC Berkeley GuidedSampler: EECS Department Rafael Dutra ...theory.stanford.edu/~barrett/fmcad/slides/18_Dutra.pdf · Rafael Dutra, Jonathan Bachrach, Koushik Sen EECS Department UC](https://reader034.fdocument.org/reader034/viewer/2022042810/5f9d6d0a0b85cd3e5f7fb51a/html5/thumbnails/68.jpg)
Conclusion
● Generating lots of solutions efficiently given an SMT formula
● Generate millions of solutions with tens of solver calls
● Achieve better coverage of the constraint space, even for user-defined coverage classes
68
σ12 = σ ⊕ δ12
δ12 = δ1 ∨ δ2
σ1
Solution σ
0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1
1 0 0 0 1 0 0 0
0 1 0 0 0 0 0 1
σ2
1 0 0 0 0 0 1 0 δ2δ1
1 1 0 0 0 0 1 1
0 1 0 0 1 0 1 1
ψ1 ψ2 ψ3
0 1 0
ψ1 ψ2 ψ3
1 0 0
MAX-SMT
x0 x1 x2 x3 y0 y1 y2 y3