CS537 Randomness in Computing - cs-people.bu.edu
Transcript of CS537 Randomness in Computing - cs-people.bu.edu
9/14/2021
Randomness in Computing
LECTURE 3Last time• Probability amplification
• Verifying matrix multiplication
Today• Finish verifying matrix
multiplication
• Probability amplification:
Bayesian approach
• Randomized Min-Cut
Sofya Raskhodnikova;Randomness in Computing
Useful conditional probability facts
Let A be an event and let 𝐸1, … , 𝐸𝑛 be mutually disjoint
events whose union is Ω.
• Total probability law.
Pr 𝐴 =
𝑖∈ 𝑛
Pr[𝐴 ∩ 𝐸𝑖] =
𝑖∈ 𝑛
Pr[𝐴 ∣ 𝐸𝑖] ⋅ Pr 𝐸𝑖 .
• Bayes’ Law. Assume Pr 𝐴 ≠ 0. Then, for all 𝑗 ∈ 𝑛 ,
Pr 𝐸𝑗 𝐴 =Pr[𝐸𝑗 ∩ 𝐴]
Pr[𝐴]
=Pr 𝐴|𝐸𝑗 ⋅ Pr 𝐸𝑗
σ𝑖∈ 𝑛 Pr[𝐴 ∣ 𝐸𝑖] ⋅ Pr[𝐸𝑖].
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
Tennis match
You will play a tennis match against opponent X or Y.
If X is chosen, you win with probability 0.7.
If Y is chosen, you win with probability 0.3.
Your opponent is chosen by flipping a coin with bias 0.6
in favor of X.
What is your probability of winning?
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
Review question: tennis match
You will play a tennis match against opponent X or Y.
If X is chosen, you win with probability 0.7.
If Y is chosen, you win with probability 0.3.
Your opponent is chosen by flipping a coin with bias 0.6 in favor of X.
What is your probability of winning?
A. < 0.3
B. In the interval [0.3,0.4).
C. In the interval [0.4,0.55).
D. In the interval [0.55,0.7).
E. ≥0.7
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
Review question: balls and bins
We have two bins with balls.
• Bin 1 contains 3 black balls and 2 white balls.
• Bin 2 contains 1 black ball and 1 white ball.
We pick a bin uniformly at random. Then we pick a ball
uniformly at random from that bin.
What is the probability that we picked bin 1, given that we
picked a white ball?
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
Task: Given three 𝑛 × 𝑛 matrices 𝐴, 𝐵, 𝐶, verify if 𝐴 ⋅ 𝐵 = 𝐶.
Idea: Pick a random vector ത𝒓 and check if 𝐴 ⋅ 𝐵 ⋅ ത𝒓 = 𝐶 ⋅ ത𝒓.
Running time: Three matrix-vector multiplications: O 𝑛2 time.
Correctness: If 𝐴 ⋅ 𝐵 = 𝐶, the algorithm always accepts.
Probability Amplification: With 𝑘 repetitions, error probability ≤ 2−𝑘
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
O(𝒏𝟐) multiplications for each matrix-vector product
§1.3 (MU) Verifying Matrix Multiplication
1. Choose a random 𝑛-bit vector ҧ𝑟 by making each bit 𝑟𝑖independently 0 or 1 with probability 1/2 each.
2. 𝐀𝐜𝐜𝐞𝐩𝐭 if 𝐴 ⋅ (𝐵 ⋅ ത𝒓) = 𝐶 ⋅ ത𝒓; o. w. 𝐫𝐞𝐣𝐞𝐜𝐭.
(input: 𝑛 × 𝑛 matrices A, B, C)Algorithm Basic Frievalds
Theorem
If 𝐴 ⋅ 𝐵 ≠ 𝐶, Basic-Frievalds accepts with probability ≤ 1/2.
Analysis of Error Probability
Proof: Suppose 𝐴 ⋅ 𝐵 ≠ 𝐶 and let 𝐷 = 𝐴𝐵 − 𝐶
𝐴 ⋅ 𝐵 ⋅ ത𝒓 ≠ 𝐶 ⋅ ത𝒓
𝐷 has a nonzero entry.
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
Theorem
If 𝐴 ⋅ 𝐵 ≠ 𝐶, Basic-Frievalds accepts with probability ≤ 1/2.
Principle of Deferred Decisions
Idea: It does not matter in which order 𝑟𝑘 are chosen!
• First choose 𝑟1, … , 𝑟6, 𝑟8, … , 𝑟𝑛. Then 𝑟7• Before 𝑟7 is chosen, the RHS of our equation is determined:
𝑟7 = −𝑑3,1 ⋅ 𝑟1 +⋯+ 𝑑3,6 ⋅ 𝑟6 + 𝑑3,8 ⋅ 𝑟8 +⋯+ 𝑑3,𝑛 ⋅ 𝑟𝑛
𝑑3,7
• Now, there is at most one choice of 𝑟7 that will satisfy it.
• Since there are two choices for 𝑟7, the equation holds w.p. ≤1
2
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
Analysis of Basic Freivalds: Formal Justifications
• Break Ω into smaller events 𝐸𝑥1,…,𝑥6,𝑥8,…,𝑥𝑛 corresponding to
𝑟1, … , 𝑟6, 𝑟8, … , 𝑟𝑛 being assigned 𝑥1, … , 𝑥6, 𝑥8, … , 𝑥𝑛 ∈ {0,1}
Pr 𝐴𝐵ത𝒓 = 𝐶ത𝒓 =
𝑥∈ 0,1 𝑛−1
Pr 𝐴𝐵ത𝒓 = 𝐶ത𝒓 ∩ 𝐸𝑥
≤
𝑥∈ 0,1 𝑛−1
Pr 𝑟7 satisfies the equality ∩ 𝐸𝑥
=
𝑥∈ 0,1 𝑛−1
Pr 𝑟7 satisfies the equality |𝐸𝑥 ⋅ Pr[𝐸𝑥]
≤
𝑥∈ 0,1 𝑛−1
1
2⋅ Pr 𝐸𝑥 ≤
1
2
𝑥∈ 0,1 𝑛−1
Pr 𝐸𝑥 =1
2
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
How does our confidence increase with the number of trials?
• C = event that identity is correct
• A = event that test accepts
Our analysis of Basic Frievalds:
• Pr[A| ҧ𝐶] ≤ 1/2
• 1-sided error: Pr[A|C]=1
Assumption (initial belief or ``prior’’): Pr 𝐶 = 1/2
By Bayes’ Law
Pr 𝐶 𝐴 =Pr 𝐴 𝐶 ⋅ Pr 𝐶
Pr 𝐴 𝐶 ⋅ Pr 𝐶 + Pr 𝐴 ҧ𝐶 ⋅ Pr ҧ𝐶
≥1 ⋅
12
1 ⋅12+12⋅12
=2
3
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
Bayesian Approach to Amplification
How does our confidence increase with the number of trials?
• C = event that identity is correct
• A = event that test accepts
Our analysis of Basic Frievalds:
• Pr[A| ҧ𝐶] ≤ 1/2
• 1-sided error: Pr[A|C]=1
Assumption (initial belief or ``prior’’): Pr 𝐶 = 𝟐/𝟑
By Bayes’ Law
Pr 𝐶 𝐴 =Pr 𝐴 𝐶 ⋅ Pr 𝐶
Pr 𝐴 𝐶 ⋅ Pr 𝐶 + Pr 𝐴 ҧ𝐶 ⋅ Pr ҧ𝐶
≥1 ⋅
𝟐𝟑
1 ⋅𝟐𝟑+12⋅𝟏𝟑
=𝟒
𝟓
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
Bayesian Approach to Amplification
How does our confidence increase with the number of trials?
• C = event that identity is correct
• A = event that test accepts
Our analysis of Basic Frievalds:
• Pr[A| ҧ𝐶] ≤ 1/2
• 1-sided error: Pr[A|C]=1
Assumption (initial belief or ``prior’’): Pr 𝐶 = 𝟐𝒊/(𝟐𝒊 + 𝟏)
By Bayes’ Law
Pr 𝐶 𝐴 =Pr 𝐴 𝐶 ⋅ Pr 𝐶
Pr 𝐴 𝐶 ⋅ Pr 𝐶 + Pr 𝐴 ҧ𝐶 ⋅ Pr ҧ𝐶
≥1 ⋅
𝟐𝒊
𝟐𝒊 + 𝟏
1 ⋅𝟐𝒊
𝟐𝒊 + 𝟏+12⋅
𝟏𝟐𝒊 + 𝟏
=𝟐𝒊+𝟏
𝟐𝒊+𝟏 + 𝟏
9/14/2021Sofya Raskhodnikova; Randomness in Computing
Bayesian Approach to Amplification
Given: undirected graph 𝐺 = (𝑉, 𝐸)
Goal: Find the min cut in 𝐺 (a cut with the smallest cutset).
Applications: Network reliability, network design, clustering
Exercise: How many distinct cuts are there in a graph 𝐺 with 𝑛 nodes?
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
§1.5 (MU) Randomized Min Cut
A global cut of 𝐺 is a partition of 𝑉 into non-empty, disjoint sets S, T.
The cutset of the cut is the set of edges that connect the parts:𝑢, 𝑣 𝑢 ∈ 𝑆, 𝑣 ∈ 𝑇}
S T
Given: undirected graph 𝐺 = (𝑉, 𝐸) with 𝑛 nodes and 𝑚 edges.
Goal: Find the min cut in 𝐺.
Algorithms for Min Cut:
• Deterministic [Stoer-Wagner `97] 𝑂(𝑚𝑛 + 𝑛2 log 𝑛) time
• Randomized [Karger `93] 𝑂(𝑛2𝑚 log 𝑛) time
but there are improvements
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
Min Cut Algorithms
Idea: Repeatedly pick a random edge and put its endpoints on the
same side of the cut.
Basic operation: Edge contraction of an edge 𝒖, 𝒗
• Merge 𝑢 and 𝑣 into one node
• Eliminate all edges connecting 𝑢 and 𝑣
• Keep all other edges, including parallel edges (but no self-loops)
9/14/2021 Sofya Raskhodnikova; Randomness in Computing
§1.5 (MU) Karger’s Min Cut Algorithm
𝒖𝒗
Claim
A cutset of the contracted graph is also a cutset of the original graph.
Probability Amplification: Repeat 𝑟 = 𝑛 𝑛 − 1 ln 𝑛 times and
return the smallest cut found.
Running time of Basic Karger: Best known implementation: O 𝑚
• Easy: 𝑂(𝑚) per contraction, so 𝑂(𝑚𝑛)
• View as Kruskal’s MST algorithm in 𝐺 with 𝑤 𝑒𝑖 = 𝜋(𝑖) run until
two components are left: 𝑂(𝑚 log 𝑛)9/14/2021 Sofya Raskhodnikova; Randomness in Computing
§1.5 (MU) Karger’s Min Cut Algorithm
1. While 𝑉 > 22. choose 𝑒 ∈ 𝐸 uniformly at random3. 𝐺 ← graph obtained by contracting 𝑒 in 𝐺4. Return the only cut in 𝐺.
(input: undirected graph 𝐺 = (𝑉, 𝐸)Algorithm Basic Karger
Theorem
Basic-Karger returns a min cut with probability ≥2
𝑛(𝑛−1).