CS537 Randomness in Computing - cs-people.bu.edu

9/14/2021

Randomness in Computing

LECTURE 3Last time• Probability amplification

• Verifying matrix multiplication

Today• Finish verifying matrix

multiplication

• Probability amplification:

Bayesian approach

• Randomized Min-Cut

Sofya Raskhodnikova;Randomness in Computing

Useful conditional probability facts

Let A be an event and let 𝐸1, … , 𝐸𝑛 be mutually disjoint

events whose union is Ω.

• Total probability law.

Pr 𝐴 =

𝑖∈ 𝑛

Pr[𝐴 ∩ 𝐸𝑖] =

𝑖∈ 𝑛

Pr[𝐴 ∣ 𝐸𝑖] ⋅ Pr 𝐸𝑖 .

• Bayes’ Law. Assume Pr 𝐴 ≠ 0. Then, for all 𝑗 ∈ 𝑛 ,

Pr 𝐸𝑗 𝐴 =Pr[𝐸𝑗 ∩ 𝐴]

Pr[𝐴]

=Pr 𝐴|𝐸𝑗 ⋅ Pr 𝐸𝑗

σ𝑖∈ 𝑛 Pr[𝐴 ∣ 𝐸𝑖] ⋅ Pr[𝐸𝑖].

9/14/2021 Sofya Raskhodnikova; Randomness in Computing

Tennis match

You will play a tennis match against opponent X or Y.

If X is chosen, you win with probability 0.7.

If Y is chosen, you win with probability 0.3.

Your opponent is chosen by flipping a coin with bias 0.6

in favor of X.

What is your probability of winning?


Review question: tennis match

You will play a tennis match against opponent X or Y.

If X is chosen, you win with probability 0.7.

If Y is chosen, you win with probability 0.3.

Your opponent is chosen by flipping a coin with bias 0.6 in favor of X.

What is your probability of winning?

A. < 0.3

B. In the interval [0.3,0.4).

C. In the interval [0.4,0.55).

D. In the interval [0.55,0.7).

E. ≥0.7


Review question: balls and bins

We have two bins with balls.

• Bin 1 contains 3 black balls and 2 white balls.

• Bin 2 contains 1 black ball and 1 white ball.

We pick a bin uniformly at random. Then we pick a ball

uniformly at random from that bin.

What is the probability that we picked bin 1, given that we

picked a white ball?


Task: Given three 𝑛 × 𝑛 matrices 𝐴, 𝐵, 𝐶, verify if 𝐴 ⋅ 𝐵 = 𝐶.

Idea: Pick a random vector ത𝒓 and check if 𝐴 ⋅ 𝐵 ⋅ ത𝒓 = 𝐶 ⋅ ത𝒓.

Running time: Three matrix-vector multiplications: O 𝑛2 time.

Correctness: If 𝐴 ⋅ 𝐵 = 𝐶, the algorithm always accepts.

Probability Amplification: With 𝑘 repetitions, error probability ≤ 2−𝑘


O(𝒏𝟐) multiplications for each matrix-vector product

§1.3 (MU) Verifying Matrix Multiplication

1. Choose a random 𝑛-bit vector ҧ𝑟 by making each bit 𝑟𝑖independently 0 or 1 with probability 1/2 each.

2. 𝐀𝐜𝐜𝐞𝐩𝐭 if 𝐴 ⋅ (𝐵 ⋅ ത𝒓) = 𝐶 ⋅ ത𝒓; o. w. 𝐫𝐞𝐣𝐞𝐜𝐭.

(input: 𝑛 × 𝑛 matrices A, B, C)Algorithm Basic Frievalds

Theorem

If 𝐴 ⋅ 𝐵 ≠ 𝐶, Basic-Frievalds accepts with probability ≤ 1/2.

Analysis of Error Probability

Proof: Suppose 𝐴 ⋅ 𝐵 ≠ 𝐶 and let 𝐷 = 𝐴𝐵 − 𝐶

𝐴 ⋅ 𝐵 ⋅ ത𝒓 ≠ 𝐶 ⋅ ത𝒓

𝐷 has a nonzero entry.


Theorem

If 𝐴 ⋅ 𝐵 ≠ 𝐶, Basic-Frievalds accepts with probability ≤ 1/2.

Principle of Deferred Decisions

Idea: It does not matter in which order 𝑟𝑘 are chosen!

• First choose 𝑟1, … , 𝑟6, 𝑟8, … , 𝑟𝑛. Then 𝑟7• Before 𝑟7 is chosen, the RHS of our equation is determined:

𝑟7 = −𝑑3,1 ⋅ 𝑟1 +⋯+ 𝑑3,6 ⋅ 𝑟6 + 𝑑3,8 ⋅ 𝑟8 +⋯+ 𝑑3,𝑛 ⋅ 𝑟𝑛

𝑑3,7

• Now, there is at most one choice of 𝑟7 that will satisfy it.

• Since there are two choices for 𝑟7, the equation holds w.p. ≤1

2


Analysis of Basic Freivalds: Formal Justifications

• Break Ω into smaller events 𝐸𝑥1,…,𝑥6,𝑥8,…,𝑥𝑛 corresponding to

𝑟1, … , 𝑟6, 𝑟8, … , 𝑟𝑛 being assigned 𝑥1, … , 𝑥6, 𝑥8, … , 𝑥𝑛 ∈ {0,1}

Pr 𝐴𝐵ത𝒓 = 𝐶ത𝒓 =

𝑥∈ 0,1 𝑛−1

Pr 𝐴𝐵ത𝒓 = 𝐶ത𝒓 ∩ 𝐸𝑥

≤

𝑥∈ 0,1 𝑛−1

Pr 𝑟7 satisfies the equality ∩ 𝐸𝑥

=

𝑥∈ 0,1 𝑛−1

Pr 𝑟7 satisfies the equality |𝐸𝑥 ⋅ Pr[𝐸𝑥]

≤

𝑥∈ 0,1 𝑛−1

1

2⋅ Pr 𝐸𝑥 ≤

1

2

𝑥∈ 0,1 𝑛−1

Pr 𝐸𝑥 =1

2


How does our confidence increase with the number of trials?

• C = event that identity is correct

• A = event that test accepts

Our analysis of Basic Frievalds:

• Pr[A| ҧ𝐶] ≤ 1/2

• 1-sided error: Pr[A|C]=1

Assumption (initial belief or ``prior’’): Pr 𝐶 = 1/2

By Bayes’ Law

Pr 𝐶 𝐴 =Pr 𝐴 𝐶 ⋅ Pr 𝐶

Pr 𝐴 𝐶 ⋅ Pr 𝐶 + Pr 𝐴 ҧ𝐶 ⋅ Pr ҧ𝐶

≥1 ⋅

12

1 ⋅12+12⋅12

=2

3


Bayesian Approach to Amplification





• Pr[A| ҧ𝐶] ≤ 1/2


Assumption (initial belief or ``prior’’): Pr 𝐶 = 𝟐/𝟑

By Bayes’ Law



≥1 ⋅

𝟐𝟑

1 ⋅𝟐𝟑+12⋅𝟏𝟑

=𝟒

𝟓







• Pr[A| ҧ𝐶] ≤ 1/2


Assumption (initial belief or ``prior’’): Pr 𝐶 = 𝟐𝒊/(𝟐𝒊 + 𝟏)

By Bayes’ Law



≥1 ⋅

𝟐𝒊

𝟐𝒊 + 𝟏

1 ⋅𝟐𝒊

𝟐𝒊 + 𝟏+12⋅

𝟏𝟐𝒊 + 𝟏

=𝟐𝒊+𝟏

𝟐𝒊+𝟏 + 𝟏

9/14/2021Sofya Raskhodnikova; Randomness in Computing


Given: undirected graph 𝐺 = (𝑉, 𝐸)

Goal: Find the min cut in 𝐺 (a cut with the smallest cutset).

Applications: Network reliability, network design, clustering

Exercise: How many distinct cuts are there in a graph 𝐺 with 𝑛 nodes?


§1.5 (MU) Randomized Min Cut

A global cut of 𝐺 is a partition of 𝑉 into non-empty, disjoint sets S, T.

The cutset of the cut is the set of edges that connect the parts:𝑢, 𝑣 𝑢 ∈ 𝑆, 𝑣 ∈ 𝑇}

S T

Given: undirected graph 𝐺 = (𝑉, 𝐸) with 𝑛 nodes and 𝑚 edges.

Goal: Find the min cut in 𝐺.

Algorithms for Min Cut:

• Deterministic [Stoer-Wagner `97] 𝑂(𝑚𝑛 + 𝑛2 log 𝑛) time

• Randomized [Karger `93] 𝑂(𝑛2𝑚 log 𝑛) time

but there are improvements


Min Cut Algorithms

Idea: Repeatedly pick a random edge and put its endpoints on the

same side of the cut.

Basic operation: Edge contraction of an edge 𝒖, 𝒗

• Merge 𝑢 and 𝑣 into one node

• Eliminate all edges connecting 𝑢 and 𝑣

• Keep all other edges, including parallel edges (but no self-loops)


§1.5 (MU) Karger’s Min Cut Algorithm

𝒖𝒗

Claim

A cutset of the contracted graph is also a cutset of the original graph.

Probability Amplification: Repeat 𝑟 = 𝑛 𝑛 − 1 ln 𝑛 times and

return the smallest cut found.

Running time of Basic Karger: Best known implementation: O 𝑚

• Easy: 𝑂(𝑚) per contraction, so 𝑂(𝑚𝑛)

• View as Kruskal’s MST algorithm in 𝐺 with 𝑤 𝑒𝑖 = 𝜋(𝑖) run until

two components are left: 𝑂(𝑚 log 𝑛)9/14/2021 Sofya Raskhodnikova; Randomness in Computing

§1.5 (MU) Karger’s Min Cut Algorithm

1. While 𝑉 > 22. choose 𝑒 ∈ 𝐸 uniformly at random3. 𝐺 ← graph obtained by contracting 𝑒 in 𝐺4. Return the only cut in 𝐺.

(input: undirected graph 𝐺 = (𝑉, 𝐸)Algorithm Basic Karger

Theorem

Basic-Karger returns a min cut with probability ≥2

𝑛(𝑛−1).

CS537 Randomness in Computing - cs-people.bu.edu

Documents

Transcript of CS537 Randomness in Computing - cs-people.bu.edu