Randomized AlgorithmsCS648

Lecture 4• Linearity of Expectation with applications(Most important tool for analyzing randomized algorithms)

1

RECAP FROM THE LAST LECTURE

2

Random variable

Definition: A random variable defined over a probability space (Ω,P) is a mapping Ω R.

Examples:o The number of HEADS when a coin is tossed 5 times.o The sum of numbers seen when a dice is thrown 3 times.o The number of comparisons during Randomized Quick Sort on an array of

size n.

Notations for random variables : • X, Y, U, …(capital letters)• X() denotes the value of X on elementary event .

3

Expected Value of a random variable(average value)

Definition: Expected value of a random variable X defined over a probability space (Ω,P) is

E[X] =

4

Ω

X= a X= b

X= c

E[X] =

Examples

Random experiment 1: A fair coin is tossed n times Random Variable X: The number of HEADS E[X] = = =

Random Experiment 2: balls into bins Random Variable X: The number of empty bins E[X] =

5

Can we solve these problems ?

Random Experiment 1 balls into bins Random Variable X: The number of empty bins

E[X]= ??

Random Experiment 2 Randomized Quick sort on elements Random Variable Y: The number of comparisons

E[Y]= ??

6

Balls into Bins(number of empty bins)

Question : X is random variable denoting the number of empty bins. E[X]= ??

Attempt 1: (based on definition of expectation) E[X] = = =

7

1 2 3 … … n

1 2 3 4 5 … m-1 m

A subset of bins

This is a right but useless answer !

Randomized Quick Sort(number of comparisons)

Question : Y is random variable denoting the number of comparisons. E[Y]= ??

Attempt 1: (based on definition of expectation) E[Y] =

8

We can not proceed from this point …

A recursion tree associated with Randomized Quick Sort

9

1 2 3 4 5 … m-1 m

1 2 3 … … n

Balls into Bins(number of empty bins)

Randomized Quick Sort(number of comparisons)

Balls into Bins(number of empty bins)

Question: Let be a random variable defined as follows. =

What is E[] ?Answer : E[] = + = =

10

1 2 3 … … n

1 2 3 4 5 … m-1 m

Balls into Bins(any relation between and ’s ?)

Consider any elementary event.

11

1 2 3 4 5

1 2 3 4 5 6

0 1 0 1 0

An elementary event

Sum of Random Variables

Definition: Let be random variables defined over a probability space (Ω,P) such that

for each ϵ Ω Then is said to be the sum of random variables and .A compact notation :

Definition: Let and be random variables defined over a probability space (Ω,P) such that

for each ϵ Ω Then is said to be the sum of random variables .A compact notation :

12

𝑼=𝑽 +𝑾

𝑼=𝑽 𝟏+𝑽 𝟐+…+𝑽 𝒏

Randomized Quick Sort(number of comparisons)

Question : Let , for any , be a random variable defined as follows. =

What is E[] ?Answer : E[] = + = =

13

Elements of A arranged in Increasing order of values

𝑒𝑖

Randomized Quick Sort(any relation between and ’s ?)

Consider any elementary event.

Question: What is relation between and and ?Answer:

14

1 0 … 0 1 1 … 0

… …

Hence

Any elementary event

What have we learnt till now?

Balls into Bin experimentX: random variable denoting the number of empty binsAim: E[X]= ??

E[] =

Randomized Quick SortY: random variable for the number of comparisonsAim: E[Y]= ??

E[] =

15

Hence

E E

The main question ?

Let be random variables defined over a probability space (Ω,P) such that ,

16

Balls into Bins(number of empty bins)

: random variable denoting the number of empty bins. Using Linearity of Expectation [] for

17

1 2 3 … … n

1 2 3 4 5 … m-1 m

Randomized Quick Sort(number of comparisons): r. v. for the no. of comparisons during Randomized Quick Sort on elements.

Using Linearity of expectation:

18

𝑯 𝑛≤ l𝑜𝑔𝑒𝑛+0.58

Linearity of Expectation

Theorem: • (For sum of 2 random variables)If are random variables defined over a probability space (Ω,P) such that , then

• (For sum of more than 2 random variables) If , then

19

Where to use Linearity of expectation ?

Whenever we need to find E[U] but none of the following work• E[] = • E[] =

In such a situation,

Try to express as , such that it is “easy” to calculate . Then calculate using

20

Think over the following questions?

• Let be random variables defined over a probability space (Ω,P) such that , for some real no. , then

Answer: yes (prove it as homework)

• Why does linearity of expectation holds always ? (even when and are not independent)

Answer: (If you have internalized the proof of linearity of expectation, this question should appear meaningless.)

21

Think over the following questions?

Definition: (Product of random variables)Let be random variables defined over a probability space (Ω,P) such that

for each ϵ Ω Then is said to be the product of random variables and .A compact notation is

• If , then

Answer: No (give a counterexample to establish it.)• If and both and are independent then

Answer: Yes (prove it rigorously and find out the step which requires independence)

22

Independent random variables

In the previous slides, we used the notion of independence of random variable. This notion is identical to the notion of independence of events:

Two random variables are said to be independent if knowing the value of one random variable does not influence the probability distribution of the other.In other words,

for all ϵ and ϵ .

23

Some Practice problemsas homework

• Balls into bin problem:• What is the expected number of bins having exactly 2 balls ?

• We toss a coin n times, what is the expected number of times pattern HHT appear ?

• A stick has n joints. The stick is dropped on floor and in this process each joint may break with probability p independent of others. As a result the stick will be break into many substicks.– What is the expected number of substicks of length 3 ?– What is the expected number of all the substicks ?

24

PROBLEMS OF THE NEXT LECTURE

25

Fingerprinting Techniques

Problem 1:Given three ⨯ matrices , , and , determine if .

Best deterministic algorithm: • ;• Verify if ? Time complexity:

Randomized Monte Carlo algorithm: Time complexity: Error probability: for any .

26

Fingerprinting Techniques

Problem 2:Given two large files A and B of bits located at two computers which are connected by a network. We want to determine if A is identical to B. The aim is to transmit least no. of bits to achieve it.

Randomized Monte Carlo algorithm: Bits transmitted : Error probability: for any .

27