Randomness Extractors: Motivation, Applications and Constructions
Embed Size (px)
description
Transcript of Randomness Extractors: Motivation, Applications and Constructions

Randomness Extractors: Motivation, Applications and ConstructionsRonen ShaltielUniversity of Haifa

Outline of talkExtractors as graphs with expansion propertiesExtractors as functions which extract randomnessApplicationsExplicit Constructions
 Extractor graphs: Definition [NZ]An extractor is an (unbalanced) bipartite graph M

Extractor graphs: expansion properties(K,)Extractor:set X of size K the dist.E(X,U) close to uniform.
=>expansion property:set X of size K,)x) (1)M.
Distribution versus Set size
XN{0,1}nM{0,1}mK(X)(1)M*A distribution P is close to uniform if PU1 2 => P supports 1 elements.xIdentify X with the uniform distribution on X

Extractors and Expander graphs
XN{0,1}nM{0,1}m(X)(1)MExtractorN{0,1}nD=2d edges(1+)ExpanderK(1+)KKN{0,1}n

Extractors and Expander graphs
Requires degree log NAllows constant degree XN{0,1}nM{0,1}m(X)(1)MExtractorN{0,1}n(1+)Expander(1+)KN{0,1}n
Balanced graph
Unbalanced graphAbsolute expansion:
K > (1+)K Relative expansion:
K > (1)M
K/N > (1)Expands sets smaller than threshold KExpands sets larger than threshold KKK

Outline of talkExtractors as graphs with expansion propertiesExtractors as functions which extract randomnessApplicationsExplicit Constructions

The initial motivation: running probabilistic algorithms with reallife sourcesWe have access to distributions in nature:Electric noiseKey strokes of userTiming of past eventsThese distributions are somewhat random but not truly random.Paradigm: [SV,V,VV,CG,V,CW,Z]. Randomness ExtractorsAssumption for this talk: Somewhat random = uniform over subset of size K.
Successful Paradigm in CS: Probabilistic Algorithms.Probabilistic Algorithms/Protocols: Use an additional input stream of independent coin tosses.Helpful in solving computational problems.Where can we get random bits? random coins Probabilistic algorithminput output Somewhat random

Extractors as functions that use few bits to extract randomnessWe allow an extractor to also receive an additional input of (very few) random bits.Extractors use few random bits to extract many random bits from arbitrary distributions which contain sufficient randomness.Parameters: (function view)Source length: n (= log N)Seed length: d ~ O(log n)Entropy threshold: k ~ n/100Output length: m ~ kRequired error: ~ 1/100 source distribution X RandomnessDefinition: A (K,)extractor is a function E(x,y) s.t. For every set. X of size K, E(X,U) is close* to uniform. Lower bounds [NZ,RT]: seed length (in bits) log nProbabilistic method [S,RT]: Exists optimal extractor which matches lower bound and extracts all the k=log K random bits in the source distribution.Explicit constructions: E(x,y) can be computed in polytime.

Simulating probabilistic algorithms using weak random sourcesGoal: Run prob algorithm using a somewhat random distribution.Where can we get a seed?Idea: Go over all seeds.Given a source element x.y compute zy= E(x,y)Compute Alg(input,zy)Answer majority vote.
Seed=O(logn) => polytime
Explicit constructions.
Probabilistic algorithminput output random coins Somewhat random

Outline of talkExtractors as graphs with expansion propertiesExtractors as functions which extract randomnessApplicationsExplicit Constructions

ApplicationsSimulating probabilistic algorithms using weak sources of randomness [vN,SV,V,VV,CG,V,CW,Z].Constructing Graphs (Expanders, Superconcentrators) [WZ].Oblivious sampling [S,Z].Constructions of various pseudorandom generators [NZ,RR,STV,GW,MV].Distributed algorithms [WZ,Z,RZ].Cryptography [CDHK,L,V,DS,MST].Hardness of approximations [Z,U,MU].Error correcting codes [TZ].

Expanders that beat the eigenvalue bound [WZ]Goal: Construct low deg expanders with huge expansion.Line up two low degree extractors.set X of size K ,)x) (1)M > M/2.sets X,X of size KX and X have common neighbour.Contract middle layer.Low degree (ND2/K) bipartite graph in which every set of size K sees NK vertices.Better constructions for large K [CRVW].
N{0,1}nN{0,1}nXX

Randomness efficient (oblivious) sampling using expandersRandom walk variables v1..vD behave like i.i.d: A of size MHitting property: Pr[i : viA] = 2(D).Chernoff style property: Pr[#i : viA far from exp.] 2(D).# of random bits used for walk: m+O(D)=m+O(log(1/))# of random bits for i.i.d. mD=m O(log(1/)) v1v2v3vDM{0,1}mRandom walk on constant degree expander

Randomness efficient (oblivious) sampling using extractors [S]Given parameters m,:Use E with K=M=2m, N=M/ and small D. Choose random x: m+log(1/) random bits.Set vi=E(x,i)Ext property Hitting propertyA of size MCall x bad if E(x) inside A.# of bad xs < KPr[x is bad] < K/N =
D edgesxN{0,1}nM{0,1}mbad xs(1)MA

Every (oblivious) sampling scheme yields an extractorAn (oblivious) sampling scheme uses a random n bit string x to generated D random variables with Chrnoff style property. Thm: [Z] The derived graph is an extractor.
Extractors oblvs Sampling
D=2d edgesxN{0,1}nM{0,1}m

Outline of talkExtractors as graphs with expansion propertiesExtractors as functions which extract randomnessApplicationsExplicit Constructions

Constructions

Extractors from error correcting codesCan construct extractors from errorcorrecting code [ILL,SZ,T].Short seed. Extract one additional bitExtractors that extract one additional bit Listdecodable errorcorrecting codes Extractors that extract many bits codes with strong listrecovering properties [TZ].

Listdecodable errorcorrecting codes [S]20% errorsList decoding49% errors EC(x) is 20%decodable if for every w there is a unique x s.t. EC(x) differs from w in 20% of positions. EC(x) is (49%,t)listdecodable if for every w there are at most t xs s.t. EC(x) differs from w in 49% of positions. There are explicit constructions of such codes.

Extractors from listdecodable errorcorrecting codes [ILL,T]Thm: If EC(x) is (,K)listdecodable then E(x,y)=(y,EC(x)y) is a (K,2)extractor.Note: E outputs its seed y. Such an extractor is called strong.E outputs only one additional output bit EC(x)yThere are constructions of listdecodable error correcting codes with y=O(log n). Strong extractors with one additional bit Listdecodable error correcting codes.Strong extractors with many additional bits translate into very strong error correcting codes [TZ].

Extractors from listdecodable errorcorrecting codes: proofThm: If EC(x) is (,K)listdecodable then E(x,y)=(y,EC(x)y) is a (K,2)extractor.Proof: by contradiction. Let X be a distribution/set of size K s.t. E(X,Y)=(Y,EC(X)Y) is far from uniform.Observation: Y and EC(X)Y are both uniform.They are correlated. Exists P s.t. P(Y)=EC(X)Y with prob > +2.

Extractors from listdecodable errorcorrecting codes: proof IIThm: If EC(x) is (,K)listdecodable then E(x,y)=(y,EC(x)y) is a (K,2)extractor.Exists P s.t.PrX,Y[P(Y)=EC(X)Y] > +2.By a Markov argument: For K xs in XPrY[P(Y)=EC(x)Y] > +.Think of P as a string Py=P(y).We have that P and EC(x) differ in  coordinates.Story so far: If E is bad then there is a string P s.t. for K xs P and EC(x) differ in few coordinates.

Extractors from listdecodable errorcorrecting codes: proof IIIThm: If EC(x) is (,K)listdecodable then E(x,y)=(y,EC(x)y) is a (K,2)extractor.Story so far: If E is bad then there is a string P s.t. for K xs P and EC(x) differ in  coordinates.
List decoding49% errorsBy listdecoding properties of the code: # of such xs < K.Contradiction!

RoadmapCan construct extractors from errorcorrecting code.Short seed. Output = Seed + 1.Next: How to extract more bits. General paradigm: Once you construct one extractor you can try to boost its quality.

Extracting more bits [WZ] Starting point: An extractor E that extracts only few bits.Idea: (XE(X,Y)) contains randomness. We can apply E to extract randomness from (XE(X,Y)).Need a fresh seed.E(X;(Y,Y))=E(X,Y),E(X,Y)Extract more randomness.Use larger seed.
Y X Extractor
Y Z Z Z X New Extractor
Y Y

Trevisans extractor: reducing the seed lengthIdea: Use few random bits to generate (correlated) seeds Y1,Y2,Y3Walk on expander?Extractor?Works but gives small savings.Trevisan: use NisanWigderson pseudorandom generator (based on combinatorial designs).[TZS,SU]: Use Y,Y+1,Y+2,... (based on the [STV] algorithm for listdecoding ReedMuller code).
X Extractor
Y1 Y2 Y

The extractor designer tool kitMany ways to compose extractors with themselves and related objects.Arguments use entropy manipulations depend on function view of extractors. Impact on other graph construction problems:Expander graphs (zigzag product) [RVW,CRVW].Ramsey graphs that beat the FranklWilson construction [BKSSW,BRSW].

Entropy manipulations: composing two extractors [Z,NZ]
Y X2 SmallExtractor
Z X1 LargeExtractor
Observation: Can compose a small ext. and a large ext. and obtain ext. which inherits small seed and large output.Paradigm: If given only one source try to convert it into two sources that are sufficiently independent.
Two independent sources

Summary: Extractors are
XM{0,1}mK=2k(X)(1)M source distribution X RandomnessFunctionsGraphs

ConclusionUnifying role of extractors: Expanders, Oblivious samplers, Error correcting codes, Pseudorandom generators, hash functionsOpen problems:More applications/connections.The quest for explicitly constructing the optimal extractor. (Current record [LRVW]).Direct and simple constructions.Things I didnt talk about:Seedless extractors for special families of sources.

Thats it

Extractor graphs
xN{0,1}nM{0,1}mD=2d edgesxD=2d edges

Extractor graphs: expansion
XN{0,1}nM{0,1}mK=2k(X)(1)M

Issues in a formal definition: 2. One extractor for all sourcesGoal: Design one extractor function E(x) that works on all sufficiently high entropy distributions.Problem: Impossible to extract even 1 bit from distributions with n1 bits of entropy.Have to settle for less!
source distribution X Randomness{0,1}nx:E(x)=0x:E(x)=1Distribution X with entropy n1 on which E(X) is fixed

Definition of extractors [NZ]We allow an extractor to also receive an additional seed of (very few) random bits.Extractors use few random bits to extract many random bits from arbitrary distributions with sufficiently high entropy.Parameters:Source length: nSeed length: d ~ O(log n)Entropy threshold: k ~ n/100Output length: m ~ kRequired error: ~ 1/100 source distribution X RandomnessDefinition: A (k,)extractor is a function E(x,y) s.t. For every distribution X with minentropy k, E(X,Y) is close* to uniform.Lower bounds [NZ,RT]: seed length log n + 2log(1/)Probabilistic method [S,RT]: Exists optimal extractor which matches lower bound and extracts k+d2log(1/) bits.
*A distribution P is close to uniform if PU1 2 => P supports 1 elements.
 Extractor graphs: Definition [NZ]An extractor is an (unbalanced) bipartite graph M

Issues in a formal definition: 1. Notion of entropyThe source distribution X must contain randomnessNecessary condition for extracting k bits: x Pr[X=x]2kDfn: X has minentropy k if x Pr[X=x]2kExample: flat distributions: X is uniformly distributed on a subset of size 2k.Every X with minentropy k is a convex combination of flat distributions.
source distribution X Randomness2k=S{0,1}n

Noisy channels and error corrections errorsGoal: Transmit messages using a noisy channelGuarantee: x differs from x in at most (say) 20% positions.Coding Theory: Encode x prior to transmission.