# Randomness Extractors: Motivation, Applications and Constructions

date post

30-Dec-2015Category

## Documents

view

30download

0

Embed Size (px)

description

### Transcript of Randomness Extractors: Motivation, Applications and Constructions

Randomness Extractors: Motivation, Applications and ConstructionsRonen ShaltielUniversity of Haifa

Outline of talkExtractors as graphs with expansion propertiesExtractors as functions which extract randomnessApplicationsExplicit Constructions

- Extractor graphs: Definition [NZ]An extractor is an (unbalanced) bipartite graph M
Extractor graphs: expansion properties(K,)-Extractor:set X of size K the dist.E(X,U) -close to uniform.

=>expansion property:set X of size K,|)x)| (1-)M.

Distribution versus Set size

XN{0,1}nM{0,1}mK(X)(1-)M*A distribution P is -close to uniform if ||P-U||1 2 => P supports 1- elements.xIdentify X with the uniform distribution on X

Extractors and Expander graphs

XN{0,1}nM{0,1}m(X)(1-)MExtractorN{0,1}nD=2d edges(1+)-ExpanderK(1+)KKN{0,1}n

Extractors and Expander graphs

Requires degree log NAllows constant degree XN{0,1}nM{0,1}m(X)(1-)MExtractorN{0,1}n(1+)-Expander(1+)KN{0,1}n

Balanced graph

Unbalanced graphAbsolute expansion:

K -> (1+)K Relative expansion:

K -> (1-)M

K/N -> (1-)Expands sets smaller than threshold KExpands sets larger than threshold KKK

Outline of talkExtractors as graphs with expansion propertiesExtractors as functions which extract randomnessApplicationsExplicit Constructions

The initial motivation: running probabilistic algorithms with real-life sourcesWe have access to distributions in nature:Electric noiseKey strokes of userTiming of past eventsThese distributions are somewhat random but not truly random.Paradigm: [SV,V,VV,CG,V,CW,Z]. Randomness ExtractorsAssumption for this talk: Somewhat random = uniform over subset of size K.

Successful Paradigm in CS: Probabilistic Algorithms.Probabilistic Algorithms/Protocols: Use an additional input stream of independent coin tosses.Helpful in solving computational problems.Where can we get random bits? random coins Probabilistic algorithminput output Somewhat random

Extractors as functions that use few bits to extract randomnessWe allow an extractor to also receive an additional input of (very few) random bits.Extractors use few random bits to extract many random bits from arbitrary distributions which contain sufficient randomness.Parameters: (function view)Source length: n (= log N)Seed length: d ~ O(log n)Entropy threshold: k ~ n/100Output length: m ~ kRequired error: ~ 1/100 source distribution X RandomnessDefinition: A (K,)-extractor is a function E(x,y) s.t. For every set. X of size K, E(X,U) is -close* to uniform. Lower bounds [NZ,RT]: seed length (in bits) log nProbabilistic method [S,RT]: Exists optimal extractor which matches lower bound and extracts all the k=log K random bits in the source distribution.Explicit constructions: E(x,y) can be computed in poly-time.

Simulating probabilistic algorithms using weak random sourcesGoal: Run prob algorithm using a somewhat random distribution.Where can we get a seed?Idea: Go over all seeds.Given a source element x.y compute zy= E(x,y)Compute Alg(input,zy)Answer majority vote.

Seed=O(logn) => poly-time

Explicit constructions.

Probabilistic algorithminput output random coins Somewhat random

Outline of talkExtractors as graphs with expansion propertiesExtractors as functions which extract randomnessApplicationsExplicit Constructions

ApplicationsSimulating probabilistic algorithms using weak sources of randomness [vN,SV,V,VV,CG,V,CW,Z].Constructing Graphs (Expanders, Super-concentrators) [WZ].Oblivious sampling [S,Z].Constructions of various pseudorandom generators [NZ,RR,STV,GW,MV].Distributed algorithms [WZ,Z,RZ].Cryptography [CDHK,L,V,DS,MST].Hardness of approximations [Z,U,MU].Error correcting codes [TZ].

Expanders that beat the eigenvalue bound [WZ]Goal: Construct low deg expanders with huge expansion.Line up two low degree extractors.set X of size K ,|)x)| (1-)M > M/2.sets X,X of size KX and X have common neighbour.Contract middle layer.Low degree (ND2/K) bipartite graph in which every set of size K sees N-K vertices.Better constructions for large K [CRVW].

N{0,1}nN{0,1}nXX

Randomness efficient (oblivious) sampling using expandersRandom walk variables v1..vD behave like i.i.d: A of size MHitting property: Pr[i : viA] = 2-(D).Chernoff style property: Pr[#i : viA far from exp.] 2-(D).# of random bits used for walk: m+O(D)=m+O(log(1/))# of random bits for i.i.d. mD=m O(log(1/)) v1v2v3vDM{0,1}mRandom walk on constant degree expander

Randomness efficient (oblivious) sampling using extractors [S]Given parameters m,:Use E with K=M=2m, N=M/ and small D. Choose random x: m+log(1/) random bits.Set vi=E(x,i)Ext property Hitting propertyA of size MCall x bad if E(x) inside A.# of bad xs < KPr[x is bad] < K/N =

D edgesxN{0,1}nM{0,1}mbad xs(1-)MA

Every (oblivious) sampling scheme yields an extractorAn (oblivious) sampling scheme uses a random n bit string x to generated D random variables with Chrnoff style property. Thm: [Z] The derived graph is an extractor.

Extractors oblvs Sampling

D=2d edgesxN{0,1}nM{0,1}m

Constructions

Extractors from error correcting codesCan construct extractors from error-correcting code [ILL,SZ,T].Short seed. Extract one additional bitExtractors that extract one additional bit List-decodable error-correcting codes Extractors that extract many bits codes with strong list-recovering properties [TZ].

List-decodable error-correcting codes [S]20% errorsList decoding49% errors EC(x) is 20%-decodable if for every w there is a unique x s.t. EC(x) differs from w in 20% of positions. EC(x) is (49%,t)-list-decodable if for every w there are at most t xs s.t. EC(x) differs from w in 49% of positions. There are explicit constructions of such codes.

Extractors from list-decodable error-correcting codes [ILL,T]Thm: If EC(x) is (-,K)-list-decodable then E(x,y)=(y,EC(x)y) is a (K,2)-extractor.Note: E outputs its seed y. Such an extractor is called strong.E outputs only one additional output bit EC(x)yThere are constructions of list-decodable error correcting codes with |y|=O(log n). Strong extractors with one additional bit List-decodable error correcting codes.Strong extractors with many additional bits translate into very strong error correcting codes [TZ].

Extractors from list-decodable error-correcting codes: proofThm: If EC(x) is (-,K)-list-decodable then E(x,y)=(y,EC(x)y) is a (K,2)-extractor.Proof: by contradiction. Let X be a distribution/set of size K s.t. E(X,Y)=(Y,EC(X)Y) is far from uniform.Observation: Y and EC(X)Y are both uniform.They are correlated. Exists P s.t. P(Y)=EC(X)Y with prob > +2.

Extractors from list-decodable error-correcting codes: proof IIThm: If EC(x) is (-,K)-list-decodable then E(x,y)=(y,EC(x)y) is a (K,2)-extractor.Exists P s.t.PrX,Y[P(Y)=EC(X)Y] > +2.By a Markov argument: For K xs in XPrY[P(Y)=EC(x)Y] > +.Think of P as a string Py=P(y).We have that P and EC(x) differ in - coordinates.Story so far: If E is bad then there is a string P s.t. for K xs P and EC(x) differ in few coordinates.

Extractors from list-decodable error-correcting codes: proof IIIThm: If EC(x) is (-,K)-list-decodable then E(x,y)=(y,EC(x)y) is a (K,2)-extractor.Story so far: If E is bad then there is a string P s.t. for K xs P and EC(x) differ in - coordinates.

List decoding49% errorsBy list-decoding properties of the code: # of such xs < K.Contradiction!

RoadmapCan construct extractors from error-correcting code.Short seed. Output = Seed + 1.Next: How to extract more bits. General paradigm: Once you construct one extractor you can try to boost its quality.

Extracting more bits [WZ] Starting point: An extractor E that extracts only few bits.Idea: (X|E(X,Y)) contains randomness. We can apply E to extract randomness from (X|E(X,Y)).Need a fresh seed.E(X;(Y,Y))=E(X,Y),E(X,Y)Extract more randomness.Use larger seed.

Y X Extractor

Y Z Z Z X New Extractor

Y Y

Trevisans extractor: reducing the seed lengthIdea: Use few random bits to generate (correlated) seeds Y1,Y2,Y3Walk on expander?Extractor?Works but gives small savings.Trevisan: use Nisan-Wigderson pseudorandom generator (based on combinatorial designs).[TZS,SU]: Use Y,Y+1,Y+2,... (based on the [STV] algorithm for list-decoding Reed-Muller code).

X Extractor

Y1 Y2 Y

The extractor designer tool kitMany ways to compose extractors with themselves and related objects.Arguments use entropy manipulations depend on function view of extractors. Impact on other graph construction problems:Expander graphs (zig-zag product) [RVW,CRVW].Ramsey graphs that beat the Frankl-Wilson construction [BKSSW,BRSW].

Entropy manipulations: composing two extractors [Z,NZ]

Y X2 SmallExtractor

Z X1 LargeExtractor

Observation: Can compose a small ext. and a large ext. and obtain ext. which inherits small seed and large output.Paradigm: If given only one source try to convert it into two sources that are sufficiently independent.

Two independent sources

Summary: Extractors are

XM{0,1}mK=2k(X)(1-)M source distribution X RandomnessFunctionsGraphs

ConclusionUnifying role of extractors: Expanders, Oblivious samplers, Error correcting codes, Pseudorandom generators, hash functionsOpen problems:More applications/connections.The quest for explicitly constructing the optimal extractor. (Current record [LRVW]).Direct and simple constructions.Things I didnt talk about:Seedless extractors for special families of sources.

Thats it

Extractor graphs

xN{0,1}nM{0,1}mD=2d edgesxD=2d edges

Extractor graphs: expansion

XN{0,1}nM{0,1}mK=2k(X)(1-)M

Issues in a formal definition: 2. One extractor for all sourcesGoal: Design one extractor function E(x) that works on all sufficiently high entropy distributions.Problem: Impossible to extract even 1 bit from distributions with n-1 bits of entropy.Have to settle for less!

source distribution X Randomness{0,1}nx:E(x)=0x:E(x)=1Distribution X with entropy n-1 on which E(X) is fixed

Definition of extractors [NZ]We allow an extractor to also receive an additional seed of (very few) random bits.Extractors use few random bits to extract many random bits from arbitrary distributions with sufficiently high entropy.Parameters:Source length: nSeed length: d ~ O(log n)Entropy threshold: k ~ n/100Output length: m ~ kRequired error: ~ 1/100 source distribution X RandomnessDefinition: A (k,)-extractor is a function E(x,y) s.t. For every distribution X with min-entropy k, E(X,Y) is -close* to uniform.Lower bounds [NZ,RT]: seed length log n + 2log(1/)Probabilistic method [S,RT]: Exists optimal extractor which matches lower bound and extracts k+d-2log(1/) bits.

*A distribution P is -close to uniform if ||P-U||1 2 => P supports 1- elements.

- Extractor graphs: Definition [NZ]An extractor is an (unbalanced) bipartite graph M
Issues in a formal definition: 1. Notion of entropyThe source distribution X must contain randomnessNecessary condition for extracting k bits: x Pr[X=x]2-kDfn: X has min-entropy k if x Pr[X=x]2-kExample: flat distributions: X is uniformly distributed on a subset of size 2k.Every X with min-entropy k is a convex combination of flat distributions.

source distribution X Randomness2k=|S|{0,1}n

Noisy channels and error corrections errorsGoal: Transmit messages using a noisy channelGuarantee: x differs from x in at most (say) 20% positions.Coding Theory: Encode x prior to transmission.