Near-Optimal Erasure List- Decodable...

Near-Optimal Erasure List-Decodable Codes

Dean Doron (Tel-Aviv University)Workshop on Coding and Information Theory, CMSA

Joint work withAvi Ben-AroyaAmnon Ta-Shma

• A code C : {0,1}n → {0,1}n̄ is (1-ε,L) erasure list-decodable, if

• Given a codeword

Erasure List-Decodable Codes

0 1 1 0 0 0 1 0 1 1

• Given a codeword where all but ε fraction of its coordinates were (adversarially) erased,

0 1 1 0 0 0 1 0 1 1

? 1 ? ? ? 0 1 ? ? 1

1 1 1 0 1 0 1 0 0 1

0 1 1 0 0 0 1 0 1 1

0 1 0 0 0 0 1 1 1 1

⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮

0 1 1 1 1 0 1 1 0 1

n̄there exists a list of size at most L that contains the original codeword.

? 1 ? ? ? 0 1 ? ? 1

1 1 1 0 1 0 1 0 0 1

0 1 1 0 0 0 1 0 1 1

0 1 0 0 0 0 1 1 1 1

⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮

0 1 1 1 1 0 1 1 0 1

n̄there exists a list of size at most L that contains the original codeword.

• We work over the binary field.

• The two key parameters:

• The code’s rate R = n̄/n.

• The list-size L.

Bounds for the binary case

Bounds on the rate

• The code’s rate — R.

Bounds on the rate

• The code’s rate — R.

• Can be as large as ε [Guruswami 03] (recall that ε is the fraction of coordinates we keep).

• Compare with the errors model, where ~ ε2 is necessary for a code of distance ½ - ε.

Bounds on the rate

Bounds on the list-size

• The list-size — L.

• Can be as tiny as log(1/ε) [Guruswami 03].

• Compare with the errors model, where poly(1/ε) is necessary.

• Can be as tiny as log(1/ε) [Guruswami 03].

• Compare with the errors model, where poly(1/ε) is necessary.

• For linear codes, the list-size is at least 1/ε [Guruswami 03].

Previous Results

• Any binary code of distance ½ - ε can handle 1-2ε erasures with polynomial list size.

Previous Results

• Drawback: We have upper bounds. Cannot break the rate-ε2 bound that way [MRRW,…].

Previous Results

• Drawback: We have upper bounds. Cannot break the rate-ε2 bound that way [MRRW,…].

• Few explicit constructions (especially for binary codes).

Previous Results

Rate R List-Size L

Non-explicit ε log(1/ε)

Optimal codes with distance ½-ε ε2 poly(1/ε)

Guruswami 01 (constant ε) ε2/log(1/ε) 1/ε

Guruswami-Indyk 03 (constant ε) ε2/log(1/ε) log(1/ε)

• In the small ε regime, we break the ε2 barrier, and manage to do this with nearly optimal parameters.

Our Result

• In the small ε regime, we break the ε2 barrier, and manage to do this with nearly optimal parameters.

Our Result

For every small enough ε and a constant γ there exists an explicit binary erasure list-decodable code with rate

R = ε1+ γ

and list-size

L = poly(log(1/ε))

Previous Results

Rate R List-Size L

Non-explicit ε log(1/ε)

Optimal codes with distance ½-ε ε2 poly(1/ε)

Guruswami 01 (constant ε) ε2/log(1/ε) 1/ε

Guruswami-Indyk 03 (constant ε) ε2/log(1/ε) log(1/ε)

Our Result(small ε) ε1+γ poly(log(1/ε))

Different Perspectives

Erasure List-

Decodable Codes

Ramsey Graphs

Strong 1-bit

Dispersers

[Guruswami 04]

[Gradwohl, Kindler, Reingold, Ta-Shma 05]

Erasure List-

Decodable Codes

Ramsey Graphs

Strong 1-bit

Dispersers

[Guruswami 04]

Erasure List-

Decodable Codes

Ramsey Graphs

Strong 1-bit

Dispersers

[Guruswami 04]

Erasure List-

Decodable Codes

Ramsey Graphs

Strong 1-bit

Dispersers

[Guruswami 04]

Erasure List-

Decodable Codes

Ramsey Graphs

Strong 1-bit

Dispersers

[Guruswami 04]

1-bit Strong Dispersers

Disp: {0,1}n × {0,1}d →{0,1}

{0,1}n

Disp(x,y)

• For every (n,k)-source X, or a set X⊆{0,1}n of size at least K=2k,

Disp: {0,1}n × {0,1}d →{0,1}

{0,1}n

X Disp(x,y)

Disp: {0,1}n × {0,1}d →{0,1}

{0,1}n

For every x, Pr[X = x] ≤ 2-k.

Disp(x,y)

Disp: {0,1}n × {0,1}d →{0,1}

{0,1}n

X Disp(x,y)

• For all but ε-fraction of the seeds y∈{0,1}d,

Disp(X,y) = {0,1}

Disp: {0,1}n × {0,1}d →{0,1}

{0,1}n

X Disp(x,y)

Disp(X,y) = {0,1}

Disp: {0,1}n × {0,1}d →{0,1}

{0,1}n

XDisp(x1,ybad)=1

Disp(xK,ybad)=1

Disp(X,y) = {0,1}

Disp: {0,1}n × {0,1}d →{0,1}

{0,1}n

Disp(x1,y)=0

Disp(xK,y)=1

Bounds for1-bit Strong Dispersers

• Dispersers are an important tool in derandomization.

• The disperser’s seed length — d.

• The disperser’s entropy requirement — k (or, how small can X be).

Bounds on the seed length

• Can be as small as log(n)+log(1/ε) [Radhakrishnan,Ta-Shma 00, Meka,Reingold,Zhou 14].

Bounds on the entropy

• The disperser’s entropy requirement — k (or, how small can X be).

• Can be as tiny as loglog(1/ε). That is, an optimal disperser works for extremely small sets — of size O(log(1/ε)) [Radhakrishnan,Ta-Shma 00, Meka,Reingold,Zhou 14].

Bounds on the entropy

Comparison with extractors

• Dispersers’ parameters outperform seeded extractors, in which we don’t only require that the output covers the entire codomain, but also require doing it uniformly.

• The seed length must be at least log(n)+2log(1/ε) [Radhakrishnan,Ta-Shma 00] (compare with log(n)+log(1/ε) for dispersers).

• The entropy requirement must be at least 2log(1/ε) [Radhakrishnan,Ta-Shma 00] (compare with loglog(1/ε) for dispersers).

Comparison with extractors

Erasure List-

Decodable Codes

Ramsey Graphs

Strong 1-bit

Dispersers

[Guruswami 04]

Codes from Dispersers

Codes from DispersersIf Disp: {0,1}n × {0,1}d →{0,1} has error ε and entropy requirement k, we can explicitly construct a code C: {0,1}n → {0,1}D for D=2d that is erasure list-decodable from 1-2ε erasures with list-size L = 2k and rate R=n/D.

Taking an optimal disperser with d=log(n)+log(1/ε) and k=loglog(1/ε), we get an optimal erasure list-decodable code with rate R=n/D=ε and list-size L=2k=log(1/ε).

Constructing Dispersers

• How can we construct small-error 1-bit strong dispersers?

• We need all but ε-fraction of the seeds to cover {0,1}.

• We don’t want to do it uniformly (this would give an extractor).

• Say, typically, Disp(X,y) is ε0-close to uniform for ε0 ≫ ε (say, ε0 = ¼).

Erasure List-

Decodable Codes

Ramsey Graphs

Strong 1-bit

Dispersers

[Guruswami 04]

2-Source Extractors

Two-Source Extractors

{0,1}n

{0,1}d

edge ↔ 2Ext = 1

2Ext: {0,1}n × {0,1}d →{0,1}

{0,1}n

• For every (n,k)-source X,

• For every (d,d’)-source Y, independent of X,

• 2Ext(X,Y) is ε0-close to uniform (think ε0 of as constant). {0,1}d

edge ↔ 2Ext = 1

2Ext: {0,1}n × {0,1}d →{0,1}

{0,1}n

|Y|=2d’

edge ↔ 2Ext = 1

2Ext: {0,1}n × {0,1}d →{0,1}

{0,1}n

• The error of the corresponding strong disperser (the faction of bad seeds) is given by ε = 2d’-d.

|Y|=2d’

edge ↔ 2Ext = 1

2Ext: {0,1}n × {0,1}d →{0,1}

Implications of Two-Source Extractors

• By constructing two-source extractors for an (n,k)-source and a (d,d’=γd) source, we get:

• A strong disperser for entropy k and seed-length d=(1/(1-γ))log(1/ε).

• An erasure list-decodable code of list-size L=2k and rate R=ε1/(1-γ).

So we must achieve γ<½ to break the ε2 bound!

Our Goal

• We are given an (n,k)-source X and an independent (d,d’=γd)-source Y. We want to extract one unbiased bit.

Our Goal

• For γ>½, the problem was already solved.

Our Goal

• Raz [Raz 05] gave a construction that supports, say, γ=0.6. In such a case, Raz(X,Y) is very close to uniform.

Our Goal

• In the code sense, it gives a list-size of poly(log(1/ε)).

Our Goal

• In the code sense, it gives a list-size of poly(log(1/ε)).

• But, γ>½ so we cannot break the rate ε2 bound.

Our Goal

Condensing Y

• We want to:

• Condense Y. From a (d,γd)-source to an (m,0.6m)-source.

Condensing Y

Condenser

• We want to:

• Condense Y. From a (d,γd)-source to an (m,0.6m)-source.

• To apply Raz’s extractor.

Condensing Y

Condenser

Seeded Condensers

• We need Cond: {0,1}d × {0,1}s →{0,1}m that increases the density of the source to rate 0.6.

Seeded Condensers

• To apply Cond, we need a uniform seed Us. How do we get it?

Seeded Condensers

• Option 1 — Use another independent source.

Seeded Condensers

• Option 1 — Use another independent source.

• Option 2 — Enumerate over all possible seeds. This gives us a table Y’.

Seeded Condensers

Condensing Y

Condenser

Cond(Y,s1)Cond(Y,s2)

Cond(Y,si)

Cond(Y,sA)

Seeded Condensers

• The number of rows of Y’ is the number of possible seeds of the condenser.

Seeded Condensers

• Composing existing constructions, we have an explicit condenser having a constant number of seeds! This fact is crucial for us [Dvir,Kopparty,Saraf,Sudan 13, Zuckerman 06].

Seeded Condensers

• More specifically, the seed length of the condenser is s=O(log(1/εcond)) and we apply it with a constant εcond.

Seeded Condensers

• More specifically, the seed length of the condenser is s=O(log(1/εcond)) and we apply it with a constant εcond.

• Y’ is still only a function of Y.

Seeded Condensers

Condensing Y

Condenser

Cond(Y,si)

Applying Raz’s extractor

• Most of the rows of Y’ are good — they have large density.

• We can now apply Raz’s extractor.

• X as the source.

• Every row of Y’ as the seed.

Condenser

Cond(Y,si)

Extractor

Raz(X,Y’i)

Y’’

• We now have that most of the rows of Y’’ are uniform and there is a small set of bad rows.

• That’s not enough for us — we need to output a single good value.

• The solution (in the spirit of [Chattopadhyay,Zuckerman 16])

• Apply a resilient function on the rows of Y’’.

Resilient Functions

• What do we need from a resilient function?

Resilient Functions

• Nearly balanced.

• Its output cannot be heavily influenced by any small set of bad rows.

Resilient Functions

• Nearly balanced.

• Its output cannot be heavily influenced by any small set of bad rows.

• For example, the Majority function is a resilient function.

Resilient Functions

• That approach is problematic…

Resilient Functions

• That approach is problematic…

• Even if there is only one bad row, the Majority function can still fail!

~U~U~U~U~U~U

Y’’1 = ¬Y’’2

Ext(X,Y’i)Y’’3 = ¬Y’’4

Resilient Functions

• How can we fix this?

Resilient Functions

• The Majority function will work if the good rows are not only uniform, but uniform and 10-wise independent [Viola 14].

Resilient Functions

• So, before applying a resilient function, we need to (partially) break the correlations in Y’’.

Resilient Functions

• So, before applying a resilient function, we need to (partially) break the correlations in Y’’.

• For that, we use a correlation breaker.

Resilient Functions

Correlation Breakers

• We think of a 10-wise correlation breaker as an extractor, with the added property that its output on 10 distinct good seeds is (very close to) 10-wise independent.

• Cohen’s correlation breaker [Cohen 16] allows for almost optimal parameters when the overall number of rows is constant.

• A correlation breaker requires a source and an independent seed.

• We use every row of Y’’, and for the seed we reuse our Y.

• The source and the seed are not independent… Y’’ is a function of both X and Y.

• A main part of the analysis — show that we’re still OK.

• This gives a large set of good rows, every 10 of them are jointly close to uniform.

• The correlation breaker outputs one bit.

• Every 10 good rows are jointly very close to uniform.

Applying a correlation breaker

Ext(X,Y’i)

Y’’ = f(X,Y)

Correlation

CB(Y,Y’’i)

Y’’’

Breaker

• We are now ready to apply a resilient function!

Applying a resilient function

Ext(X,Y’i)

Y’’ = f(X,Y)

Correlation

CB(Y,Y’’i)

Y’’’

Breaker

• We are now ready to apply a resilient function!

Applying a resilient function

Ext(X,Y’i)

Y’’ = f(X,Y)

Correlation

CB(Y,Y’’i)

Y’’’

Breaker

Resilient

FunctionU1

Overview

Condenser

Cond(Y,si)

Extractor

Raz(X,Y’i)

Y’’

Correlation

CB(Y,Y’’i)Breaker

Y’’’

ResilientFunction

Constantbias ε0

Wrapping-Up

• Given an (n,k)-source X and a (d,d’=γd) source Y, we extracted one bit with an arbitrarily small constant error.

Wrapping-Up

• k can be extremely small — Ω(loglog(n)). Corresponds to a small list-size of our erasure list-decodable code.

Wrapping-Up

• d can be small — Ω(log(n)), and the entropy-rate γ is an arbitrary constant. Corresponds to the large rate of our code.

Wrapping-Up

• d can be small — Ω(log(n)), and the entropy-rate γ is an arbitrary constant. Corresponds to the large rate of our code.

• In this, unbalanced, regime of parameters, our two-source construction is close to optimal.

Wrapping-Up

Our Result

For every small enough ε and a constant γ there exists an explicit binary erasure list-decodable code with rate

R = ε1+ γ

and list-size

L = poly(log(1/ε))

• We skipped over many delicate details which are crucial for our construction to work. Examples are:

• Reusing the source Y.

• Errors cannot be aggregated, but must be handled with care (e.g., separating errors that depend on the number of rows and errors that depend on the row’s length).

• Handling a set of bad rows that depend on the samples and not only on the distributions.

• The resilient function cannot be Majority and we need to achieve t-wise independence for a large t.

Wrapping-Up

• Efficient list-decoding?

• Achieving linear erasure list-decodable codes that beat the rate ε2 bound (they would necessarily have a larger list-size).

• Getting even closer to optimal codes…

• Obtaining strong dispersers outputting many bits.

Open Questions

Thanks for listening.

Near-Optimal Erasure List- Decodable...

Documents

Transcript of Near-Optimal Erasure List- Decodable...

Tables of Optimal Allocations for Selected Settings ...faculty.smu.edu/ngh/optall_tables.pdf · Tables of Optimal Allocations for Selected Settings: Optimal Experimental Plan for

Precision Measurement of Electroproduction of near Threshold Precision Measurement of Electroproduction of ˇ0 near Threshold ABSTRACT Electromagnetic production of neutral pions near

Optimal Stopping for Dynamic Convex Risk Measures

Optimal Control Theory

Optimal Curve Straightening is 9R-Complete

A Near-Optimal Algorithm for Estimating the Entropy of a ...ac/Pubs/talg-entropy-opt.pdf · Amit Chakrabarti was supported by NSF Grant EIA-98-02068, NSF CAREER award and by Dartmouth

INTERFACE FOLIATION NEAR MINIMAL SUBMANIFOLDS IN ...

Energy-Aware Wireless Scheduling with Near Optimal Backlog and Convergence Time Tradeoffs Michael J. Neely University of Southern California INFOCOM 2015,

Soft decoding, dual BCH codes, and better list-decodable -biased codes

Near-Optimal Control of Queueing Systems via Approximate ...1.Simulate ˇto estimate its average cost gˇand the long-run frequency with which each state is visited. 2.Based on Step

Optimal 2-pebbling number

Optimal ADMM Parameters for Distributed Optimal Power Flow

GMT’s Near IR Multiple Object Spectrograph - NIRMOS

Quantifying Near-Surface Attenuation via Site-Specific ...

Adaptive Optimal Illumination in Clutter George Papanicolaou

Optimal High-Frequency Market Making

Neutrino interaction study in the K2K near detectorssakemi/neu/slide/hasegawa.pdf · Neutrino interaction study in the K2K near detectors ... ¾Introduction - K2K experiment and Near

Optimal collapse simulator for three-dimensional structures

Asymptotical problems in optimal location

Adaptively Secure Garbling with Near Optimal Online Complexity · · 2018-02-08Adaptively Secure Garbling with Near Optimal Online Complexity Sanjam Garg University of ... (logjCj;