Scalar quantization with Renyi entropy constraintWolfgang Kreitmeier and Tamas Linder

Abstract—We consider optimal scalar quantization with rthpower distortion and constrained Renyi entropy of order α. Forsources with absolutely continuous distributions the high rateasymptotics of the quantizer distortion has long been known forα = 0 (fixed-rate quantization) and α = 1 (entropy-constrainedquantization). For a large class of absolutely continuous sourcedistributions we determine the sharp asymptotics of the optimalquantization distortion for Renyi entropy constraints of orderα ∈ [−∞, 0) ∪ (0, 1). The proof of achievability is based oncompanding quantization and is thus constructive.

I. INTRODUCTION

With the exception of a few very special cases the exactanalysis of the performance of optimal quantizers is a notori-ously hard problem. The asymptotic theory of quantization fa-cilitates such analyses by assuming that the quantizer operatesat asymptotically high rates. The seminal work by Zador [17]determined the asymptotic behavior of the minimum quantizerdistortion under a constraint on either the log-cardinalityof the quantizer codebook (fixed-rate quantization) or theShannon entropy of the quantizer output (entropy-constrainedquantization). (See the review article by Gray and Neuhoff [9]for a historical overview and related results.) Zador’s resultswere later clarified and generalized by Bucklew and Wise [3]and Graf and Luschgy [6] for the fixed-rate case, and by Grayet al. [8] for the entropy-constrained case.

Recently, approaches that incorporate both the fixed andentropy-constrained cases have been suggested. In [7] a La-grangian formulation is developed which puts simultaneousconstraints on entropy and codebook size, including fixed-rate and entropy-constrained quantization as special cases.Another approach that has been suggested in [7] and furtherdeveloped in [11], [12] uses the Renyi entropy of order α ofthe quantizer output as (generalized) rate. One obtains fixed-rate quantization for α = 0, while α = 1 yields the usual(Shannon) entropy-constrained quantization approach.

The choice of Renyi entropy as the quantizer’s rate can befurther motivated from a purely mathematical viewpoint sinceRenyi’s entropy is a canonical extension of Shannon-entropy,satisfying fewer of the entropy axioms [15]. From an opera-tional point of view, the use of Renyi entropy as quantizer rateis supported by Campbell’s work [4] who considered losslesscodes with an exponentially weighted average codeword length

W. Kreitmeier is with the Department of Informatics and Mathematics,University of Passau, Passau, Germany (email: wolfgang.kreitmeier@uni-passau.de). T. Linder is with the Department of Mathematics and Statis-tics, Queen’s University, Kingston, ON, Canada K7L 3N6 (email: lin-der@mast.queensu.ca).

This research was supported in part by the German Research Foundation(DFG) and the Natural Sciences and Engineering Research Council (NSERC)of Canada.

and showed that Renyi’s entropy plays an analogous role toShannon entropy in this more general setting. An overview ofrelated results can be found in [1].

The only available general result on quantization withconstrained Renyi entropy appears to be [11] where thesharp asymptotic behavior of the rth power distortion ofoptimal d-dimensional vector quantizers has been derived forα ∈ [1 + r/d,∞]. The proof shows that for these α valuesthe optimal quantization error is asymptotically determined bythe distortion of a ball with appropriate radius around the mostlikely values of the source distribution, a fact that remarkablysimplifies the derivation and makes the case α ≥ 1 + r/dquite unique. In the classical (α = 0 and α = 1) settings,the contributions by the codecells of an optimal quantizer tothe overall distortion are asymptotically of the same order.Bounds on the optimal performance in [11] suggest a similarsituation for α < 1 + r/d, making the problem significantlymore challenging than the case α ≥ 1 + r/d.

In this paper, at the price of restricting the treatment tothe scalar (d = 1) case, we are able to determine theasymptotics of the optimal quantization error under a Renyientropy constraint of order α ∈ [−∞, 0) ∪ (0, 1) for a fairlylarge class of source densities. The achievability part of theproof (i.e., providing a sharp upper bound on the asymptoticperformance) is constructive via companding quantization. Inparticular, we determine the optimal point density function foreach α ∈ [−∞, 0) ∪ (0, 1) and provide rigorous performanceguarantees for the associated companding quantizers. We notethat in proving the matching lower bounds, one cannot simplyapply the techniques established for α = 0 or α = 1. Inour case the distortion and Renyi entropy of a quantizer mustbe simultaneously controlled, a difficulty not encountered infixed-rate quantization. On the other hand, the Lagrangianformulation that facilitated the corrected proof of Zador’sentropy-constrained quantization result in [8] cannot be usedsince it relies on the special functional form of the Shannonentropy.

II. PRELIMINARIES AND NOTATION

Let α ∈ [−∞,∞] and p = (p1, p2, . . .) ∈ [0, 1]N be aprobability vector, i.e.,

∑∞i=1 pi = 1. The Renyi entropy of

order α, Hα(p) ∈ [0,∞], is defined as (see [15] and [10])

Hα(p) =

1

1−α log(∑

i:pi>0 pαi

), α ∈ R \ {1}

−∑∞i=1 pi log pi, α = 1

− log (max{pi : i ∈ N}) , α =∞− log (inf{pi : i ∈ N, pi > 0}) , α = −∞.

2011 IEEE International Symposium on Information Theory Proceedings

978-1-4577-0595-3/11/$26.00 ©2011 IEEE 2587

We use the conventions 0 · log 0 := 0 and 00 := 0. Alllogarithms are to the base e. Note that

H0(p) = log (card{i ∈ N : pi > 0}) ,

where card denotes cardinality. Using l’Hospital’s rule it iseasy to see, that the case α = 1 follows from the case α 6= 1by taking the limit α → 1. Similar limit relations hold forα = ±∞ making the mapping [−∞,∞] 3 α → Hα(p)continuous. We note that the usual definition of Renyi entropyis restricted to nonnegative values of α, but the case α < 0 canbe handled without too much additional technical difficulty,and we believe that this generalization may turn out to haveuseful implications.

Now let X be an real-valued random variable with distribu-tion µ. Let I ⊂ N be a countable set of positive integers,S = {Si : i ∈ I} a measurable partition of R, andC = {ci : i ∈ I} a set of (distinct) points in R. Then (S, C)defines a scalar quantizer q : R→ C such that

q(x) = ci if and only if x ∈ Si.

We call C the codebook and its elements ci the codepoints.Each Si ∈ S is called codecell. Fix r ≥ 1. Given a quantizerq we measure the approximation error between X and and itsquantized version q(X) using the rth power distortion

Dµ(q) = E|X − q(X)|r =

∫|x− q(x)|r dµ(x).

The Renyi α-entropy of q with respect to µ is defined as

Hαµ (q) = Hα(µ(Si1), µ(Si2), . . .)

where {Si1 , Si2 , . . .} is an arbitrary enumeration of the code-book. For any R ≥ 0 the optimal quantization distortion of µunder Renyi α-entropy bound R is

Dαµ(R) = inf{Dµ(q) : q ∈ Q, Hα

µ (q) ≤ R}

where Q denotes the set of all scalar quantizers.For ∆ > 0 let q∆ denote the infinite-level uniform quantizer

defined by q∆(x) = (i + 1/2)∆ if and only if x ∈ [i∆, (i +1)∆), i ∈ Z.

We denote by λ the one-dimensional Lebesgue measure. Forany probability distribution on R we let supp(µ) denote thesupport of µ, i.e., the smallest closed set whose complementhas µ measure zero. For a measurable real function f on Rand measurable nonempty set A ⊂ R, ess infA f = sup{b :λ({x ∈ A : f(x) < b}) = 0} denotes that the essentialinfimum of f on A. Similarly, ess supA f = inf{b : λ({x ∈A : f(x) > b}) = 0} is the essential supremum of f on A.

III. MAIN RESULTS

First we summarize the known results regarding the high-rate asymptotics of the distortion of optimal scalar quantizers.In order to unify the treatment, we reformulate the classical(resolution and entropy) rate constraints in terms of the Renyientropy with appropriate order. Throughout the paper we let

C(r) =1

(1 + r)2r.

Theorem 1 ([17], [3], [6], [8], [11]): Let r ≥ 1 and as-sume that the distribution µ of X is absolutely continuouswith respect λ having density f .(i) If α = 0 and E|X|r+δ <∞ for some δ > 0, then

limR→∞

erRDαµ(R) = C(r)

(∫f1/(1+r) dλ

)1+r

. (1)

(ii) If α = 1,∫f log f dλ exists and is finite, and H1

µ(q∆) isfinite for some ∆ > 0, then

limR→∞

erRD1µ(R) = C(r)e−r

∫f log f dλ.

(iii) If α ∈ [1 + r,∞], E|X|r+δ < ∞ for some δ > 0, andess supR f <∞, then

limR→∞

e(1+r)β(α)RDαµ(R) = C(r) (ess sup Rf)

−r,

where β(α) = (α−1)/α if α ∈ [1+r,∞) and β(α) = 1if α =∞.

Part (i) is originally due to Zador [17] who consideredthe multidimensional case; corrected and generalized proofswere given by Bucklew and Wise [3] and Graf and Luschgy[6]. Part (ii) is also due to Zador [17] with corrections andgeneralizations by Gray et al. [8]. Part (iii) is due to [11]where upper and lower bounds for the case α ∈ (1, 1 + r) arealso given.

Definition 1: A continuous one-dimensional probabilitydensity function f is called weakly unimodal if there exists anl0 > 0 such that f−1([l,∞)) is a compact interval for everyl ∈ (0, l0).

Remark 1: Note that all continuous unimodal densities areweakly unimodal. Thus the class of weakly unimodal densitiesincludes most parametric source density classes commonlyused in modeling information sources such as exponential,Laplacian, Gaussian, and generalized Gaussian densities.

For α ∈ (−∞, r + 1) \ {1} we define

a1 =1− α+ αr

1− α+ r, a2 =

1− α+ r

1− α. (2)

The following is our main result.Theorem 2: Let r > 1 and assume that the distribution

µ of X is absolutely continuous with respect to λ hav-ing density f . Assume that f is bounded and let M =(inf(supp(µ))), sup(supp(µ))). In either of the followingcases:(i) α ∈ (0, 1), E|X|r+δ < ∞ for some δ > 0, and f is

weakly unimodal,(ii) α ∈ (−∞, 0), f is continuous on M , and infM f > 0

we have

limR→∞

erRDαµ(R) = C(r)

(∫M

fa1 dλ

)a2

. (3)

If f is continuous on M and infM f > 0, then

limR→∞

erRD−∞µ (R) = C(r)

(∫M

f1−r dλ

). (4)

2588

Remark 2: (a) Note that if we formally substitute α = 0in (3), it reduces to (1). Since infM f > 0 (and thus M isbounded) the right hand side of (4) is finite. For the samereason, the right hand side of (3) is finite for all α < 0. Forα ∈ [0, 1) the right hand side of (3) can be shown to be finiteby an application of Holder’s inequality.

(b) The weak unimodality and continuity conditions on fare the results of our approximation techniques in provinglower bounds and are probably not necessary. In fact, one canshow that the right hand sides of (3) and (4) still upper boundthe asymptotic performance if these conditions are dropped.

In the next section we provide the main steps of the proofof achievability in Theorem 2 for all α ∈ [−∞, 0)∪ (0, 1 + r)using companding quantization. The matching lower boundsfor α ∈ [−∞, 0) ∪ (0, 1) are proved by considering increas-ingly more general classes of source densities starting with theuniform distribution on the unit interval. The rather technicalproof of the lower bound and the missing details in the proofof the upper bound can be found in [13].

IV. ACHIEVABILITY VIA COMPANDING QUANTIZATION

A. Companding quantizers

Let N ≥ 2 and QN ∈ Q denote the N -level uniform scalarquantizer with step size 1/N for sources supported in the unitinterval [0, 1] defined by QN (x) = 1/2N if x ∈ [0, 1/N ] and

QN (x) =i− 1

N+

1

2Nif x ∈

(i− 1

N,i

N

], i = 2, . . . , N.

The compressor G derived from a probability density g is thecumulative distribution function

G(x) =

∫ x

−∞g(y) dλ(y).

The generalized inverse G of G is defined by

G(y) := max{x : G(x) ≤ y}

for y ∈ (0, 1). Note that if g is positive almost everywherewith respect to λ (a.e. for short), then G is strictly increasingand G is its (ordinary) inverse.

The N -level companding quantizer Qg,N associated with gis defined by

Qg,N (x) = G(QN (G(x))).

In this context g is often called the point density for Qg,N .The following result represents the error asymptotics of the

compander if the number of output levels increases withoutbound. The result originates with Bennett [2] for r = 2 andhas appeared in the literature in several different forms (butoften without precise conditions and a rigorous proof); see [9]for a historical overview. The proposition essentially followsfrom [14] which gives a rigorous proof for the limit (5) underdifferent conditions that include the continuity of g and certaintail conditions, but allow f and g to have unbounded support.

Proposition 1: Let X be a random variable with distribu-tion µ which is absolutely continuous with respect to λ and letf denote its density. Let G be a compressor with point density

g. Assume that the support of µ is included in a compactinterval I such that ess infI g > 0 and g(x) = 0 a.e. on R \ I .Then

limN→∞

NrDµ(Qg,N ) = C(r)

∫I

f

grdλ. (5)

Remark 3: The condition ess infI g > 0 clearly implies∫Ifgr dλ <∞.

B. Renyi entropy asymptotics of companding quantizers

In order to be able to construct asymptotically optimal com-panding quantizers, in addition to controlling the asymptoticdistortion, we also have to control the quantizer’s entropy, atleast for high rates. We will derive a result (Proposition 2)which asymptotically describes the Renyi entropy of the com-panding quantizer as a function of the number of quantizationpoints. Let 1A denote the indicator function of A ⊂ R.

Let µ be absolutely continuous with respect to λ withdensity f and define M = (inf(supp(µ)), sup(supp(µ))). Letα ∈ [−∞,∞] and assume that(i) 1supp(µ)f

α is integrable if α ∈ R \ {1},(ii) ess infM f > 0 if α = −∞,

(iii) f log f is integrable if α = 1,(iv) ess supR f <∞ if α = +∞.Then the Renyi differential entropy of order α of µ is definedby

hα(µ) =

1

1−α log(∫

supp(µ)fα dλ), α ∈ R \ {1}

−∫f log f dλ, α = 1

− log(ess infM f), α = −∞− log(ess supR f), α = +∞.

Remark 4: Just as in the case of Renyi entropy the mapping[−∞,∞] 3 α→ hα(µ) is continuous.

Recall that q∆ denotes the infinite-level uniform quantizerwith step-size ∆ > 0. Recall M from Definition 6 and letA(∆,M) = {a ∈ q∆(R) : q−1

∆ (a) ⊂M} and

q∆,M =∑

a∈A(∆,M)

a · 1q−1∆ (a)(·).

The following result is due to Renyi [16] and Csiszar [5]for α ∈ (0,∞).

Lemma 1: Let µ be absolutely continuous with respect toλ having density f . Let α ∈ [−∞,∞) and assume that theRenyi differential entropy of order α of µ exists. Assume thatHαµ (q∆) <∞ for some ∆ > 0. If α ∈ (−∞,∞), then

lim∆→0

(Hαµ (q∆) + log(∆)

)= hα(µ).

Moreover,

lim∆→0

(H−∞µ (q∆,M ) + log(∆)

)= h−∞(µ).

Let µ and ν be probability measures which are absolutelycontinuous with respect to λ. Denote by f and g the densitiesof µ and ν. Moreover, we assume that µ is absolutely continu-ous with respect to ν. Therefore, we may assume without lossof generality that {g = 0} ⊂ {f = 0}. Setting E = {f > 0}

2589

and M = (inf(supp(µ)), sup(supp(µ))), the relative Renyientropy of order α between the distributions µ and ν is definedas

Dα(µ‖ν) =

1α−1 log

(∫Efαg1−α dλ

), α ∈ R \ {1}∫

Ef log f

g dλ, α = 1

log(ess supEfg ), α =∞

log(ess infMfg ), α = −∞.

(6)

(For D−∞(µ‖ν) to be well defined, we need the conditioness infM f > 0.)

The following result determines the asymptotics of theRenyi entropy of a companding quantizer.

Proposition 2: Let α ∈ [−∞,∞) and assume thatDα(µ‖ν) <∞. Then

limN→∞

(Hαµ (Qg,N )− logN

)= −Dα(µ‖ν).

Remark 5: For the sake of the distortion analysis we pre-viously specified that g has bounded support, but in thisproposition the only condition is that D(µ‖ν) be finite.

Proof: We proceed in two steps.1) We show that hα(µ ◦ G−1) = −Dα(µ‖ν) for every

α ∈ [−∞,∞).Consider the image measure µ ◦ G−1 (the distribution of

G(X) if X ∼ µ). It can be shown that under our conditionsµ ◦ G−1 is absolutely continuous with respect to λ and itsdensity is given a.e. on (0, 1) by

fG(y) = f(G(y))G′(y) =f(G(y))

g(G(y))1{g>0}(G(y)).

(This is trivial if g everywhere positive.) Let α ∈ (−∞,∞) \{1}. Definition 6 and a change of variables give

hα(µ ◦G−1)

=1

1− αlog

∫(fG)α dλ

=1

1− αlog

∫(f(G(y))G′(y))α dλ(y)

=1

1− αlog

(∫E

f(G(y))αg(G(y))1−αG′(y) dλ(y)

)=

1

1− αlog

(∫E

f(x)αg(x)1−α dλ(x)

)= −Dα(µ‖ν).

One can deduce the assertion hα(µ ◦G−1) = −Dα(µ‖ν) forα ∈ {−∞, 1} in a very similar manner.

2) Since G is increasing and continuous, G is strictlyincreasing on (0, 1). Let QN be the restriction of q1/N to(0, 1). Then

Hαµ (Qg,N ) = Hα

µ◦G−1(QN ) (7)

for all α ∈ [−∞,∞). Since µ ◦ G−1((0, 1)) = 1, weobtain Hα

µ◦G−1(QN ) = Hαµ◦G−1(q1/N ). In view of (7) we

deduce Hαµ (Qg,N ) = Hα

µ◦G−1(q1/N ). From step 1 and by theassumption we know that hα(µ ◦ G−1) is finite. Since q1/N

has no more than N cells with nonzero µ ◦ G−1−measure,the entropy Hα

µ◦G−1(q1/N ) is also always finite. Lemma 1and step 1 imply

limN→∞

(Hαµ (Qg,N )− logN

)= lim

N→∞

(Hαµ◦G−1(q1/N )− logN

)= hα(µ ◦G−1) = −Dα(µ‖ν).

C. Optimal point densities

Combining the previous results we can find a point densityand corresponding companding quantizers which provide anoptimized (asymptotic) upper bound for the optimal quanti-zation error. Matching lower bounds (proved in [13]) showthat these quantizers are asymptotically optimal. Recall thedefinitions of a1 and a2 in (2).

Corollary 1: Let r ≥ 1 and α ∈ [−∞, 1 + r). Assume thatµ is supported on a compact interval I and has density f suchthat ess infI f > 0. Moreover, assume that fa1 is integrable ifα ∈ (1, 1 + r) and f log f is integrable if α = 1. Let

f∗ =

(∫If1/a2 dλ)−1f1/a2 , α ∈ (−∞, 1 + r) \ {1}

(λ(I))−11I , α = 1

f, α = −∞.(8)

Then,

limN→∞

erHαµ (Qf∗,N )Dµ(Qf∗,N )

=

C(r)

∫If1−r dλ, α = −∞

C(r)e−r∫f log f dλ, α = 1

C(r)(∫Ifa1 dλ)a2 , otherwise.

Proof: It is not hard to show using Holder’s inequalitythat fa11I is integrable for every α ∈ (−∞, 1 + r) \ {1}.Moreover f1−r is integrable. Clearly, f1/a21I is integrablefor every α ∈ (−∞, 1+r). Applying Propositions 1 and 2 weobtain

limN→∞

erHαµ (Qf∗,N )Dµ(Qf∗,N )

= limN→∞

e−rDα(µ‖f∗λ)NrDµ(Qf∗,N )

= e−rDα(µ‖f∗λ)C(r)

∫f

(f∗)rdλ.

Now (8) and (6) yield the assertion.Remark 6: For α ∈ [−∞, 1] the point density g =

f∗ in the corollary minimizes the asymptotic performancelimN→∞

erHαµ (Qg,N )Dµ(Qg,N ). For α = 0 and α = 1 this

optimal choice of g has long been known (see, e.g., [9]). Forα ∈ (−∞, 1], by Propositions 1 and 2 the asymptotic perfor-mance is proportional to

(∫fαg1−α dλ

) r1−α

∫fg−r dλ, and

Holder’s inequality can be used to show that this functional isminimized by g = f∗. The case α = −∞ follows by lettingα→ −∞.

2590

D. Achievability in Theorem 2

For α ∈ (−∞, 1 + r)\{1} and f meeting the conditions ofCorollary 1, we obtain a sequence of companding quantizers(qN ) =

(Qf∗,N

)such that

limN→∞

erHαµ (qN )Dµ(qN ) = C(r)

(∫M

fa1 dλ

)a2

. (9)

Noting that by Proposition 2, limN Hαµ (qN ) = ∞ and

limN

(Hαµ (qN ) − Hα

µ (qN−1))

= 0, one can easily show that(9) implies

lim supR→∞

erRDαµ(R) ≤ C(r)

(∫M

fa1 dλ

)a2

for α ∈ (−∞, 1 + r) \ {1}. An identical argument shows thatunder the conditions of Corollary 1 (and hence of Theorem 2)

limR→∞

erRD−∞µ (R) ≤ C(r)

(∫M

f1−r dλ

).

The upper bound for α ∈ (0, 1 + r) \ {1} can be extendedto densities of unbounded support, and in particular to allweakly unimodal densities (Definition 1) meeting conditionsof Theorem 2, thus proving the achievability part.

V. CONCLUDING REMARKS

We have determined the sharp distortion asymptotics foroptimal scalar quantization with Renyi entropy constraint forvalues α ∈ [−∞, 0)∪(0, 1) of the order parameter. Our results,together with the classical α = 0 and α = 1 cases, and therecent result [11] for α ∈ [r+ 1,∞], leave only open the caseα ∈ (1, 1 + r) for which only non-matching upper and lowerbounds are known to date (cf. [11]).

Apart from the question of high-rate asymptotics, it remainsopen if optimal quantizers exist for all α ∈ [−∞, 1 + r]. Thenon-existence of optimal quantizers in case of α > 1 + r hasalready been shown in [12]. Also of special interest is thequestion whether companding quantizers with point densityf∗ are still asymptotically optimal for source densities withunbounded support. The definition of f∗ needs the integrabilityof f1/a2 in order to guarantee a finite number of quantizationpoints for the (asymptotically optimal) companding quantizer.On the other hand, the right hand side of (3) is defined onlywhen fa1 is integrable. It remains an open problem if (3) stillholds for some α ∈ (−∞, 1 + r)\{1} and distributions wherefa1 is integrable but f1/a2 is not. Such an example wouldshow that the companding approach is not always applicableto generate asymptotically optimal quantizers, but the knownasymptotics (3) are still in force.

Now consider the d-dimensional case and rth power distor-tion based on some norm on Rd. For α ∈ [−∞, 1) and underappropriate conditions we conjecture that

lim supR→∞

erdRDα

µ(R) = C(r, d)

(∫Rdfa1 dλ

)a2

wherea1 =

1− α+ α rd1− α+ r

d

, a2 =1− α+ r

d

1− α

and C(r, d) is a positive constant that depends only on r, d,and the underlying norm.

This conjecture is based on arguments for the scalar casethat can easily be generalized to higher dimensions. However,some important steps in our proofs are definitely restricted toone dimension. One of the key problems concerns the proofof the result for a finite mixture of uniform distributions ondisjoint intervals [13]. In higher dimensions one has to controlthe contribution to distortion and entropy of cells straddlingthe common boundary of at least two touching cubes in thesupport of µ. The “firewall” construction used in case of α = 0(see [6, p.87]) does not seem to work in the general case. Forα 6= 0 it seems to be very hard to control the entropy of thequantizer when adding or changing codecells and codepointsin a certain region. In order to progress in this direction,one would need more refined knowledge about the codecellgeometry of (asymptotically) optimal quantizers. In particular,it appears that generalization to higher dimensions wouldnecessitate the development of isodiametric inequalities forthe (bounded) codecells of asymptotically optimal quantizers.

REFERENCES

[1] M. B. Baer, Coding for General Penalties. Ph.D. Thesis, StanfordUniversity, June 2003.

[2] W. R. Bennett, “Spectra of quantized signals,” Bell. Syst. Tech. J., vol. 27,pp. 446–472, 1948.

[3] J. A. Bucklew and G. L. Wise, “Multidimensional asymptotic quantiza-tion theory with rth power distortion measures,” IEEE Trans. Inform.Theory, vol. 28, pp. 239–247, Mar. 1982.

[4] L. L. Campbell, “A coding theorem and Renyi’s entropy,” Informationand Control, vol. 8, pp. 423–429, 1965.

[5] I. Csiszar, “Generalized entropy and quantization problems,” in Trans. ofthe Sixth Prague Conference on Information Theory, Statistical DecisionFunctions, Random Processes, (Prague), pp. 29–35, Akademia, 1973.

[6] S. Graf and H. Luschgy, Foundations of Quantization for ProbabilityDistributions. Berlin, Heidelberg: Springer Verlag, 2000.

[7] R. M. Gray, T. Linder, and J. T. Gill, “Lagrangian vector quantizationwith combined entropy and codebook size constraints,” IEEE Trans.Inform. Theory, vol. 54, pp. 2220–2242, May 2008.

[8] R. M. Gray, T. Linder, and J. Li, “A Lagrangian formulation of Zador’sentropy-constrained quantization theorem,” IEEE Trans. Inform. Theory,vol. 48, pp. 695–707, Mar. 2002.

[9] R. M. Gray and D. L. Neuhoff, “Quantization,” IEEE Trans. Inform.Theory, vol. 44, pp. 2325–2383, Oct. 1998.

[10] P. Harremoes, “Joint range of Renyi entropies,” Kybernetika, vol. 45,pp. 901–911, 2009.

[11] W. Kreitmeier, “Error bounds for high-resolution quantization withRenyi-α-entropy constraints,” Acta Math. Hungar., vol. 127, pp. 34–51,2010.

[12] W. Kreitmeier, “Optimal quantization for the one-dimensional uniformdistribution with Renyi-α-entropy constraints,” Kybernetika, vol. 46,pp. 96–113, 2010.

[13] W. Kreitmeier and T. Linder, “High-resolution scalar quantization withRenyi entropy constraint,” submitted to IEEE Trans. Inform. Theory,2010. http://arxiv.org/pdf/1008.1744

[14] T. Linder, “On asymptotically optimal companding quantization,” Probl.Control Inf. Theory, vol. 20, no. 6, pp. 383–393, 1991.

[15] A. Renyi, “On measures of entropy and information.” in Proc. 4thBerkeley Symp. Math. Stat. Probab., vol. 1, pp. 547–561, 1960.

[16] A. Renyi, “Some fundamental problems of information theory (inHungarian),” Magyar Tud. Akad. Mat. Fiz. Oszt. Kozl, vol. 10, pp. 251–282, 1960.

[17] P. Zador, Development and evaluation of procedures for quantizingmultivariate distributions. PhD thesis, Stanford Univ., 1963. Univ.Microfilm no. 64-9855.

2591