Spectral deconvolution of unitary invariant modelsjgarzav/Slides_Pierre_Tarrago.pdf · 2020. 10....

Deconvolution problem Proof of the theorem

Spectral deconvolution of unitary invariantmodels

Pierre Tarrago

October 8, 2020

1 / 24


(General) spectral deconvolution problem

Spectral measure of A :

A = A∗ ∈ MN(C) µA =1

N

N∑λ eigvals of A

δλ

Three ingredients :

1. A ∈ MN(C), A = A∗, the unknown ”signal” matrix,2. X = (X1, . . . ,Xr ) ∈ MN(C)r , the unknown, random ”noise”

matrices, and

3. W = f (A,X1,X∗1 , . . . ,Xr ,X

∗r ), the known ”data” matrix, with

f ∈ C〈U,V1,V ∗1 , . . . ,Vr ,V ∗r 〉 self-adjoint polynomial.

Main goal : recover µA from W .

2 / 24


Applications

• Wireless communication :

W = (X1A + X2)∗(X1A + X2),

with X1,X2 Ginibre matrices (A matrix of emission powers, X1matrix of random messages, X2 noise matrix from environment, Wsignal matrix) General linear model (Ryan, Debbah, 2007).

• Cleaning of covariance matrix :

W = XAX ∗

with X Ginibre matrix (Ledoit-Péché (2011), Ledoit-Wolf (2013))

• Generalized/structured covariance matrix :

W = XAX ∗

with X = BG B ≥ 0, G Ginibre matrix (Buns, Allez, Bouchaud,Potter (2016)). For example (B2)ij = exp(|i − j |/τ).

3 / 24


Other approaches:

• Covariance matrices :I El Karoui (2008) : first results on the subject.I Rao (2008) : atomic case.I Bai and al. (2010) : deconvolution using moments.I Ledoit and Wolf (2013) : ”QuEST” algorithm for estimating

covariance matrices, numerical inversion of the Marchenko-Pasturequation.

• Bun, Allez, Bouchaud, Potters (2016) : Rotationally InvariantEstimator of noisy matrix models, using the spectral deconvolutionas an oracle (some spectral deconvolution are achieved using theR/S-transform).

• Hayase (2020) : use of Cauchy distribution.• Mäıda et al. (preprint, 2020) : study of the backward Fokker-Planck

equation.

4 / 24


Spectral deconvolution : main assumptions

1. Simplest algebraic combinations: f = X + Y (additive case) orf = ZYZ∗ (multiplicative case), and we set X = Z∗Z .

2. Control on the spectral distribution of X : E(dWass,1(µX , µ0)2) ≤ CN2for some µ0 ∈M1(R).

3. Independence between the eigenbasis of A and X : X =law

UXU∗ for

U Haar unitary.

Main phenomenon : for N large, µW ' µA � /� µX , where �,� denotesthe free additive/multiplicative convolution of measures.

• as N goes to infinity in law : Voiculescu (1991),• as N goes to infinity, almost surely : Speicher (1993),• concentration results for fixed N : Kargin (2015), Bao, Erdösz and

Schnelli (2017), Meckes and Meckes (2013).

• recently, large deviation principle : Belinschi, Guionnet and Huang(2020).

5 / 24


Free operations : analytic transforms

Suppose that µ3 = µ1 � µ2. How to compute µ3 from µ1 and µ2 ?

Cauchy transform of µ ∈M1(R) : write C± = {z ∈ C,±=z > 0},

Gµ =

{C+ −→ C−z 7→

∫R

1z−t dµ(t)

Stieltjes inversion formula :

dµ(t) = − 1π

limy→0=Gµ(t + iy).

We also define Fµ, hµ : C+ → C+ as

Fµ(z) =1

Gµ(z),

hµ(z) = Fµ(z)− z .

6 / 24


Free operations : the subordination phenomenon

(Belinschi, Bercovici, Biane, Voiculescu)Suppose that µ3 = µ1 � µ2. How to compute µ3 from µ1 and µ2 ?

There exist ω1, ω2 : C+ → C+ such that for z ∈ C+,1. Gµ3(z) = Gµ1(ω1(z)) = Gµ2(ω2(z)) (subordination property)

2. ω1(z) + ω2(z) = z +1

Gµ3 (z)(free additive relation)

Using 1. and 2., one has access to ω1(z) as a fixed pointequation/iteration limit

ω1(z) = Kz(ω1(z)) = limn→∞

K◦nz (w),

for any w ∈ C+, with

Kz(w) = hµ2(hµ1(w) + z) + z .

7 / 24


Free operations : the subordination phenomenon

(Belinschi, Bercovici, Biane, Voiculescu)Suppose that µ3 = µ1 � µ2. How to compute µ3 from µ1 and µ2 ?

(µ1, µ2 ∈M1(R+))

There exist ω1, ω2 : C+ → C+ such that for z ∈ C+,1. zGµ3(z) = ω1(z)Gµ1(ω1(z)) = ω2(z)Gµ2(ω2(z)) (subordination

property for d µ̃(t) = td µ̃(t))

2. ω1(z)ω2(z) =z

1−z−1Fµ(z) (free multiplicative relation)

Using 1. and 2., one has access to ω1(z) as a fixed pointequation/iteration limit

ω1(z) = Hz(ω1(z)) = limn→∞

H◦nz (w),

for any w ∈ C+, with

Hz(w) = −z

hµ2

(−z

hµ1 (w)

) .8 / 24


Free deconvolution : the subordination method


Recall :

1. Gµ3(z) = Gµ1(ω1(z)) = Gµ2(ω2(z))

2. ω1(z) + ω2(z) = z +1

Gµ3 (z)

Suppose that ω2 is invertible on some domain ? ⊂ C+ :1. Gµ3(ω

−12 (z)) = Gµ1(ω1(ω

−12 (z))) = Gµ2(z)

2. ω1(ω−12 (z)) + z = ω

−12 (z) +

1Gµ3 (ω

−12 (z))

= ω−12 (z) +1

Gµ2 (z)

Set ω3(z) = ω−12 (z), ω1′(z) = ω1(ω

−12 (z)) :

1. Gµ3(ω3(z)) = Gµ1(ω1′(z)) = Gµ2(z)

2. ω1′(z) + z = ω3(z) + Fµ2(z)

Domain of validity ?

9 / 24



Set Cσ = {z ∈ C,=z > σ}, and write σ21 = Var(µ1).

Theorem (Arizmendi, Vargas, T.)There exist two analytic functions ω1, ω3 : C2√2σ1 → C

+ such that for allz ∈ C2√2σ1 ,

1. Gµ2(z) = Gµ1(ω1(z)) = Gµ2(ω3(z)) (subordination property).

2. ω1 + z = ω3 + Fµ1(ω1(z)) = ω3 + Fµ3(ω3(z)).

Moreover, ω3(z) is the unique fixed point of the functionKz(w) = z − hµ1(w + Fµ3(w)− z) in C3=(z)/4 and we have

ω3(z) = limK◦nz (w), w ∈ C3=(z)/4.

[Interesting point : except for the first equality in 1., the latter theorem isnot related to µ2, and still holds without ”µ1 � µ2 = µ3”]

10 / 24




Deconvolution method :

1. using the latter theorem, compute ω3 and Gµ2 = Gµ3 ◦ ω3 on thehorizontal line L = {x + 2

√2σ1i , x ∈ R}.

2. Fact : using the Stieltjes inversion formula on L instead of R, weobtain the density f of the classical convolution µ2 ∗ C2√2σ1 , wheredCλ(t) = dCauchyλ(t) = λπ(t2+λ2) .

3. Using classical deconvolution tools, recover µ2 by doing thedeconvolution of f by C2√2σ1 .

Exemple of deconvolution tools (regularization needed !):

• regularized Fourier transform,• Tychonov’s regularization,• Total Variation minimization procedure.

11 / 24


Back to the spectral deconvolution :

Suppose that W = X + A, with µX ' µ0 and µW ' µX � µ1. How tocompute µA from µ0 and µW ?

Deconvolution method : set σ = Var(µ0)1/2.

1. using subordination approach, compute ω3 and G = GµW ◦ ω3 on thehorizontal line L = {x + 2

√2σ, x ∈ R} from µ0 and µW (up to now,

G is just a function).2. Fact : using the Stieltjes inversion formula with G on L instead of R,

we obtain a function Ĉ close to dµA ∗ C2√2σ on R.

3. Using classical deconvolution tools, achieve the deconvolution of Ĉby C2√2σ to obtain an estimator µ̂A of µA.

Main question : how well does µ̂A approximate µA ?

12 / 24


Simulations

Figure: Additive case W = X + A : X ∈ MN(C) Gaussian Wigner matrix,A ∈ MN(C) diagonal with entries iid ∼ N(0, 1), N = 500.Histogram of µW , graph of Ĉ , and comparison between µ̂A and µA.

Figure: Multiplicative case W = XAX ∗ : X ∈ MN(C) Ginibre matrix, A ∈ CWigner with Gaussian entries, N = 500.Histogram of µW , graph of Ĉ , and comparison between µ̂A and µA.

13 / 24


Stability of the free deconvolution : heuristic from thesubordination property

R R0 0

C+C−

14 / 24



R R0 0

Gµ2

C+C−

Gµ2(R)

14 / 24



R R0 0

Gµ2

Gµ3C+

C−

Gµ3(R)

Gµ2(R)

Gµ3(C+) ⊂ Gµ2(C+).

14 / 24


Stability of the free deconvolution : heuristic from thesubordination property.

R R0 0

Gµ2

Gµ3C+

C−

Gµ3(R)

Gµ2(R)

Ĉ

R + i2√

2σ

C2√2σ

14 / 24



R R0 0

Gµ3 ◦ ω3

Gµ2

Gµ3C+

C−

Gµ3(R)

Gµ2(R)

Ĉ

R + i2√

2σ

C2√2σ

Subordination of the deconvolution.

14 / 24



R R0 0

Gµ3 ◦ ω3

Gµ2

Gµ3C+

C−

Gµ3(R)

Gµ2(R)

Ĉ

R + i2√

2σ

Theoretical limit

D

Maximum domain of recovery of Gµ2 .

14 / 24



R R0 0

Gµ3C+

C−

Gµ3(R)

Gµ2(R)

Ĉ

R + i2√

2σ

C2√2σ/D

Extension of Gµ2 from C2√

2Var(µ0)/Dmax to C+.

In the case C2√

2Var(µ0), this equivalent to a classical deconvolution.

14 / 24


What about the classical case ?

Suppose

• µ1, µ2 ∈M1(R),• we know µ1 and µ3 with ‖dµ3, d(µ1 ∗ µ2)‖L2 ≤ �.

How well can we recover µ2 ? Theoretically :

Φµ3 ' Φµ1Φµ2 ,

with Φµ the Fourier transform of µ. BUT, in most cases,

Φµ1(t) −−−→t→∞ 0,

which prevent the naive method µ̂2 ' Φ−1(Φµ3/Φµ1).1. Without regularization, nothing can be done : need for hypothesis

on µ2.

2. Smoother µ1, rougher µ2 ⇒ Faster exploding of Φµ3/Φµ1 at ∞ ⇒Harder regularization.

15 / 24


The classical case :

(Butucea, Fan, Lacour and al.)

µ̂2 ' Φ−1(Φµ3/Φµ1)

16 / 24




µ̂2 ' Φ−1(1[−K ,K ]Φµ3/Φµ1)

16 / 24




µ̂2 ' Φ−1(1[−K ,K ]Φµ3/Φµ1)

Then, suppose there exist bi , γi , si > 0 with

• a ≤ |Φµ1(t)|(1 + t2)γ1/2 exp(b1|t|s1) ≤ A, a,A > 0•∫R |Φµ2(t)|

2(1 + t2)γ2 exp(2b2|t|s2) < L with L 0

s2 = 0 �2γ2

2γ1+2γ2+1 (− log �)−2s2/s1

s2 > 0 (− log �)2γ1+1

b2 � complicated, if s1 = s2 : �b2

b2+b1 (− log(�))ξ

Optimal bounds !

16 / 24


The classical/free Cauchy deconvolution :


If µ1 = Cλ, Φµ1(t) = exp(−λ|t|).Hence,

• a1 = 1,γ1 = 0, b1 = λ, s1 = 1.•∫R |Φµ2(t)|

2(1 + t2)γ2 exp(2b2|t|s2) < L with L 0

s2 = 0 �2γ2

2γ1+2γ2+1 (− log �)−2s2/s1

s2 > 0 (− log �)2γ1+1

b2 � complicated, if s1 = s2 : �b2

b2+b1 (− log(�))ξ

17 / 24


The classical classical/free Cauchy deconvolution :

For us : µ1 = C2√2σ1 , Φµ1(t) = exp(−λ|t|), µ2 = µA and dµ3 = Ĉ .Hence, if

‖Ĉ − d(C2√2σ ∗ µA)‖L2 ≤ �

an appropriate K = K (�) yields that ‖d µ̂A − dµA‖2L2 is bounded by

dµA is Ck C (k)(− log �)−2

dµA can be analytically �b2

b2+2√

2σ (− log(�))ξextended to R×]− b2, b2[

There is a version for atomic measures : the condition to ensures wellrecovery is a minimum separation distance t(λ) between atoms of µ2.

Main goal : to estimate ‖Ĉ − d(C2√2σ ∗ µA)‖L2 .

17 / 24


Convergence result on the subordination methodRecall : W = UXU∗ + A, U Haar unitary, A,X ∈ MN(C) self-adjoint.

TheoremSuppose that N ≥ K1.Then,

E(‖Ĉ − d(C2√2σ1 ∗ µA)‖

2L2

)≤ K2

N2+

K3N3

+K4N4

,

with K1,K2,K3,K4 depending explicitly on the first six moments of Aand X .

CorollarySuppose that dLevy (µA, µ) ≤ 1N , with dµ analytically extendable toR + i ]− τ, τ [, τ ∈]0,∞], then for N ≥ K1,

P(‖µ̂A − µA‖L2 > δ) ≤C

δ2

(K2N2

+K3N3

+K4N4

) ττ+2√

2σ1

,

for some C depending on the regularity of µ.

18 / 24


Convergence result on the subordination method :simulations

Figure: Simulation of ‖Ĉ − d(C2√2σ1 ∗ µA)‖L2 in the additive case for N from50 to 2000 (with a sampling of size 100 for each size), theoretical bound inTheorem, and ratio of the theoretical bound on the simulated error.

19 / 24


Convergence result on the subordination method :simulations

Figure: Simulation of ‖Ĉ − d(Cκ ∗ µA)‖L2 in the multiplicative case for N from100 to 2000 (with a sampling of size 100 for each size) , theoretical bound inTheorem, and ratio of the theoretical bound on the simulated error.

20 / 24


Matricial subordination in the additive case

(Pastur and Vasilchuk, Kargin)

W = UXU∗ + A

[For R ∈ MN(C),R = R∗, GR(z) = (z − R)−1, GµR (z) = tr((R − z)−1)]Set fA(z) = − tr(AGW (z)) and fX (z) = − tr(XGW (z)). Then, set

ωA(z) = z +E(fX (z))E(GµW (z))

, ωX (z) = z +E(fA(z))E(GµW (z))

. (1)

The functions ωA, ωB are called matricial subordination functions.µ3 = µA � µX W = UXU∗ + A

ω̃A(z) + ω̃X (z) = z +1

Gµ3 (z)ωA(z) + ωX (z) = z +

1EGµW (z)

Gµ3(z) = GµA(ω̃A(z)) = GµX (ω̃X (z))EGµW (z) =GµA(ω̃A(z)) + RA(z)

=GµX (ω̃X (z)) + RX (z)

1. Multiplicative case ? (XUAU∗X − z)−1 X−1(UAU∗− zX 2)−1X−1

2. Estimates on RA(z),RX (z) ? Estimates on |GµW − EGµW | ?21 / 24


Matricial subordination in the additive case

RA(z) =E [(GµW − EGµW )(φ− Eφ)]− E [(fX − EfX )(ψ − Eψ))]

EGµW,

with φ = tr(GA(ωA(z))UXU∗G̃H) and ψ = tr(GA(ωA(z))GH).

• Using Nevanlinna theory and Weingarten formulas gives an upperbound on 1EGµW

in terms of first moments of A and X ,

• Using Poincaré inequality on UN , matrix Hölder inequalities andWeingarten formulas give an upper bound on Var(GµW ), Var(fX ),Var(φ), Var(ψ) in terms of the first moments of X and A.[Reminder of Poincaré inequalities : if f : UN → R has mean zero,then

∫UN|f |2 ≤ 1N

∫UN‖∇f ‖2]

Finally,

|RA(z)| ≤K (=z)N2

,

where K depends on the first moments of A and X .

22 / 24


Back to the deconvolutionWe suppose here for simplicity µX = µ0. We compare then two systems :

1. the deconvolution system

ω1(z) + z = ω3(z) +

1

GµX (ω1(z)), (1)

ω1(z) + z = ω3(z) +1

GW (ω3(z)). (2)

2. the matricial subordination system of W = UXU∗ + A at ω3(z) :�X := RX (ω3(z), �A := RA(ω3(z)).ωX (ω3) + ωA(ω3) = ω3 +

1

GµA(ωA(ω3)) + �A, (a)

ωX (ω3) + ωA(ω3) = ω3 +1

GµX (ωX (ω3)) + �X= ω3 +

1

EGW (ω3). (b)

Successively deduce :

1. ωX (ω3) ' ω1 : (1), (2), (b) and the invertibility domain of 1/Gµ,2. ωA(ω3) ' z : ωX ' ω1, (2) and (b),3. GµA(z) ' GµA(ωA(ω3)) ' EGW (ω3) ' GW (ω3) : ωA(ω3) ' z , then

(a) and (b), and finally Poincaré inequalities.

23 / 24




ω1(z) + z = ω3(z) +

1

GµX (ω1(z)), (1)

ω1(z) + z = ω3(z) +1

GW (ω3(z)). (2)


1

GµA(ωA(ω3)) + �A, (a)

ωX (ω3) + ωA(ω3) = ω3 +1

GµX (ωX (ω3)) + �X= ω3 +

1

EGW (ω3). (b)


1. ωX ' ω1 : (1), (2), (b) and the invertibility domain of 1/Gµ,2. ωA(ω3) ' z : ωX ' ω1, (2) and (b),3. GµA(z) ' GµA(ωA(ω3)) ' EGW (ω3) ' GW (ω3) : ωA(ω3) ' z , then


23 / 24




ω1(z) + z = ω3(z) +

1

GµX (ω1(z)), (1)

ω1(z) + z = ω3(z) +1

GW (ω3(z)). (2)


1

GµA(ωA(ω3)) + �A, (a)

ωX (ω3) + ωA(ω3) = ω3 +1

GµX (ωX (ω3)) + �X= ω3 +

1

EGW (ω3). (b)


1. ωX ' ω1 : (1), (2), (b) and the invertibility domain of 1/Gµ,2. ωA(ω3) ' z : ωX ' ω1, (2) and (b),3. GµA(z) ' GµA(ωA(ω3))'EGW (ω3) ' GW (ω3) : ωA(ω3) ' z , then


23 / 24




ω1(z) + z = ω3(z) +

1

GµX (ω1(z)), (1)

ω1(z) + z = ω3(z) +1

GW (ω3(z)). (2)


1

GµA(ωA(ω3)) + �A, (a)

ωX (ω3) + ωA(ω3) = ω3 +1

GµX (ωX (ω3)) + �X= ω3 +

1

EGW (ω3). (b)


1. ωX ' ω1 : (1), (2), (b) and the invertibility domain of 1/Gµ,2. ωA(ω3) ' z : ωX ' ω1, (2) and (b),3. GµA(z) ' GµA(ωA(ω3)) ' EGW (ω3) ' GW (ω3) : ωA(ω3) ' z , then


23 / 24


L2-estimates

We finally get for =(z) ≥ 2√

2Var(µ0)

|GµA(z)− GW (ω3)| ≤C1(=z)|z |N2

+C2(=z)|GW (ω3)− EGW (ω3)|

|z |.

Recall

• E(|GW (ω3(z))− EGW (ω3(z))|2) ≤ C(=z)N2 ,• t 7→ 1|t+iη| ∈ L

2(R).Thus

E(∫

R

∣∣∣=GW (ω3(t + 2√2Var(µ0)i))−=GµA(t + 2√2Var(µ0)i)∣∣∣2dt)≤K2N2

+K3N3

+K4N4

.

24 / 24


L2-estimates

We finally get for =(z) ≥ 2√

2Var(µ0)

|GµA(z)− GW (ω3)| ≤C1(=z)|z |N2

+C2(=z)|GW (ω3)− EGW (ω3)|

|z |.

Recall

• E(|GW (ω3(z))− EGW (ω3(z))|2) ≤ C(=z)N2 ,• t 7→ 1|t+iη| ∈ L

2(R).Thus,

E(‖Ĉ − d(µA ∗ C2√2Var(µ0)‖

2L2

)≤K2N2

+K3N3

+K4N4

.

24 / 24


THANK YOU !

24 / 24

Deconvolution problemThe free deconvolution

Proof of the theorem

Spectral deconvolution of unitary invariant modelsjgarzav/Slides_Pierre_Tarrago.pdf · 2020. 10....

Documents

Transcript of Spectral deconvolution of unitary invariant modelsjgarzav/Slides_Pierre_Tarrago.pdf · 2020. 10....