Greedy algorithm (How to select sample points) · Greedy sampling Characteristics of greedy...

Greedy algorithm (How to select sample points)Goal: Selection of sample points µ1, . . . ,µN such that

VN = span{uδ(µ1), . . . , uδ(µN )} ≈Mδ

Thursday, August 23, 2012

Greedy algorithm (How to select sample points)

Estimated error feedback (A posteriori estimates): Consider

µ −→ solve: a(uN (µ), vN ;µ) = f(vN ;µ), ∀vN ∈ VN −→ ηN (µ)

where ηN (µ) is an a posteriori estimation for �uN (µ)− uδ(µ)�Vδ .

Goal: Selection of sample points µ1, . . . ,µN such that

Greedy algorithm:

Set N = 1, choose µ1 ∈ P arbitrarily.

1. Compute uδ(µN ) (truth problem: computationally expensive)

2. Set VN = span{VN−1, uδ(µN )}

3. Find µN+1 = arg maxµ∈P ηN (µ)

4. Set N := N + 1 and goto 1. while maxµ∈P ηN (µ) > Tol

• Only N truth problems need to be solved (compared to POD approach)• Error control through estimator for any parameter value

Greedy algorithm:

1. Compute uδ(µN ) (truth problem: computationally expensive)

2. Set VN = span{VN−1, uδ(µN )}

3. Find µN+1 = arg maxµ∈P ηN (µ)

4. Set N := N + 1 and goto 1. while maxµ∈P ηN (µ) > Tol

Greedy algorithm: illustration

One-dimensional parameter space

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

rError estimateTrue error

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

Error estimateTrue error

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

Greedy algorithm

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

10 12 14 16 18 20k

1x10-8

1x10-7

1x10-6

0.00001

0.0001

Dorfler marking vs. greedy-selection:

Greedy sampling

Characteristics of greedy RB-space construction:

o Cheap: N truth problems need to be solved for an N -dimensional RB-space.o Accuracy is guaranteed by theoretical results.o Provides a hierarchical family of reduced basis spaces: VN−1 ⊂ VN .

Greedy sampling

Practical hint: The basis functions uδ(µ1), . . . , uδ(µN ) might be highly linearlydependent and lead to a poor conditioning of the system matrix

Bij(µ) = a(uδ(µj), uδ(µi);µ).

Greedy sampling

Remedy: Apply the Gram-Schmidt orthonormalization procedure to the set ofbasis functions using the inner product (·, ·)V. This leads to a new set {ξ1, . . . , ξN}of basis functions with better conditioning properties.

Practical hint: The basis functions uδ(µ1), . . . , uδ(µN ) might be highly linearlydependent and lead to a poor conditioning of the system matrix

Bij(µ) = a(uδ(µj), uδ(µi);µ).

Theoretical estimates of the greedy algorithm

Here: PN : V→ VN denotes the projection onto VN .

Exact greedy algorithm:

1. Set VN = span{VN−1, u(µN )}

2. Find µN+1 = arg maxµ∈P �u(µ)− PNu(µ)�V

3. Set N := N + 1 and goto 1. while maxµ∈P �u(µ)− PNu(µ)�V > Tol

Difference to practical greedy implementation:

1. Exact solutions u(µ) available (instead of uδ(µ)).

2. Exact error �u(µ)− PNu(µ)�V is supposed to be known/computable.

3. The projection PN is used instead of the Galerkin-projection based onbilinear form a(·, ·;µ): Find uN (µ) ∈ VN s.t.

a(uN (µ), vN ;µ) = f(vN ;µ), ∀vN ∈ VN .

Recall: V : Hilbert space,

M = {u(µ) ; ∀µ ∈ P} ⊂ V : Exact solution manifold.

What is provided by the exact greedy algorithm? A N-dimensional sub-space Vgr

N ⊂ Vδ with approximation error

d(VgrN ) = sup

v∈Minf

vN∈VgrN

�v − vN�V.

VgrN is of the particular form Vgr

N = span{uδ(µ1), . . . , uδ(µN )}.

What means optimal? Find the N-dimensional subspace VN ⊂ Vδ such thatthe approximation error

d(VN ) = supv∈M

infvN∈VN

�v − vN�V

is minimized. We denote this space by VKolN and d(VKol

N ) as the Kolmogorovwidth.

d(VgrN ) = sup

v∈Minf

vN∈VgrN

�v − vN�V.

N = span{uδ(µ1), . . . , uδ(µN )}.

What means optimal? Find the N-dimensional subspace VN ⊂ Vδ such thatthe approximation error

d(VN ) = supv∈M

infvN∈VN

�v − vN�V

is minimized. We denote this space by VKolN and d(VKol

N ) as the Kolmogorovwidth.

Note: The best approximation space VKolN is in general not a subspace ofM.

d(VgrN ) = sup

v∈Minf

vN∈VgrN

�v − vN�V.

N = span{uδ(µ1), . . . , uδ(µN )}.

1 10 100N

0.0001

100000

Kolmogorov widthupper bound for greedy

M = 1α = 2⇒ C = 524�288

Theorem. (Polynomial decay of greedy approximation)Suppose that d(XKol

N ) ≤ M and

d(VKolN ) ≤ MN−α, N > 0,

for some M > 0 and α > 0. Then,

d(VgrN ) ≤ CMN−α, N > 0,

with C := q12 (4q)α and q := �2α+1�2.

5 10 15 20N

1x10-381x10-361x10-341x10-321x10-301x10-281x10-261x10-241x10-221x10-201x10-181x10-161x10-141x10-121x10-101x10-81x10-6

0.00010.01

1Kolmogorov widthupper bound for greedy

Theorem. (Exponential decay of greedy approximation)There holds

d(VgrN ) ≤ 2√

32N d(VKol

d(VKolN ) = e−N1.5

Remarks:

1. More estimates are available.

2. Estimates of the form d(VgrN ) ≤ C d(VKol

N ) can not be established(counter-example exists).

3. The assumptions on the greedy algorithm can be relaxed:o Error estimates η(µ) can be used instead of the exact error.o Discrete solutions uδ(µ) can be considered instead of the exact

solutions u(µ).o Yields different estimates (of course).

Remarks:

References:

[1] Binev et al. Convergence rates for greedy algorithms in reduced basis methods. SIAM J. Math. Anal. (2011) vol. 43 (3) pp. 1457-1472

[2] Buffa et al. A priori convergence of the Greedy algorithm for the parametrized reduced basis method. Esaim-Math Model Num (2012) vol. 46 (3) pp. 595-603

Remarks:

References:

[1] Binev et al. Convergence rates for greedy algorithms in reduced basis methods. SIAM J. Math. Anal. (2011) vol. 43 (3) pp. 1457-1472

[2] Buffa et al. A priori convergence of the Greedy algorithm for the parametrized reduced basis method. Esaim-Math Model Num (2012) vol. 46 (3) pp. 595-603

Open problem:

Parametrized problem ⇒ Decay of Kolmogorov width

EfficiencyHow to solve: “For µ ∈ P, find the solution uN (µ) ∈ VN of

a(uN (µ), vN ;µ) = f(vN ;µ), ∀vN ∈ VN .”

fastly?

Further assumption: The output functional s(·;µ) is linear.

It shall be avoided that, if VN = span{ξ1, . . . , ξN}, the matrix

(Aµ)ij = a(ξj , ξi;µ), ∀ 1 ≤ i, j ≤ N,

needs to be reassembled for each new parameter value µ ∈ P.Why? ⇒ Assembly process depends on N = dim(Vδ).

Efficiency

Assumption:

a(w, v;µ) =Ma�

Θma (µ) am(w, v),

f(v;µ) =Mf�

Θmf (µ) fm(v),

s(v;µ) =Ms�

Θms (µ) sm(v),

Θma ,Θm

f Θms : D → R µ− dependent functions,

am : V× V→ R µ− independent forms,fm, sm : V→ R µ− independent forms,

Efficiency: example

Example: Convection-diffusion

a(u, v; ε) = ε

0u�(x)v�(x) dx +

0u�(x)v(x) dx,

a1(u, v; ε) =� 1

0u�(x)v�(x) dx, Θ1

a(ε) = ε,

a2(u, v; ε) =� 1

0u�(x)v(x) dx, Θ2

a(ε) = 1,

Efficiency: example

Example: Convection-diffusion

a(w, v;µ) =15�

∇w ·∇v +�

∇w ·∇v,

ai(w, v;µ) =�

∇w ·∇v, Θia(µ) = µi, i = 1, . . . , 15,

a16(w, v;µ) =�

∇w ·∇v, Θ16a (µ) = 1,

Example: Heat conduction on thermal blocks

a(u, v; ε) = ε

0u�(x)v�(x) dx +

0u�(x)v(x) dx,

a1(u, v; ε) =� 1

0u�(x)v�(x) dx, Θ1

a(ε) = ε,

a2(u, v; ε) =� 1

0u�(x)v(x) dx, Θ2

a(ε) = 1,

Off-line:

On-line:

Affine assumption: Off-line/on-line

For each new parameter value µ ∈ P

Given VN = span{ξi | i = 1, . . . , N} precompute

(Am)i,j = am(ξj , ξi), ∀ 1 ≤ i, j ≤ N,

(Fm)i = fm(ξi), ∀ 1 ≤ i ≤ N,

(Sm)i = sm(ξi), ∀ 1 ≤ i ≤ N.

Rem. Depends on N = dim(Vδ).Rem. Size of Am and Fm, Sm is N2 resp. N .

1. Assemble (depending on M and N , i.e. ∼MN2 resp. ∼MN)

Aµ =Ma�

Θma (µ)Am Fµ =

Θmf (µ)Fm

2. Solve Aµu(µ) = Fµ. (depending on N , i.e ∼ N3 for LU factorization)3. Compute

s(uN (µ);µ) =Ms�

un(µ) Θms (µ) (Sm)n.

Off-line/On-line procedure: Is it worth?

Computational costs:

o Off-line procedure: Toff.

o One on-line evaluation: Ton.

o One truth solve: Ttr � Ton.

Evaluation for M parameter values:1. Brute force approach: M · Ttr.2. Reduced basis method: Toff + M · Ton.

Theoretical considerations:

o The parameter space P is a continuous space (not discrete): M is potentiallyarbitrarily high.

o Whenever the number M of parameter evaluations is high enough, thereduced basis method is always cheaper.

1x103 1x104 1x105 1x106

Number of parameter evaluations: M

Realistic example:

o Ton = 1.o Ttr = 102 (realistic speed-up).o Toff = 104 (N = 100).

Brute force: M · Ttr = M · 102

RBM: Toff + M · Ton = 104 + M

Schematic overview of Reduced Basis Method (so far)

Offline procedure:

1. Construct the reduced basis space VN empirically (Greedy)2. Precompute the matrices Am and the vectors Fm, Sm

uN (µ) =N�

un(µ) ξn

Online procedure:

µ −→ solve: uN (µ) −→ s(uN (µ);µ)

which consists of

1. AssembleAµ =

Θma (µ) Am Fµ =

Θmf (µ) Fm

2. Solve Aµu(µ) = Fµ (N -dimensional linear system, N � N )3. Compute s(uN (µ);µ)

Characteristics: Independent of N = dim(Vδ): cheap. Feasible in a many-query context.

Schematic overview of Reduced Basis Method (so far)

Offline procedure:

1. Construct the reduced basis space VN empirically (Greedy)2. Precompute the matrices Am and the vectors Fm, Sm

uN (µ) =N�

un(µ) ξn

Online procedure:

µ −→ solve: uN (µ) −→ s(uN (µ);µ)

which consists of

1. AssembleAµ =

Θma (µ) Am Fµ =

Θmf (µ) Fm

2. Solve Aµu(µ) = Fµ (N -dimensional linear system, N � N )3. Compute s(uN (µ);µ)

Characteristics: Independent of N = dim(Vδ): cheap. Feasible in a many-query context.

Idea: Restrict the solution space from Vδ to VN ≈ {uδ(µ) : µ ∈ P}

empirically. →Discard unnecessary

modes.

A posteriori error estimation

A posteriori estimate

So far we assumed the a posteriori estimation process:

o Estimates the discrete error �uδ(µ)− uN (µ)�V by ηN (µ).

o Crucial for selection process in greedy algorithm. (Off-line)

o Should be cheap, i.e., independent on N = dim(Vδ). (Off- and On-line)

o Certifies the model order reduction error with a computable bound:

�uδ(µ)− uN (µ)�V ≤ ηN (µ)

A posteriori estimate

Why only �uδ(µ)− uN (µ)�V?

Error of truthapproximation

Error of modelorder reduction

Given by theapproximationspace Vδ

Depends on thereduced basisspace VN

Needs to be estimated

�u(µ)− uN (µ)�V ≤ �u(µ)− uδ(µ)�V� �� + �uδ(µ)− uN (µ)�V� ��

Assumption:

supµ∈P

�u(µ)− uδ(µ)�V ≤ tol

by the choice of Vδ.

The error estimate ηTruth solution: Find uδ(µ) ∈ Vδ such that

a(uδ(µ), vδ;µ) = f(vδ;µ), ∀vδ ∈ Vδ.

Define the errore(µ) = uδ(µ)− uN (µ) ∈ Vδ.

The error estimate η

Error committed by the modelorder reduction, i.e. by re-stricting the solution spacefrom Vδ to VN .

Truth solution: Find uδ(µ) ∈ Vδ such that

The error equation yields

a(e(µ), vδ;µ) = r(vδ;µ), ∀vδ ∈ Vδ,

r(vδ;µ) = f(vδ;µ)− a(uN (µ), vδ;µ), ∀vδ ∈ Vδ.

The error equation yields

a(e(µ), vδ;µ) = r(vδ;µ), ∀vδ ∈ Vδ,

r(vδ;µ) = f(vδ;µ)− a(uN (µ), vδ;µ), ∀vδ ∈ Vδ.

Using the Riesz representation theorem yields e(µ) ∈ Vδ s.t.

(e(µ), vδ)V = r(vδ;µ), ∀vδ ∈ Vδ,

�r(·,µ)�V�δ

= supvδ∈Vδ

r(vδ;µ)�vδ�V

= �e(µ)�V, ∀vδ ∈ Vδ.

Error certificationRecall: The solvability requires αδ(µ) > 0 s.t.

αδ(µ) �vδ�V ≤ supwδ∈Vδ

a(vδ, wδ;µ)�wδ�V

, or αδ(µ) �vδ�2V ≤ a(vδ, vδ;µ),

for all vδ ∈ Vδ, from which one recovers the estimator

ηN (µ) =�e(µ)�VαLB(µ)

, 0 < αLB(µ) ≤ αδ(µ).

Error certification

Difficult task but doable! (later)

Recall: The solvability requires αδ(µ) > 0 s.t.

, 0 < αLB(µ) ≤ αδ(µ).

Error certification

, 0 < αLB(µ) ≤ αδ(µ).

Theorem: The estimator is reliable

�uδ(µ)− uN (µ)�V ≤ ηN (µ).

Error certification

, 0 < αLB(µ) ≤ αδ(µ).

�uδ(µ)− uN (µ)�V ≤ ηN (µ).

Proof: (Coercivy problem)

�uδ(µ)− uN (µ)� �� =e(µ)

�2V ≤1

αδ(µ)a(e(µ), e(µ);µ) =

1αδ(µ)

supwδ∈Vδ

r(e(µ);µ)

αδ(µ)sup

wδ∈Vδ

(e(µ), e(µ))V ≤1

αδ(µ)�e(µ)�V�e(µ)�V

≤ 1αLB(µ)

�e(µ)�V�e(µ)�V = ηN (µ)�e(µ)�V.

�Thursday, August 23, 2012

Error certification

, 0 < αLB(µ) ≤ αδ(µ).

�uδ(µ)− uN (µ)�V ≤ ηN (µ).

Proof: (Saddle-point problem)

�uδ(µ)− uN (µ)� �� =e(µ)

�V ≤1

αδ(µ)sup

wδ∈Vδ

a(e(µ), wδ;µ)�wδ�V

αδ(µ)sup

wδ∈Vδ

r(wδ;µ)�wδ�V

αδ(µ)sup

wδ∈Vδ

(e(µ), wδ)V�wδ�V

≤ 1αδ(µ)

�e(µ)�V

≤ 1αLB(µ)

�e(µ)�V = ηN (µ).

�Thursday, August 23, 2012

Error certification

Corollary: Effectivity is bounded from below

ηN (µ)�uδ(µ)− uN (µ)�V

≥ 1.

Error certification

≥ 1.

Theorem: The estimator is efficient

ηN (µ) ≤ γ(µ)αLB(µ)

�uδ(µ)− uN (µ)�V

Error certification

≥ 1.

Theorem: The estimator is efficient

ηN (µ) ≤ γ(µ)αLB(µ)

Proof:

�e(µ)�2V = (e(µ), e(µ))V = a(e(µ), e(µ);µ) ≤ γδ�e(µ)�V�e(µ)�V

and thusηN (µ) =

�e(µ)�VαLB(µ)

≤ γδ(µ)αLB(µ)

�e(µ)�V

Output functional

o Sharper bounds can be obtained using a primal-dual approach: Requiresto also assemble a reduced basis for the dual problem.

Theorem: If the output functional s : V → R is linear, then the error in theoutput functional can be bounded by

|s(u(µ))− s(uN (µ))| ≤ �s�V�δ

ηN (µ).

Proof:

|s(uδ(µ))− s(uN (µ))| = |s(uδ(µ)− uN (µ))|

≤ supvδ∈Vδ

s(vδ)�vδ�V

≤ �s�V�δ

ηN (µ).

On-line computation of error estimate

Question: How to compute ηN (µ) independently of N = dim(Vδ)?

Recall:ηN (µ) =

�e(µ)�Vδ

αLB(µ)

o µ → αLB(µ) will be discussed later.o Here: Computation of �e(µ)�Vδ .

Goal: Decompose �e(µ)�V into a pre-computable parameter independent Off-

line part and a parameter dependent On-line part which is independent on N .

Recall:

(e(µ), vδ)V = r(vδ;µ) = f(vδ;µ)− a(uN (µ), vh;µ), ∀vδ ∈ Vδ.

Greedy algorithm (How to select sample points) · Greedy sampling Characteristics of greedy...

Documents

Transcript of Greedy algorithm (How to select sample points) · Greedy sampling Characteristics of greedy...

MAGNETOESTATICA - materias.df.uba.armaterias.df.uba.ar/ft1a2020c1/files/2020/08/clase9.pdf · No hay cargas magneticas´ rB = 0 ... Campo magnetico´ B QL 2 Se ve que las constantes

Greedy Approximation with Non- submodular Potential Function

Hybrid-ε-greedy for Mobile Context-Aware …µ-greedy for Mobile Context-Aware Recommender System 469 • Finding situations that are similar to the current one (CBR); • Making

INFORMACION SUMINISTRADA POR LOS RADARES … · • Relación entre reflectividad y precipitación • Z = a Rb Relación de Marshall-Palmer – a y b constantes experimentales. –

2015 - Cheap-GSHP€¦ · της υρωπαϊκή ς Ένωσης ³Horizon 2020. 2 Cheap-GSHPs ... sk ass, ds & ul o ns. 6 Cheap-GSHPs ΑΝΑΠΤΥΞΗ ... ευρεία χρήση

FEATURES COMPARISON Air conditioning - Air Super Cheap · Wall Mounted Air Conditioner AJ Series – Air conditioning âËÁ´ (ÅŒÒ§μÑÇàÍ§)* your home Ã m ¡¯ ¯´ ´ª

-BMC: A Bayesian Ensemble Approach to Epsilon-Greedy …auai.org/uai2019/proceedings/papers/162.pdf · "-BMC: A Bayesian Ensemble Approach to Epsilon-Greedy Exploration in Model-Free

LABORATORY OF ΕΡΓΑΣΤΗΡΙΟ DEPARTMENT ΤΜΗΜΑ …cheap-gshp.eu/wp-content/uploads/2016/10/SBE16-The... · study of lawns, green walls and green roofs Thermal behaviour

Tutorial: Sparse Signal Recovery - Home | Institute for ... · Tutorial: Sparse Signal Recovery ... kuSc k1: Greedy, iterative algorithms for sparse matrices ... Decoding via linear

RB Cromo Italiano - RAYS

ME425/525: Advanced Topics in Building Science · ─Pros: i. It’s cheap ii. It’s relatively harmless iii. Occupants emit CO 2, can get some idea of AER form this ─Cons: i.

New Double Chooz: Latest results in the multiple detector … · 2017. 2. 27. · Reactor antineutrino and signal Advantages of reactor experiments •cheap source / no matter effect

Dictionary geometry, greedy algorithms, and convex relaxation - IAS

Kruskal’s Greedy Algorithmmath.mit.edu/research/highschool/primes/Switzerland/2018/5-Trees.pdfKruskal’s Greedy Algorithm Jonas Hofmann Kantonsschule Uster 5th year Mentor: Kaloyan

Recent Topics in Machine Consciousness and the Evolution ...und+Bachelor_Arbeiten/MasterThe… · 7 Future Work and Discussion 58 ... is also a cheap and fast way to test theories

Thessaloniki rb-24

Bfdx cheap discount handheld two way radios bf-7110

TGF-β Signaling Regulates the Expression of Multiple Target … · 2015. 7. 24. · Bcl-xL, Bcl-2 TGF-β Target ... Rb bHLH Rb Transcription Factor G1 S Nucleus Ras p15 ... ©2011,

- DRAFT - NOT FOR CONSTRUCTION · 2014. 11. 13. · rb-1a-3 rb-013 rb-2a-3 rb-023 rb-3a-3 rb-033 rb-4a-3 rb-043 rb-5a-3 rb-053 301 abbreviations: bwgw: cagw: cipr: etoh: exgw: fw:

Inference in first-order logic - Northern Arizona Universityedo/Classes/CS470-570_WWW/slides/...♦ Reducing first-order inference ... – proposition symbols: King(John), Greedy(John),