Ensemble Calibration forUncertainty Estimation

Damien Garaud 1,2

Vivien Mallet 2,1

CEREA1, INRIA2

27 October 2009

Introduction to Air Quality

∂ci∂t

= −div(Vci) + div(ρK∇ci

ρ

)+ χi(c, t) + Si − Pi

Air Quality Forecast

-10 -5 0 5 10 15 20

35

40

45

50

55

41

53

65

78

90

102

115

127

139

Ozone map (µg m−3)

May Jun Jul Aug20

40

60

80

100

120

140

160

Concentration

SimulationObservation

Ozone daily peak concentration

Uncertainty Sources

Input Data

• Emission data• Meteorological fields

Physical Parameterizations

• Chemical mechanism• Vertical diffusion coefficient

Numerical Approximations

• Time step• Vertical resolution• Schemes

Ensemble Approach

Different alternatives

• Kz : T&M or Louis• Chemical mechanism: RACM

ou RADM2• Numerical approximations: ∆t ,

Nz, . . .• Perturbations: winds,

emissions, boundaryconditions. . .

Large dispersion

0 5 10 15 20Hour

40

60

80

100

120

140

Concentration

Ozone daily profiles from 101members (µg m−3)

∂ci∂t

= −div(Vci) + div(ρK∇ci

ρ

)+ χi(c, t) + Si − Pi

Uncertainty Estimation

• Concentration: random vector — a normal distribution for instanceN (µ,Σ)

• Measure example: standard deviation (Σ)

Why is it important?

• Confidence in forecasts• Risk prediction ([O3] ≥ 240 µg m−3)• Economic and health issues• Data assimilation (matrix B)

Uncertainty Example

-10 -5 0 5 10 15 20

42

44

46

48

50

52

54

56

1.5 4.5 7.5 10.5 13.5 16.5 19.5 22.5 25.5 28.5

Standard deviation (µg m−3)

Strategy

1 List all possible alternatives and so available models (150 billion)

2 Sample possible model space efficiently in order to obtain asmaller space (100 models)

3 Select according to an objective criterion (30–50 models)

Ensemble Assessment

Rank Histogram

0 5 10 15Member

0

500

1000

1500

2000

2500

3000

3500

Num

ber

of

obse

rvati

ons

Brier Score

Assessment for a given event.c ≥ 240µg m−3

BS = 1N∑N

i=1(pi − oi)2 .

pi : probability for the date i .

oi : observed probability for thedate i .

The best Brier Score is 0.

Objective Criterion and Method

Rank Histogram Variance

S ⊆ E

Cost function:

J(S) =NS∑i=0

(bi − b̂S)2

minS⊆E

J(S)

Genetic Algorithm

1 Population {S1,S2, . . . ,SK}

2 Assessment and selection{Si/J(Si) ≤ δ}

3 Crossover (Sa,Sb)→ (Sc ,Sd )

4 Mutation Si → S ′i

5 New population of Ksubensembles

Rank Histograms

0 20 40 60 80 100Member

0.0

0.5

1.0

1.5

2.0

2.5

Num

ber

of

obse

rvati

ons

1e4

limitobservation

Large ensemble

0 2 4 6 8 10 12 14Member

0

1

2

3

4

5

6

Num

ber

of

obse

rvati

ons

1e4

limitobservation

Random subensemble

0 2 4 6 8 10 12 14Member

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Num

ber

of

obse

rvati

ons

1e4

limitobservation

Calibrated subensemble

Uncertainty Maps

Large ensemble Random subensemble Calibrated subensemble

Time series

0 10 20 30 40 50Week

16

18

20

22

24

26

28

30

Sta

ndard

Devia

tion

LargeRandomCalibrated

Weekly standard deviation (µg m−3)

Conclusion

• Ensemble calibration to estimate uncertainty

• D.Garaud and V.Mallet. Automatic generation of a largeensemble for air quality forecasting using the Polyphemussystem. Geoscientific Model Development Discussion, 2,889-933, 2009

• Comparison with other ensembles

• Other pollutants (aerosols, NO, . . . )

• Risk prediction