MOSWOC flare forecast verification Sophie Murray

FLARECAST Consortium Meeting

2015 January 14-15

Forecast methods

MOSWOC flare forecasting method

Forecasters first classify all active regions on disk.

Average flare rate, μ, for each McIntosh class determined from data archive (see Bloomfield et al, 2012).

Poisson flare probability, P, of observing N flares in next 24 hours is calculated for each active region,

Pμ(N) = μN/N! e(-μ)

Sunspot region summaries

Issued every six hours for each classified active region.

Total % = 1 - (1 - Regiona%)*(1 - Regionb%)*(1 - Regionc%)*…

% probability McIntosh class Hale class

Radio blackout forecasts

Forecasters edit SRS values before issuing total forecasts.

Whole disk forecasts as part of Space Weather Guidance Documents (via email or online).

Issued at midnight, with a midday update if necessary.

Results M-class flare forecasts

Python and R

Sunspot Region Summaries Reliability diagrams

Raw Issued

50 M-class flares since 2015 July 1

Sunspot Region Summaries Relative Operator Characteristic curves

Raw Issued

50 M-class flares since 2015 July 1

Area = 0.844 Area = 0.908

FPR

TP

R

Radio blackout forecasts Reliability diagram

333 M-class flares since 2014 January 1

Day 1

Radio blackout forecasts Reliability diagram

Day 2

333 M-class flares since 2014 January 1

Radio blackout forecasts Reliability diagram

Day 3

333 M-class flares since 2014 January 1

Radio blackout forecasts Reliability diagram

Day 4

333 M-class flares since 2014 January 1

Radio blackout forecasts Relative Operator Characteristic curve

Day 1

333 M-class flares since 2014 January 1

Area = 0.771

Radio blackout forecasts Relative Operator Characteristic curve

Day 2

333 M-class flares since 2014 January 1

Area = 0.729

Radio blackout forecasts Relative Operator Characteristic curve

Day 3

333 M-class flares since 2014 January 1

Area = 0.673

Radio blackout forecasts Relative Operator Characteristic curve

Day 4

333 M-class flares since 2014 January 1

Area = 0.641

Real time verification Ranked Probability Score

MOSWOC real time forecast verification

In numerical weather prediction, for forecasts that are categorical and probabilistic, Ranked Probability Score is the obvious choice. For geomagnetic storms:

where P(Gi) = probability that the observed category is ≤ Gi O(Gi) =

25

0

)()(

i

GiOGiPRPS

0 if observed category < Gi 1 if observed category ≥ Gi RPS range is [0,1]

0 is a perfect score

M. Sharpe

Real time forecast verification Individual forecasts

M. Sharpe

To determine what is a ‘good’ forecast Compare the performance to a reference forecast: random chance persistence climatology

Calculate a Skill Score:

refRPS

RPSRPSS 1

RPSS range is (-∞,1] 1 = perfect score 0 = no additional skill compared to the reference

MOSWOC real time forecast verification

M. Sharpe

Real time forecast verification Rolling monthly results

M. Sharpe