A statistical approach to PVA validation: an example using diffusion approximation models

Eli HolmesREUT/NOAA/NWFSCEli.Holmes@noaa.gov

Updated with a few comments after talk

Overviewυ Different validation approachesυ Example: a cross-validation study of diffusion

approximation PVAsυ Presenting uncertainty.

FOR MORE INFO...

faculty.washington.edu/eeholmes

Methods for testing PVAsfrom McCarthy et al. 2002. “Testing the Accuracy of PVA” Cons. Bio.

Subjective, ignores variability, single trajectories unlikely to be similar to mean

Compare mean or median predictions with observations

Only assesses average number or frequency of occurrences within a group, ignores variability

Compare observed vs predicted frequency of events

Assesses both the mean and variability, generally requires transformation of data to a standard variate, lots of data

Compare probability distributions of population size or parameters

Diffusion approximation PVA

eggs 1 2 3 4

Nt=N0 exp(µt+εt) where ε ~ Normal(0,σ)

reproduction

Survival to next age class

Let reproduction and survival vary yearly

Basic Idea of DA PVA

0

20

40

60

80

100

120

1952

1955

1958

1961

1964

1967

1970

1973

1976

1979

1982

1985

1988

1991

1994

1997

redd

s p

er m

ile

0.5 1 1.5 2 2.5 3 3.5 4

ln(N) at 1985

prob

abili

ty d

ensi

ty

Parameters of a DA model

Nt=N0*exp(µt+εt) where ε ~ N(0,σ)

Parameter that governs the median rate of decline.

“Process error”: parameter that describes the long-term variability of the process.

µ̂ σ̂

Cross-validationυ 147 chinook and 42 steelhead 30-70 year time series from ESUs in

WA, OR, and CA

01020304050607080

1952

1955

1958

1961

1964

1967

1970

1973

1976

1979

1982

1985

1988

1991

1994

1997

red

ds

per

mil

e

parameterization evaluation

Does the DA model predict the frequency of actual declines?

0

0.2

0.4

0.6

0.8

1

10 8 6 4 2 1 1/2 1/4 1/6 1/8 1/10

Observed indata

threshold (rel. to size at the start of the eval. period)

fraction of stocks that

dip below a threshold

Do the projected population sizes follow the expected theoretical distribution?

0

20

40

60

80

100

120

1952

1955

1958

1961

1964

1967

1970

1973

1976

1979

1982

1985

1988

1991

1994

1997

redd

s p

er m

ile

0.5 1 1.5 2 2.5 3 3.5 4

ln(N) at 1985

prob

abili

ty d

ensi

ty

Do the projected population sizes follow the expected theoretical distribution?

1 5 9 13 171 5 9 13 17

1 5 9 13 17 1 5 9 13 17

µ̂

µ̂ of dist.

Do my estimates of σ follow the expected theoretical distribution?

0

5

1

5

2

σ̂

σ̂ of dist.1 5 9 13 17 1 5 9 13 17

1 5 9 13 17 1 5 9 13 17

Transforming data to a common currency

For other examples see McCarthy et al. 2002. Conservation Biology

Problem: don’t view the same population process over and over Actual data: many different processes with different underlying parameters (growth rates and variability)Solution: transform data to a standardized metric that has the same statistical distribution for all processes

slpdf is )ˆ var(offreedom of degrees

µ

Standardized ln(Nt+15/Nt)= ψdistribution

slpdf

epslp

eslppslp

ep t

LnLndfdfdf

222

1~11

2ˆˆ

)(γσσ

ψψ

⎟⎟⎠

⎞⎜⎜⎝

⎛

−+

−+

−

like~)ˆvar(likeNormal~ˆ

2 −

−

χµ

µ

-0

no trend

)ˆvar(of estimatein bias no

µ

Results for population distribution

Predicted t distribution

Histogram of actual t statistics

Trend in the rate of decline?υ Fluctuating or declining stocks

υ No significant trend

υ Rapidly increasing stocksυ Significant negative trendυ Estimate of µ lower for bigger population size

Standardized σ distribution

slpp dfσ is ˆ offreedom of degrees

2

x 1

no trend

( ) ),(~ˆˆ 22slpslppe dfdfFσσ

like~ˆ 22 −χσ

Results for σ Predicted F distribution

Histogram of actual F statistics

Trend in σ?

υ Estimate of σ was higher when counts were really smallυ Demographic stochasticity?υ Sampling effect?

Estimate of σ sensitive to percent of sampling error in the observation

Percent error tends to be larger when counts are smalle.g. Dunham and Rieman. 2001. Sources and magnitudes of

sampling error in redd counts for Bull Trout. North American Journal of Fisheries Management 21:343–352

Can the model predict actual declines?

υ Simple model makes many simplifying assumptionsυ density-independenceυ no environmental correlationυ no trendsυ diffusion approximation of age-structured

population

Actual frequency of declines

Estimate using Dennis-Holmes

Estimate using Dennis

Maximum yield relationship for modeling

Belief

Yie

ld

True believerNon-believer

25 yr 100 yr

support for different probs of 90% decline

support for different λ’s

25 yr 100 yr

pdf of prob of 90% decline

pdf of λsupport for different λ’s

support for different probs of 90% decline

25 yr 100 yr

support for different probs of 90% decline

support for different λ’s

FOR MORE INFO...

A variety of matlab and Splus code for DA PVAs is at faculty.washington.edu/eeholmes

Holmes, E. E. 2001. Estimating risks in declining populations with poor data. Proceedings of the National Academy of Science 98: 5072-5077.

Holmes and Fagan. 2002. Validating population viability analysis for corrupted data sets. Ecology in press.

Holmes, E. E. Beyond theory to application and evaluation: diffusion approximations for population viability analysis.

Reprints or pre-prints available at faculty.washington.edu/eeholmes