Decision making as a model 5.a. more models and measures b. costs and benefits c. the optimal...

Decision making as a model

5. a. more models and measuresb. costs and benefitsc. the optimal criterion (Bayes is back)

Unequal variances:

Same sensitivity? (model uneqal variances)

A B

Different d'-s (model equal variances)

PH

PFA

zH

zFA-μs

μs/σsPFA = Φ(-λ), zFA = -λ

Unequal variance model

σn=1, σs

tg(tg(θθ) = 1/) = 1/σσss

θzH

= ---- + --- z FA

μ s

σ s

σ s

μs – λ μs – λ PH = Φ zH = σs σs

0 λ μs

zH

zFAΔΔmm

e ae a

Δm does not distinguish between large and small σs

distance to origin analogous to d' :

OO

Measures:

de = Oe√2

da = Oa√2 μs √1 + σs

2

ZH = -Z

FA

(Pythagoras and similar triangles)

To get Az , the surface under the ROC-curve according to the Gaussian model with unequal variances:

Produce a formula for the proportion correct 2AFC-experiment under that model: PCZ

According to the area theorem PC equals A so PCZ equals Az

PPHHArea under Gaussian ROC-curve: Az

PPFAFA

0

0,05

0,1

0,15

0,2

0,25

0,3

0,35

0,4

0,45

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

Gaussian 2AFC:

PC = p(xs>xn)

= p(xs-xn>0)

n s

0

0,1

0,2

0,3

0,4

0,5

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

PCZ = p(xs>xn)

= p(xs- xn>0)

-μs =1 - Φ √1 + σs

2

-μs

= Az according to area theorem!

The variance of the difference of two independent random variables is the sum of both variances

μs

= Φ √1

+ σs2

PPHH Area under Gaussian ROC-curve:

Az

PPFAFA

Az = Φ(da /√2)

Equal variances: Az = Ad' = Φ(d′ /√2)

(already shown)

tg

μs/σs = Φ √ √1/1/σσss

2 2 + 1+ 1

μs

= Φ √1

+ σs2

zH

zFA-μs

μs/σsPFA = Φ(-λ), zFA = -λ

tg(tg(θθ) = 1/) = 1/σσss

θ

μs – λ μs – λ PH = Φ zH = σs σs

λ

β with unequal variances: h / f = fs(λ) / fn(λ)

hf

1 -λ2/2 1 -zFA2/2

fn(λ) = --------- e = --------- e √(2π) √ (2π)

1 -(λ – μs )2 /2σ2 1 -zH2/2

fs(λ) = ------------ e = ------------ e σs√(2π) σs√ (2π) 1 (zFA

2 – zH2)/2

Divide: ------------------------- = --- e σs

So: βunequal = βequal / σs

(from slide 5)

A A' Az

da de

Ad'

d'

S LRc B'' S β β c

Sensitivity

Criterium/bias

General. Rough Gaussianmany pts (one pt) σn ≠ σs σn = σs

Survey of signal detection measures

With these measures the sensitivity and the criteria of humans, machines and systems can be expressed independently

What are the costs of missing a weapon/explosive at an airport?

What are the costs of a false alarm?

What are the costs and benefits of baggage screening?

Costs and benefits:Pay-off matrix

CMiss VHit

VCR CFA

“no” “yes”

S(+N)

N

NB. C is a positive number: “a false alarm will cost you € 5”

EV = p(Hit)•Vhit - p(Miss)•CMiss+ p(CR)•VCR - p(FA)•CFA

= p(s)•{PH• VHit – (1-PH)•CMiss} + p(n)•{(1-PFA)•VCR - PFA•CFA}

Compare with doing nothing:EV = p(n)•VCR – p(s)CMiss

NB.: no free lunch, no free screening!

NB.:PH∙p(s)!

- Cscr

An optimal decison in uncertainty:

Set criterion at the value of x (xc) at which expected value/utility of “Yes” equals expected value/utility of “No”

EV(Yes|xc) = EV(No|xc)

xxc


We want this one

We know (in principle):

p(x|noise)

p(x|signal)

required: a way to get from p(A|B) to p(B|A)

Bayes’ Rule!


p(xc|signal) p(signal) VCR + CFA -------------- • --------- --- = ----------------------------- • --------- --- = --------------- p(xc|noise) p(noise) VHit + CMiss

p(xc|signal) p(noise) VCR+CFA

----------------- = ------------- •-- = ------------- • -------------------------- p(xc|noise) p(signal) VHit+CMiss

Bayes

LRc prior odds payoff matrix

S, β

So: an ideal observator, knowing prior odds and pay-off matrix, can compute an optimal criterion.

People are not that good at arithmetic, but adapt reasonably well to pay-off matrix and prior odds

p(xc|signal) p(noise) VCR+CFA

----------------- = ------------- •-- = ------------- • -------------------------- p(xc|noise) p(signal) VHit+CMiss

Nog meer weten over signaaldetectie?

Wickens, T. D.(2002). Elementary signal detection theory. Oxford University Press.

Macmillan, N. A. & Creelman C. D.(2005). Detection Theory: A user’s guide, 2nd ed. New York: Lawrence Erlbaum

Associates

Decision making as a model 5.a. more models and measures b. costs and benefits c. the optimal...

Documents

Transcript of Decision making as a model 5.a. more models and measures b. costs and benefits c. the optimal...