Decision making as a model
5. a. more models and measuresb. costs and benefitsc. the optimal criterion (Bayes is back)
PH
PFA
zH
zFA-μs
μs/σsPFA = Φ(-λ), zFA = -λ
Unequal variance model
σn=1, σs
tg(tg(θθ) = 1/) = 1/σσss
θzH
= ---- + --- z FA
μ s
σ s
σ s
μs – λ μs – λ PH = Φ zH = σs σs
0 λ μs
zH
zFAΔΔmm
e ae a
Δm does not distinguish between large and small σs
distance to origin analogous to d' :
OO
Measures:
de = Oe√2
da = Oa√2 μs √1 + σs
2
ZH = -Z
FA
(Pythagoras and similar triangles)
To get Az , the surface under the ROC-curve according to the Gaussian model with unequal variances:
Produce a formula for the proportion correct 2AFC-experiment under that model: PCZ
According to the area theorem PC equals A so PCZ equals Az
PPHHArea under Gaussian ROC-curve: Az
PPFAFA
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
0,4
0,45
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
Gaussian 2AFC:
PC = p(xs>xn)
= p(xs-xn>0)
n s
0
0,1
0,2
0,3
0,4
0,5
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
PCZ = p(xs>xn)
= p(xs- xn>0)
-μs =1 - Φ √1 + σs
2
-μs
= Az according to area theorem!
The variance of the difference of two independent random variables is the sum of both variances
μs
= Φ √1
+ σs2
PPHH Area under Gaussian ROC-curve:
Az
PPFAFA
Az = Φ(da /√2)
Equal variances: Az = Ad' = Φ(d′ /√2)
(already shown)
tg
μs/σs = Φ √ √1/1/σσss
2 2 + 1+ 1
μs
= Φ √1
+ σs2
zH
zFA-μs
μs/σsPFA = Φ(-λ), zFA = -λ
tg(tg(θθ) = 1/) = 1/σσss
θ
μs – λ μs – λ PH = Φ zH = σs σs
λ
β with unequal variances: h / f = fs(λ) / fn(λ)
hf
1 -λ2/2 1 -zFA2/2
fn(λ) = --------- e = --------- e √(2π) √ (2π)
1 -(λ – μs )2 /2σ2 1 -zH2/2
fs(λ) = ------------ e = ------------ e σs√(2π) σs√ (2π) 1 (zFA
2 – zH2)/2
Divide: ------------------------- = --- e σs
So: βunequal = βequal / σs
(from slide 5)
A A' Az
da de
Ad'
d'
S LRc B'' S β β c
Sensitivity
Criterium/bias
General. Rough Gaussianmany pts (one pt) σn ≠ σs σn = σs
Survey of signal detection measures
With these measures the sensitivity and the criteria of humans, machines and systems can be expressed independently
What are the costs of missing a weapon/explosive at an airport?
What are the costs of a false alarm?
What are the costs and benefits of baggage screening?
Costs and benefits:Pay-off matrix
CMiss VHit
VCR CFA
“no” “yes”
S(+N)
N
NB. C is a positive number: “a false alarm will cost you € 5”
EV = p(Hit)•Vhit - p(Miss)•CMiss+ p(CR)•VCR - p(FA)•CFA
= p(s)•{PH• VHit – (1-PH)•CMiss} + p(n)•{(1-PFA)•VCR - PFA•CFA}
Compare with doing nothing:EV = p(n)•VCR – p(s)CMiss
NB.: no free lunch, no free screening!
NB.:PH∙p(s)!
- Cscr
An optimal decison in uncertainty:
Set criterion at the value of x (xc) at which expected value/utility of “Yes” equals expected value/utility of “No”
EV(Yes|xc) = EV(No|xc)
xxc
EV(Yes|xc) = EV(No|xc)
VHit• p(Hit) – CFA• p(FA) = VCR•p(CR) - CMiss•p(Miss)
“cost”: CFA positive!
VHit• p(signal|xc) – CFA• p(noise|xc) = VCR•p(noise|xc) - CMiss•p(signal|xc)
p(signal|xc) VCR + CFA ---------------- = --------------- p(noise|xc) VHit + CMiss
But do we know that one?
VHit•p(signal|xc) + CMissp(signal|xc) = VCR•p(noise|xc) + CFA•p(noise|xc)
p(signal|xc)(VHit + CMiss) = p(noise|xc)(VCR + CFA)
p(signal|xc) VCR + CFA ---------------- = --------------- p(noise|xc) VHit + CMiss
We want this one
We know (in principle):
p(x|noise)
p(x|signal)
required: a way to get from p(A|B) to p(B|A)
Bayes’ Rule!
p(A|B) p(B|A) p(A) --------- =---------- • ------- p(A|¬B) p(B|¬A) p(¬A)
(odds form)
Applied to signal detection:
p(signal|xc) --------------- p(noise|xc)
p(xc|signal) p(signal)
p(xc|noise) p(noise) •=
p(signal|xc) VCR + CFA ---------------- = --------------- p(noise|xc) VHit + CMiss
p(xc|signal) p(signal) VCR + CFA -------------- • --------- --- = ----------------------------- • --------- --- = --------------- p(xc|noise) p(noise) VHit + CMiss
p(xc|signal) p(noise) VCR+CFA
----------------- = ------------- •-- = ------------- • -------------------------- p(xc|noise) p(signal) VHit+CMiss
Bayes
LRc prior odds payoff matrix
S, β
So: an ideal observator, knowing prior odds and pay-off matrix, can compute an optimal criterion.
People are not that good at arithmetic, but adapt reasonably well to pay-off matrix and prior odds
p(xc|signal) p(noise) VCR+CFA
----------------- = ------------- •-- = ------------- • -------------------------- p(xc|noise) p(signal) VHit+CMiss
Top Related