Statistics 101

Upload
olivierteytaud 
Category
Education

view
110 
download
0
Transcript of Statistics 101
A little bit of statistics
P( waow  news ) = ?
Posterior probability
● In case of independent items,● P( Observations  Θ) = product of
P( Observation1  Θ)
x P( Observation2  Θ)
x …
x P( ObservationZ  Θ)
Bayes theorem
● Bayes :
P( Θ  observations) P(observations)
= P( observations  Θ) P(Θ)
● So :
P( Θ  observations) = P(observations  Θ)
x P(Θ) / P(observation)
So, by independ. Items + Bayes,
● P( Θ  observations ) is proportional to
P(Θ) x P( obs1  Θ) x … x P(obsZ  Θ)
● Definitions :– MAP (maximum a posteriori) : find Θ* such that
P(Θ*observations) is max
– BPE (Bayesian posterior expectation): find ΘE = expectation of (Θobservations)
– Maximum likelihood : P(Θ) uniform
– there are other possible tools
– ErrorEstimate = Expect. (Θ – estimator)2
loglikelihood
● Instead of probas, use logprobas.● Because :
– Products become sums ==> more precise on a computer for very small probabilities
Finding the MAP (or others estimates)
● Dimension 1 :– Golden Search (unimodal)
– Grid Search (multimodal, slow)
– Robust search (compromise)
– Newton Raphson (unimodal, precise expensive computations)
● Dimension large :– Jacobi algorithm
– Or GaussSeidel, or Newton, or NewUoa, or ...
Jacobi algorithm for maximizing in dimension D>1
● x=clever initialization, if possible
● While ( x' – x > epsilon )
– x'=current x
– For each parameter x(i), optimize it
● by a 1Dim algorithm● with just a few iterates
Jacobi = great when the objective function
– can be restricted to 1 parameter
– and then be much faster
Jacobi algorithm for maximizing in dimension D>1
● x=clever initialization, if possible
● While ( x' – x > epsilon )
– x'=current x
– For each parameter x(i), optimize it
● One iteration of robust search● But don't decrease the interval if optimum = close to current bounds
Jacobi = great when the objective function
– can be restricted to 1 parameter
– and then be much faster
Possible use
● Computing student's abilities, given item parameters
● Computing item parameters, given student abilities
● Computing both item parameters and student abilities (need plenty of data)
Priors
● How to know P(Θ) ?● Keep in mind that difficulties and abilities are
translation invariant– ==> so you need a reference
– ==> possibly reference = average Θ = 0
● If you have a big database and trust your model (3PL ?), you can use Jacobi+MAP.
What if you don't like Jacobi's result ?
● Too slow ? (initialization, epsilon larger, better 1D algorithm, better implementation...)
● Epsilon too large ?
● Maybe you use Map whereas you want Bpe ?==> If you get convergence and don't like the result, it's not because of Jacobi, it's because of the criterion.
● Maybe not enough data ?
Initializing IRT parameters ?
● Roughy approximations for IRT parameters :– Abilities (Θ)
– Item parameters (a,b,c in 3PL models)
● Priors can be very convenient for that.
Find Θ with quantiles !1. Rank students per performance.
Find Θ with quantiles !2. Cumulative distribution
ABILITIES
Find Θ with quantiles !3. Projections
Mediumstudent
BestN/(N+1)
Worst1/(N+1)
ABILITIES
Find Θ with quantiles !3. Projections
Mediumstudent
BestN/(N+1)
Worst1/(N+1)
ABILITIES
Equation version for approximating abilities Θ
if you have a prior (e.g. Gaussian), then a simple solution : – Rank students per score on the test
– For student i over N, Θ initialized at the prior's quantile 1 – i/(N+1)
E.g. With Gaussian prior mu, sigma,
then ability(i)=mu+sigma*norminv(1i/(N+1))
With norminv e.g. as in http://www.wilmott.com/messageview.cfm?catid=10&threadid=38771
Equation version for approximating item parameters
Much harder !
There are formulas based on correlation. It's a very rough approximation.
How to estimate b if c=0 ?
Approximating item parameters
Much harder !
There are formulas based on correlation. It's a very rough approximation.
How to estimate b=difficulty if c=0 ?
Simple solution :– Assume a=1 (discrimination)
– Use the curve, or approximate
b = 4.8 x (1/2  proba(success))
– If you know students' abilities, it's much easier
And for difficulty of items ?Use curve or approximation...
Codes
● IRT in R : there are packages, it's free, and R is a widely supported language for statistics.
● IRT in Octave : we started our implementation, but still very preliminary :– No missing data (the main strength of IRT) ==>
though this would be easy
– No userfriendly interface to data
● Others ? I did not check● ==> Crossvalidation for comparing ?
How to get the percentile from the ability
● percentile is normcdf( (theta*mu)/sigma).(some languages have normcdf included)
● Slow/precise implementation of normcdf: http://stackoverflow.com/questions/2328258/cumulativenormaldistributionfunctionincc
● Fast implementation of normcdf: http://finance.bi.no/~bernt/gcc_prog/recipes/recipes/node23.html
● Maybe fast Exp, if you want to save up time :)