I discovered this while implementing various type of fit and tying to force the minimumfor each type of fits to 0 to avoid float precision problem. It shows how unbinned likelihoodfit, binned likelihood fit, extended likelihood fit using poisson probability and χ2 fit are allrelated.I found this to be quite interesting especially the link between binned likelihood fitand extended likelihood fit using poisson probability. First, let define:

P (x; ~θ) = Normalized PDF given parameters ~θ (1)

andEP (x; ~θ,E) = Extended PDF given parameters ~θ,E = E × P (x; ~θ) (2)

where E is the extended parameter. Without loss of generality this E will already factorin our choice of bin width which we will use in histogram-ing our data. Think of this asbin width of one in some unit. So that I don’t have binwidth factor floating around andmake the equation looks prettier.

For our data,~x = {x0, x1, . . . , xi, . . . , xN} = our data points (3)

and~h = {h0, x1, . . . , hj , . . . , hM} = Histogram of x (4)

Here, in this note I’ll use i for index for ~x and j for index of ~h.N is number of events wehave and M is number of bins in h.

1 Unbinned likelihood fit

Unbinned likelihood fit is finding ~θ that minimize

UBL =∑xi

− logP (xi; ~θ) (5)

2 Binned likelihood fit

Binned likelihood fit is done by first histograming ~x to obtain ~h. Then for each bin of~h, we find the representative value of x in that bin, x̄, then calculate the likelihood of x̄then multiply by number of events in that bin. Mean value theorem guarantee that suchx̄ exists to preserve the integral.

Going from Unbinned likelihood to Binned likelihood is just making all x that’s betweenx̄± binwidth to have value of x̄

This is just minimizing the following expression

BL =∑j

−hj logP (x̄j ; ~θ) (6)

1

Before we go on to next one, let a a constant from this expression. This will not change thecentral value of parameters nor the estimated uncertainty on the parameters. The constantof choice is hj log

hjN . This is a constant because given data ~h is fixed. The rationale of this

expression is thathjN ≈ P (x̄j ; ~θ). The subtraction/addition will then make this quantity

minimum to be near zero.

BLZ =∑j

−hj logP (x̄j ; ~θ) + hj loghjN

=∑j

−hj logP (x̄j , ~θ)

hjN

(7)

Now if we are using extended likelihood P = EP/E. With my excellent choice of confusingnotation this becomes

BLZ =∑j

−hj log

(E × P (x̄j , ~θ)

hj× N

E

)(8)

Expanding the log

BLZ =∑j

−hj log

(EP (x̄j , ~θ)

hj

)−∑j

hj logN

E(9)

For the last sum N and E doesn’t depend on j and∑

j hj = N

BLZ =∑j

−hj log

(EP (x̄j , ~θ)

hj

)+N log

E

N(10)

Now, using the following trick

BLZ =∑j

−hj log

(EP (x̄j , ~θ)

hj

)+N log

(E −N +N

N

)(11)

This becomes

BLZ =∑j

−hj log

(EP (x̄j , ~θ)

hj

)+N log

(1− N − E

N

)(12)

For N ≈ E using log 1− ε expansion

BLZ =∑j

−hj log

(EP (x̄j , ~θ)

hj

)− (N − E)− (N − E)2

2N+ . . . (13)

Let us stop here and go to the next one.

2

3 Extended Likelihood Fit Using Poisson Statistics

Extended likelihood fit using poisson statistics is calculating the probability of each binusing Poisson distribution asking the question: if we expect EP (x̄j , ~θ) for number of eventsin bin j what’s the probability that we found hj events in that bin. That is minimizingthe following quantity

BLP =∑j

− log(

Poisson(hj ;λ = EP (x̄j , ~θ)))

(14)

Expanding Poisson and log

BLP =∑j

−hj logEP (x̄j , ~θ) + EP (x̄j , ~θ) + log(Γ(1 + hj)) (15)

Now we do the same trick subtracting a constant off so that the minimum is ≈ 0. Thistime our constant of choice is

∑j Poisson(hj , hj). This yields

BLP =∑j

−hj logEP (x̄j , ~θ) +EP (x̄j , ~θ) + log(Γ(1 +hi)) +hj log hj −hj − log(Γ(1 +hj))

(16)Simplifying the expression gives

BLP =∑j

−hj logEP (x̄j , ~θ)

hj+∑j

EP (x̄j , ~θ)−∑j

hj (17)

The second term becomes E and the last term becomes N .

BLP =∑j

−hj logEP (x̄j , ~θ)

hj− (N − E) (18)

which is just 13 ignoring (N−E)2

N and higher order terms.Now we can use the same trick on the first log

BLP =∑j

−hj logEP (x̄j , ~θ)− hj + hj

hj− (N − E) (19)

that is

BLP =∑j

−hj log

(1− hj − EP (x̄j , ~θ)

hj

)− (N − E) (20)

Taylor expand log 1− ε gives

BLP =∑j

((((((((

((hj − EP (x̄j , ~θ)) +

(hj − EP (x̄j , ~θ))2

2hj+ Higher order−���

��(N − E) (21)

Which is just χ2/2

3

4 conclusion

So, by the order of closeness to the our holy grail unbinned likelihood fit we got.

Unbinned Likelihood

Binned Likelihood

Poisson Binned Likelihood

χ2

Lumping Values

Ignoring (N−E)2

N and higher order

Ignoring∑

j(hj−EP (x̄j ,~θ))

3

h2jand higher order

5 Bonus: Variance on extended parameter E

Let us look again at 13 this time we also expand the first log like we did in 21 going frompoisson to χ2. This time I’m putting binwidth b back in for clarity. We have

BL =∑j

(hj − EP (x̄j , ~θ)b)2

2hj− (N − E)2

2N+ H.O. (22)

4

Taking the second derivative w.r.t E we have (This is how migrad gives error on E)

d2

dE2BL

∣∣∣∣E=N

≈ var(E)−1BL =

∑j

P (x̄j , ~θ)2b2

hj− 1

N(23)

Note that in Binned Poisson case we do not have the term (N−E)2

2N . Thus, the 1N is gone.

That isd2

dE2BLP

∣∣∣∣E=N

≈ var(E)−1BLP =

∑j

P (x̄j , ~θ)2b2

hj(24)

At the minimum P (x̄j , ~θ)b ≈ hj/N the two above equation become and using∑

j hj = N

d2

dE2BL

∣∣∣∣E=N

≈ var(E)−1BL = 0 (25)

andd2

dE2BLP

∣∣∣∣E=N

≈ var(E)−1BLP =

1

N(26)

Thus, the variance arevar(E)BL =!?!!!?! (27)

andvar(E)BLP = N (28)

Thus, the lesson is . . . do not trust migrad error matrix for Extended Parameter?

5