Financial Econometrics

Material : solutionYear : 2008Class : 5

Teacher(s) : Zacharias Psaradakis, Marian Vavra

Example 1.1: Let X1, ..., Xn be a random sample from the exponential distribution withdensity function

f(x) = (1/) exp{x/}, x > 0, > 0,

(a) Derive the MLE and .

(b) Show that is consistent for .

(c) Is a fully efficient estimator?

(d) Find the MLE of 1/.

(e) What is the asymptotic distribution of ?

Solution 1.1: When estimating unknown parameters by the maximum likelihood [ML], wehave to specify the likelihood function, which is related to a probability density. Since wehave a sample of independent observations (this is a very important assumption for the ML)from the exponential density, the likelihood function is a product of individual densities

L() =n

i=1

f(xi). (1)

We sometimes use the log-likelihood function l() log L() instead of the original likelihoodfunction L(). This is very useful especially if the density function f(x) is a memeber of theexponential family of distributions. So, the log-likelihood function for a given exponentialdensity f(x) is defined as

l() log L() =n

i=1

( log xi

) = n log n

i=1 xi

. (2)

Although the ML approach is very useful due to some properties of estimated parameters:asymptotic unbiasedness, consistency, efficiency, and asymptotic normality, it is worth notingthat all these results are conditional on a correct specification of density function (or atleast a correct class of the density function). That means, if we misspecify the density, thenwe can simply get biased and inconsistent estimates of parameters! Moreover, compared tothe OLS, there is no guarantee that likelihood equations have a closed-form solution. If this isthe case, then we have to apply some iterative optimization procedures, see Green (2005),Appendix E.

1

mailto:z.psaradakis@bbk.ac.ukmailto:marian.vavra@gmail.com

Financial Econometrics

(a) The MLE of is derived from the FOC

l()

= n

+

ni=1 xi2

= 0, (3)

which gives the solution in the form: = 1n

ni=1 xi = x, where x denotes a sample

average.

(b) An estimator is said to be consistent if plim() = . In this case, the solution is trivial

plim() = plim(x) = plim(1

n

ni=1

xi) = E(xi) = , (4)

since we know that E(Xi) = provided that xi is drawn from an exponential distribution.

(c) An estimator is said to be efficient if its variance achieves Rao-Cramer lower boundderived from Fishers information matrix. So, the variance of is given by

V() = V(x) = V(1

n

ni=1

xi) =1

n2V(

ni=1

xi) =V(xi)

n=

2

n. (5)

The asymptotic variance of modified estimator is defined as

Va() = [I()]1 =

2

n, (6)

where I() is Fishers information matrix given by

I() = E(

2l()

2

)= E

(

2

n

)=

2

n. (7)

Since V() = Va(), the ML estimator is an efficient estimator of .

(d) Since we already know that = x, then we immediately also know that 1/ = 1/x.

(e) We know that the MLE leads to asymptotically normally distributed parameters, a simplesketch of the proof was presented in the class and a rigorous proof can be found in Green(2005), Ch. 17. It holds in our case that

n

(

)d N(0, 2), (8)

where since Va() = 2/n, then we know that Va(

n) = nVa() =

2 is the modifiedasymptotic variance (based on inverse of Fishers information matrix, see sub-questionc) above.

2

Financial Econometrics

Note: You are strongly advised to re-specify the original model using another density func-tion, e.g. Poisson density, and proceed the all calculations again.

Example 1.2: Consider the regression model

y = X + u,

where it is assumed that: (i) u N(0, 2); (ii) X is stochastic with full column rank; (iii)X and u are independently distributed; (iv) the distribution of the regressors does not dependon the parameters and 2.

(a) Derive the MLE = (, 2) of = (, 2).

(b) What is the limiting distribution of

n( ) as n ?

(c) Prove that is fully efficient. Are and 2 asymptotically efficient?

Solution 1.2: In the case of regression models, we can use a small trick for specification ofthe likelihood function. We know that likelihood function is in fact a joint density function,which is defined as

f(y,X; ) = g(y|X; )h(X; ), (9)

where g(y|X; ) denotes a conditional density, and h(X; ) is a density function of explana-tory variables. What is important is the independence of parameter vector , influencing theconditional density, from , parameters of the density of explanatory (exogenous) variables.Therefore, maximizing f(y,X; ) over is equivalent to maximizing g(y|X; ) over . So,the likelihood function is specified as the conditional density function

L(|X) = g(y|X; ). (10)

We usually assume that errors in regression models are independent each other and normallydistributed. In such a case, the likelihood function takes the form as follows

L(|X) = g(y|X; ) =n

t=1

g(yt|xt; ) =n

t=1

122

exp

{(yt x

t)

2

22

}, (11)

where = (, 2) denotes a vector of unknown parameters. Since the normal distributionis a member of the exponential family, then it is very useful to work with the log-likelihoodfunction in the form as

l(|X) = n

t=1

log 22

2 (yt x

)2

22= n

2log 22 1

2

nt=1

u2t2

= n2

log 22 12

uu

2.

(12)Sometimes we use l() instead of l(|X), only for simplicity of notation.

3

Financial Econometrics

(a) The MLE of is derived from the FOCs, which take the form as follows

l(|X)

=Xy XX

2= 0, (13)

which leads to the closed-form solution in the form: = (XX)1Xy, which is equivalentto the OLS estimator.

l(|X)2

= n22

+uu

24= 0, (14)

which leads to the solution: 2 = uu/n.

(b) We know that if the density is correctly specified, then the MLE is asymptotically nor-mally distributed. A rigorous proof can be found in Green (2005), Ch.17. It meansthat

n( ) d N(0, nV()),

where V() is asymptotic variance of derived from the Fishers information matrix

V() = [I()]1 =

[E

(2l(|X)

)]1=

E

2l(|X)

2l(|X)2

2l(|X)2

2l(|X)22

1

(15)

=

[12

E(XX) 00 2n

4

]1.

(c) An estimator is said to be efficient if V() achieves the Rao-Cramer lower bound. In ourcase, we know that the MLE of is equivalent to the OLS estimator. The variance of is given by

V() = E[( )( )] = 2[E(XX)]1, (16)

which is equivalent to the first element of the asymptotic variance matrix V(), see pre-vious question. Therefore, the MLE is efficient estimator.

An estimator is said to be asymptotically efficient if it is consistent, asymptoticallynormally distributed, and achieves the Rao-Cramer lower bound. Due to assumption iii)in the setup of the question we can conclude that the MLE of is consistent (see one ofthe previous classes for a derivation). From the previous section we already know thatthe MLE of is also asymptotically normally distributed (see question 2b)), and fromprevious paragraph we also know that the MLE of achieves Rao-Cramer lower boundfor variance. Therefore, the MLE of is also asymptotically efficient.

Note: You should understand why distribution of

n( ) is sometimes preferred to dis-tribution of !

4

Financial Econometrics

Example 1.3: Suppose X1, ..., Xn is a random sample from a distribution with density func-tion

f(x) =x

exp

{x2

2

}, x 0, > 0.

In order to answer questions below we need to know the following two moments: E(X2i ) = 2,and E(X4i ) = 8

2.

(a) Obtain the MLE of and show that it is unbiased.

(b) Is the MLE of fully efficient?

Solution 1.3: A specified density is again a member of the exponential family of distri-butions. So, it will be useful to base our calculations on log-likelihood density l(). Thelog-likelihood function takes the form as follows

l() log L() = n log +n

i=1

log xi 1

2

ni=1

x2i . (17)

(a) The MLE of is given by the FOC

l()

= n

+

1

2

ni=1

x2i = 0, (18)

which gives: = 12n

ni=1 x

2i . An estimator is said to be unbiased if E() = . The

expected value of the MLE of is given by

E() = E

(ni=1 x

2i

2n

)=

E(n

i=1 x2i )

2n=

ni=1 E(x

2i )

2n=

E(x2i )

2= . (19)

So, the MLE results in unbiased estimate of .

(b) An estimator is said to be efficient if variance of estimated parameter V() achieves theRao-Cramer lower bound. The estimated variance is as follows

V() = V

(ni=1 x

2i

2n

)=

V(n

i=1 x2i )

4n2=

ni=1 V(x

2i )

4n2, (20a)

=V(x2i )

4n=

E(x4i ) E2(x2i )4n

=82 42

4n, (20b)

=2

n. (20c)

And the asymptotic variance is given by

V() = [I()]1 =

[E

(2l()

2

)]1=

[E

(n

2

ni=1 x

2i

3

)]1, (21a)

=

[ n

2+

2n

3

]1=

[ n2

]1=

2

n. (21b)

It means that MLE of is efficient.

5

Financial Econometrics

6