Solution to Rao Crammer Bound
11
Uniformly Minimum Variance Unbiased Estimator (UMVUE) a. UMVUE (or Best Unbiased Estimator): - Under the class of unbiased estimators ˆ ˆ { : [] } E θ θ θ = , if for any * ˆ θ we have * ˆ [] [ ] Var Var θ θ ˆ θ θ ≤ for all θ ∈ Ω , then ˆ θ is the UMVUE. - Note: ˆ θ is an unbiased estimator with uniformly minimum variance . b. Ways to find UMVUE if exists: - Considering lower bound of variance. - Using sufficiency and completeness. c. Cramer-Rao Inequality: - Suppose that the regularity conditions hold, e.g. the range of the pdf { : ( | ) 0} x f x θ > is independent of θ . - If is any unbiased estimator of 1 ( ) ( ,..., ) n W WX X = X () τ θ , then 2 2 [ ( )] ( ( )) [( log ( | )) ] Var W E f θ θ τθ θ θ θ ∂ ∂ ≥ ∂ ∂ X X . - Define the information number as 2 () [( log ( | )) ] n I E f θ θ θ θ ∂ = ∂ X . Note: () n I θ depends only on ( | ) f X θ , not on () τ θ . As () n I θ gets bigger and then we have more information about θ , we have the smaller bound on the variance of the best unbiased estimator. - If 1 ,..., n X X are iid, 2 1 () [( log ( | )) ] () n I nE f X nI θ θ θ θ θ ∂ = ⋅ = ∂ , Further, in general case, 2 2 () [ log ( | )] n I nE f X θ θ θ θ ∂ = ⋅ − ∂ . It’s easier - If an unbiased estimator attains its C-R lower bound, * ( ) W X 2 * [ ( )] ( ( )) () n Var W I θ τ θ θ θ ∂ ∂ = X is the UMVUE * ( ) W X 1
-
Upload
peter-kamfydj-ngutu -
Category
Documents
-
view
90 -
download
0
description
biased and unbiased estimators
Transcript of Solution to Rao Crammer Bound
Uniformly Minimum Variance Unbiased Estimator (UMVUE) a. UMVUE (or Best Unbiased Estimator): -
Under the class of unbiased estimators { : E[ ] = } , if for any * we have Var [ ] Var [ * ] for all , then is the UMVUE.
-
Note: is an unbiased estimator with uniformly minimum variance.
b. Ways to find UMVUE if exists: Considering lower bound of variance. Using sufficiency and completeness.
c. Cramer-Rao Inequality: Suppose that the regularity conditions hold, e.g. the range of the pdf {x : f ( x | ) > 0} is independent of . If W ( X) = W ( X 1 ,..., X n ) is any unbiased estimator of ( ) , then [ ( )]2
Var (W ( X))
E [( log f ( X | )) 2 ]
.
-
Define the information number as I n ( ) = E [(
log f ( X | )) 2 ] .
Note: I n ( ) depends only on f ( X | ) , not on ( ) . As I n ( ) gets bigger and then we have more information about , we have the smaller bound on the variance of the best unbiased estimator. If X 1 ,..., X n are iid, I n ( ) = n E [( log f ( X | )) 2 ] = nI1 ( ) ,
Further, in general case, I n ( ) = n E [ -
2 log f ( X | )] . 2
Its easier
If an unbiased estimator W * ( X) attains its C-R lower bound, ( )]2 Var (W * ( X)) = I n ( ) [
W * ( X) is the UMVUE
1
-
Note: If the range of the pdf depends on , then the C-R lower bound is not applicable, that is, there may exist an unbiased estimator W ( X) with ( )]2 Var (W ( X)) < . I n ( ) [
-
Shortcoming: Even if the C-R lower bound is applicable, there is no guarantee that the bound is sharp, that is, the C-R lower bound is strictly smaller than the variance of any unbiased estimator, even for the UMVUE. ( )]2 * , where W * ( X) is the UMVUE. Var (W ( X)) > I n ( ) [ Use sufficiency and completeness to find the best unbiased estimator.
2
1. Let Y be a random variable having the binomial distribution, b( n, p ) . (a) Find the MLE p of p . (b) Find the Cramer-Rao lower bound. (c) Is MLE p the UMVUE ? 2. X 1 ,..., X n is a random sample from N ( ,1) .
(a) Find the method of moment estimatior(MME) of . (b) Find the Cramer-Rao lower bound. (c) Is MME the UMVUE ?3. Let Y1 ,..., Yn be a random sample from the exponential distribution with p.d.f. f ( y) = 1
e , y > 0 , > 0
y
(a) Find the MLE of . (b) Find MLE of ( ) = 2 . (c) Find the Cramer-Rao lower bound for unbiased estimator of ( ) . (d) Can you find UMVUE of ( ) ?x x2 2 2
4. Let X 1 ,..., X n be a random sample from the p.d.f. f ( x | ) =
Find the Cramer-Rao lower bound on the variance of any unbiased estimate of . 5. X 1 ,..., X n is a random sample from U (0, ) . (a) Find the MLE of . Is MLE unbiased ? If it is not, please find a function of MLE that is unbiased. (b) Find the variance of this unbiased estimator. (c) Find the Cramer-Rao lower bound. Does the variance of unbiased estimator also achieve the Cramer-Rao lower bound ? 6. Let X 1 ,..., X n be a random sample from the p.d.f. f ( x | , ) = 1
2
e
, x 0 , >0
e
( x )
,
