1

Schaum’s OutlinePROBABILTY and STATISTICS

Chapter 6 ESTIMATION THEORY

Presented by Professor Carol Dahl

Examples from D. Salvitti

J. Mazumdar C. Valencia

2

Outline Chapter 6Trader in energy stocks

random variable Y = value of share

want estimates µy, σY

Y = ß0 + ß1X+ want estimates Ŷ, b0, b1

Properties of estimatorsunbiased estimatesefficient estimates

3

Outline Chapter 6Types of estimators

Point estimates µ = 7

Interval estimatesµ = 7+/-2confidence interval

4

Outline Chapter 6

Population parameters and confidence intervals Means

Large sample sizes

Small sample sizes

Proportions

Differences and Sums

Variances

Variances ratios

5

Properties of Estimators - Unbiased

Unbiased Estimator of Population Parameter

estimator expected value = to population parameter

)ˆ(E

6

Unbiased Estimates

Population Parameters:

Sample Parameters:

are unbiased estimates

Expected value of standard deviation not unbiased

22 σ)SE( μ)XE(

2σ ; μ

2S , X

2S ; X

σ)SΕ(

7

Properties of Estimators - Efficient

Efficient Estimator –

if distributions of two statistics same

more efficient estimator = smaller variance

efficient = smallest variance of all unbiased estimators

8

Unbiased and Efficient Estimates

Target

Estimates which are efficient and unbiased

Not always possible

often us biased and inefficient

easy to obtain

9

Types of Estimates for Population Parameter

Point Estimate

single number

Interval Estimate

between two numbers.

10Estimates of Mean – Known Variance

Large Sample or Finite with Replacement

X = value of share

sample mean is $32

volatility is known σ2 = $4.00

confidence interval for share value

Need

estimator for mean

need statistic with

mean of population

estimator

11

Estimates of Mean- Sampling Statistic

P(-1.96 < <1.96) = 95%

)1,0(N

n

X

n

X

2.5%

12

Confidence Interval for Mean

P(-1.96 < <1.96) = 95%

P(-1.96 < <1.96 ) = 95%

P(-1.96 -X < -µ <1.96 - X ) = 95%

Change direction of inequality

P(+1.96 +X > µ > -1.96 + X ) = 95%

n

X

Xn

n

n

n

n

n

13

Confidence Interval for Mean

P(+1.96 +X > µ > -1.96 + X ) = 95%

Rearrange

P(X - 1.96 < µ <X + 1.96 ) = 95%

Plug in sample values and drop probabilities

X = value of share, sample = 64

sample mean is $32

volatility is σ2 = $4

{32 – 1.96*2/64, 32 + 1.96*2/64} = {31.51,32.49}

n

n

n

n

14

Estimates for Mean for Normal

Take a sample

point estimate

compute sample mean

interval estimate – 0.95 (95%+) = (1 - 0.05)

X +/-1.96

X +/-Zc

(Z<Zc) = 0.975 = (1 – 0.05/2)

95% of intervals contain

5% of intervals do not contain

n

n

15

Estimates for Mean for Normal

interval estimate – 0.95 (95%+) = (1 - 0.05)

X +/-Zc

(Z<Zc) = 0.975 = (1 – 0.05/2)

interval estimate – (1-) %

X +/-Zc

(Z<Zc) = (1 – /2)

% of intervals don’t contain

(1- )% of intervals do contain

n

(Z<Zc) = 0.975 = (1 – 0.05/2)

n

16

Common values for corresponding to various confidence levels used in practice are:

Confidence level 99.73% 99% 98% 96% 95.45% 95% 90% 68.27% 50%3.00 2.58 2.33 2.05 2.00 1.96 1.645 1.00 0.6745

Confidence Interval Estimatesof Population Parameters

17

Functions in EXCEL

Menu Click on Insert Function or

=confidence(,stdev,n)

=confidence(0.05,2,64)= 0.49

X+/-confidence(0.05,2,64)

=normsinv(1-/2) gives Zc value

X+/-normsinv(1-/2)

32 +/- 1.96*2/64

Confidence Interval Estimatesof Population Parameters

n

18

Confidence interval Confidence level

)%-(1 @ 1-Nn-N

nσZX C

Confidence Intervals for MeansFinite Population (N) no Replacement

19

Evaluate density of oil in new reservoir

81 samples of oil (n)

from population of 500 different wells

samples density average is 29°API

standard deviation is known to be 9 °API

= 0.05

Example: Finite Population without replacement

20

X = 29 , N= 500, n = 81 , σ = 9 , = 0.05

Zc = 1.96

1-Nn-N

nZX C

Confidence Intervals for MeansFinite Population (N) no Replacement

Known Variance

1.8029 1500

81 - 50081

929μ So

95% @ 80.31μ27.20 or

21

= N(0.1)

df

2/df= tdf

t-Distribution

But don’t know Variance

n

X

1n

s)1n(2

2

=

22Confidence Intervals of Means

t- distribution

n

X

1n

s)1n(2

2

=

n

X

2

2

2

22

2

s

)1n(1s)1n(

11n

s)1n(

n

X

n

X

= =n

sX

=

23Confidence Intervals of Means

Normal compared to t- distribution

n

X

ns

X

Normal t distribution

X +/-Zc n

ns

X +/-tc

24

Example:

Eight independent measurements diameter of drill bit

3.236, 3.223, 3.242, 3.244, 3.228, 3.253, 3.253, 3.230

99% confidence interval for diameter of drill bit

Confidence Interval Unknown Variance

ns

X +/-tc

25Confidence Intervals for Means

Unknown Variance

X = ΣXi/n

3.236+3.223+3.242+3.244+3.228+3.253+3.253+3.230

8X = 3.239

ŝ2 = Σ(Xi - X) = (3.236- X)2 + . . .(3.230 - X)2

(n-1) (8-1)

ŝ = 0.0113

ns

X +/-tc

26

X = 3.239, n = 8, ŝ = 0.0113, =0.01,

1- /2=0.995

From the t-table with 7 degrees of freedom, we

find tc= t7,0.995=3.50

Confidence Intervals for MeansUnknown Variance

ns

X +/-tc

.005%

tc-tc

Find tc from Table of Excel

1-/2=.975

27Confidence Intervals for Means

Unknown Variance

3.499483

/2= 0.005%

tc-tc

Depends on Table

1-/2=.975

GHJ /2 = 0.005 tc = 2. 499

Schaums 1- /2 = 0.995 tc = 2.35

Excel =tinv(0.01,7) = 3.4994833.499483

28

X = 3.239, n = 8, ŝ = 0.0113, =0.01,

8

0.011350.33.239diam So

99% @ 3.253diam3.225 or

Confidence Intervals for MeansUnknown Variance

ns

X +/-tc

29

600 engineers surveyed

250 in favor of drilling a second exploratory well

95% confidence interval for

proportion in favor of drilling the second well

Approximate by Normal in large samples

Solution: n=600, X=250 (successes), = 0.05

zc = 1.96 and %7.414167.0

600250

P

Example

Confidence Intervals for Proportions

30

600 engineers surveyed

250 in favor of drilling a second exploratory well.

95% confidence interval for

proportion in favor of drilling the second well

Approximate by Normal in large samples

Solution: n=600, X=250 (successes), = 0.05

zc = 1.96 and %7.414167.0600250

P

Example

Confidence Intervals for Proportions

31

Confidence Intervals for Proportions

n

p)-p(1zp c

sampling from large population

or finite one with replacement

32Confidence Intervals Differences and Sums

Known Variances

Samples are independent

2 21 2

1 2

1 2cX X z

n n

33

Example

sample of 200 steel milling balls

average life of 350 days - standard deviation 25 days

new model strengthened with molybdenum

sample of 150 steel balls

average life of 250 days - standard deviation 50 days

samples independent

Find 95% confidence interval for difference μ1-μ2

Confidence Interval for Differences and Sums – Known Variance

34

Example

2 2

1 2

25 50Then μ μ 350 250 1.96 or 100 8.72

200 150

Solution: X1=350, σ1=25, n1=200, X2=250, σ2=50, n2=150

Confidence Intervals for Differences and Sums

2 21 2

1 2

1 2cX X z

n n

35

Where: P1, P2 two sample proportions,

n1, n2 sizes of two samples

1 1 2 21 2

1 2

1- 1-c

p p p pP P z

n n

Confidence Intervals for Differences and Sums – Large Samples

36

0.75 1-0.75 0.33 1-0.33So 0.75 0.33 1.96

200 300

random samples

200 drilled holes in mine 1, 150 found minerals

300 drilled holes in mine 2, 100 found minerals c

Construct 95% confidence interval difference in proportions

Solution: P1=150/200=0.75, n1=200, P2=100/300=0.33,n2=300

Example

With 95% of confidence the difference of proportions {0.42, 0.08}

Confidence Intervals for Differences and Sums

37

0.75 1-0.75 0.33 1-0.33So 0.75 0.33 1.96

200 300

Solution: P1=150/200=0.75, n1=200,

P2=100/300=0.33, n2=300

Example

95% of confidence the difference of proportions

[0.08, 0.42]

Confidence Intervals Differences and Sums

38

Confidence Intervals for Variances

Need statistic with

population parameter 2

estimate for population parameter ŝ2

its distribution - 2

39

Confidence Intervals for Variances

1)

s)1n((P 2

above2/2

22

below2/

has a chi-squared distribution

n-1 degrees of freedom.

Find interval such that σ lies in the interval for

95% of samples

95% confidence interval

22

2 2

ˆ1n SnS

40

Confidence Intervals for Variances

1)

s)1n((P 2

above2/2

22

below2/

Rearrange

1)s)1n(s)1n(

(P2

below2/

22

2

above2/

2

Take square root if want confidence interval for

standard deviation

41Confidence Intervals for Variances and

Standard Deviations

2

below2/

2

2

above2/

2 s)1n(,

s)1n(

Drop probabilities when substitute in sample values

1 - confidence interval for variance

2

below2/

2

2

above2/

2 s)1n(,

s)1n(

1 - confidence interval for standard deviation

42

Variance of amount of copper reserves

16 estimates chosen at random

ŝ2 = 2.4 thousand million tons

Find 99% confidence interval variance

Solution: ŝ2=2.4, n=16,

degrees of freedom = 16-1= 15

ExampleConfidence Intervals for Variance

2

below2/

2

2

above2/

2 s)1n(s)1n(

43How to get 2

Critical Values

/2/2

Not symmetric

2 lower 2 upper

44How to get 2

Critical Values

/2/2

1-/2

GHJ area above 20.995, 20.005 4.60092, 32.8013

Schaums area below 20.005, 20.995 4.60, 32.8

Excel = chiinv(0.995,15) = 4.60091559877155

Excel = chiinv(0.005,15) = 32.8013206461633

Not symmetric

1-/2

45

99% confidence interval variance of reserves

Solution: ŝ=2.4 (n-1)=15

2lower = 4.60, 2upper = 32.8

Example

Confidence Intervals for Variances and Standard Deviations

2

below2/

2

2

above2/

2 s)1n(,

s)1n(

6.44.2*15

,8.32

4.2*15

46

Two independent random samples

size m and n

population variances

estimated variances ŝ21, ŝ2

2

interested in whether variances are the same

21/ 2

2

2 21 2,

Confidence Intervals for Ratio of Variances

47

Need statistic with

population parameter 21/ 2

2

estimate for population parameter ŝ21/ ŝ2

2

its distribution - F

Confidence Intervals for Ratio of Variances

48

F-Distribution

df1

df2

2df,1df2df

1df F2

2

49

F-Distribution

)1n,1n(2

1

2

2

2

2

2

1

2

2

2

22

1

2

2

11

2

21

2

21

2

1

Fss

)1n(

s)1n()1n(

s)1n(

df

df

50

Need statistic with

population parameter 21/ 2

2

estimate for population parameter ŝ21/ ŝ2

2

its distribution - F

Confidence Intervals for Ratio of Variances

)1n,1n(2

1

2

2

2

2

2

1

21F

ss

51

Confidence Intervals for Ratio of Variances

1)F

ss

F(P /2 above)1n,1n(2

1

2

2

2

2

2

1/2 below)1n,1n( 2121

Rearrange

1)F

1ss

F1

ss

(P/2 above)1n,1n(

2

2

2

12

2

2

1

/2 below)1n,1n(

2

2

2

1

2121

52

Confidence Intervals for Ratio of Variances

1)F

1ss

F1

ss

(P/2 below)1n,1n(

2

2

2

12

2

2

1

/2 above)1n,1n(

2

2

2

1

2121

Put smallest first, largest second

/2 below)1n,1n(

2

2

2

1

/2 above)1n,1n(

2

2

2

1

2121F

1ss

,F

1ss

When substitute in values drop probabilities

1- confidence interval for 21/ 2

2

53

Example

Two nickel ore samples

of sizes 16 and 10

unbiased estimates of variances 24 and 18

Find 90% confidence limits for ratio of variances

Solution: ŝ21 = 24, n1 = 16, ŝ2

2 = 18, n2 = 10,

Confidence Intervals for Variances

/2 below)1n,1n(

2

2

2

1

/2 above)1n,1n(

2

2

2

1

2121F

1ss

,F

1ss

54

Confidence Intervals for Ratio of Variances

/2/2

F upperF lower

GHJ area above F0.95,15,9, F0.05,15,9 ? 3.01 Schaums area below F0.05,15,9, F0.95,15,9 ? 3.01Area aboveExcel = Finv(0.95,15,9) = 0.386454546279388Excel = Finv(0.05,15,9) = 3.00610197251669

Tablesdf1d

f2

55

Confidence Intervals for Ratio of Variances

/2/2

F upperF lower

GHJ area above F0.95,15,9

P(F15,9>Fc) = 0.95

P(1/F15,9<1/Fc) = 0.95

But 1/F15,9 = F9,15

P(F9,15<1/Fc) = 0.95

P(F9,15<1/Fc) = 0.05

1/Fc = 2.59 Fc = 0.3861

56

Example

Two nickel ore samples

Solution: ŝ21 = 24, n1 = 16, ŝ2

2 = 18, n2 = 10,

Confidence Intervals for Variances

/2 below)1n,1n(

2

2

2

1

/2 above)1n,1n(

2

2

2

1

2121F

1ss

,F

1ss

]45.3,44.0[3865.01

*1824

,0061.3*1

1824

57

Maximum Likelihood Estimates

Point Estimates

x is population with density function f(x,)

if know - know the density function

2 where = degrees of freedom

Poisson λxe-λ/x! = λ (the mean)

If sample independently from f n times

x1, x2, . . .xn

a sample

if consider all possible samples of n

a sampling distribution

58

Maximum Likelihood Estimates

If sample independently from f n times

x1, x2, . . .xn

a sample

if consider all possible samples of n

a sampling distribution

called likelihood function

1 2 2, , ,L f x f x f x

59

which maximizes the likelihood function

Derivative of L with respect to and setting it to 0

Solve for

Usually easier to take logs first

log(L) = log(f(x1,) + log(f(x2,)+ . . .+ log(f(xn,)

Maximum Likelihood Estimates

1 2 2, , ,L f x f x f x

60

log(L) = log(f(x1,) + log(f(x2,) +. . .+ log(f(xn,)

Solution of this equation is maximum likelihood estimator

work out example 6.25

work out example 6.26

1

1

, ,1 1+ + 0

, ,n

n

f x f x

f x f x

Maximum Likelihood Estimates

61

Sum Up Chapter 6

Y = ß0 + ß1X

Ŷ, b0, b1

Properties of estimatorsunbiased estimatesefficient estimates

Types of estimatorsPoint estimates Interval estimates

62

Sum Up Chapter 6

Y- µY, Y, Y, ŝ2

In 590-690

Y = ß0 + ß1X

Ŷ, b0, b1

Properties of estimators

unbiased estimates

efficient estimates

Types of estimators

Point estimates

Interval estimates

63

Sum Up Chapter 6Need statistic with

population parameter estimate for population parameterits distribution

64

Sum Up Chapter 6

Population parameters and confidence intervals

Mean – Normal

Know variance and population normal

Large sample size can use estimated variance

n

X

nZX2

c

65

Sum Up Chapter 6

Proportions

large sample approximate by normal

Differences of means (known variance)

n

p)-p(1zp c

2 21 2

1 2

1 2cX X z

n n

66

Sum Up Chapter 6

Mean

population normal - unknown variance

ns

X

nstX

2

1n

67

Sum Up Chapter 6

Variances

2

2s)1n(

2

below2/

2

2

above2/

2 s)1n(,

s)1n(

68

Sum Up Chapter 6

Variances ratios

)1n,1n(2

1

2

2

2

2

2

1

21F

ss

/2 below)1n,1n(

2

2

2

1

/2 above)1n,1n(

2

2

2

1

2121F

1ss

,F

1ss

69

Sum Up Chapter 6

Maximum Likelihood Estimators

Pick which maximizes the function

1 2 2, , ,L f x f x f x

70

End of Chapter 6!