Download - Power and Sample Size

Transcript
Page 1: Power  and Sample Size

Power and Sample Size

Boulder 2006

Benjamin Neale

Page 2: Power  and Sample Size

To Be Accomplished

• Introduce concept of power via correlation coefficient (ρ) example

• Identify relevant factors contributing to power

• Practical:• Empirical power analysis for univariate twin

model (simulation)• How to use mx for power

Page 3: Power  and Sample Size

Simple example

Investigate the linear relationship (between two random variables X and Y: =0 vs. 0 (correlation coefficient).

• draw a sample, measure X,Y

• calculate the measure of association (Pearson product moment corr. coeff.)• test whether 0.

Page 4: Power  and Sample Size

How to Test 0• assumed the data are normally

distributed

• defined a null-hypothesis ( = 0)

• chosen level (usually .05)

• utilized the (null) distribution of the test statistic associated with =0

• t= [(N-2)/(1-2)]

Page 5: Power  and Sample Size

How to Test 0• Sample N=40

• r=.303, t=1.867, df=38, p=.06 α=.05

• As p > α, we fail to reject = 0

have we drawn the correct conclusion?

Page 6: Power  and Sample Size

= type I error rate probability of deciding 0

(while in truth =0)

is often chosen to equal .05...why?

DOGMA

Page 7: Power  and Sample Size

N=40, r=0, nrep=1000 – central t(38), =0.05 (critical value 2.04)

-5 -4 -3 -2 -1 0 1 2 3 4 50

200

400

600

800

-5 -4 -3 -2 -1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

NREP=1000

2.5% 2.5%

central t(38)

4.6% sign.

Page 8: Power  and Sample Size

Observed non-null distribution (=.2) and null distribution

-5 -4 -3 -2 -1 0 1 2 3 4 50

20

40

60

80

100

-5 -4 -3 -2 -1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

0.5

rho=.20N=40Nrep=1000

abs(t)>2.04 in23%

null distribution t(38)

Page 9: Power  and Sample Size

In 23% of tests of =0, |t|>2.024 (=0.05), and thus draw the correct conclusion that of rejecting 0.

The probability of rejecting the null-hypothesis (=0) correctly is 1-, or the power, when a true effect exists

Page 10: Power  and Sample Size

Hypothesis Testing

• Correlation Coefficient hypotheses:– ho (null hypothesis) is ρ=0

– ha (alternative hypothesis) is ρ ≠ 0• Two-sided test, where ρ > 0 or ρ < 0 are one-sided

• Null hypothesis usually assumes no effect

• Alternative hypothesis is the idea being tested

Page 11: Power  and Sample Size

Summary of Possible Results

H-0 true H-0 false

accept H-0 1-reject H-0 1-

=type 1 error rate

=type 2 error rate

1-=statistical power

Page 12: Power  and Sample Size

Rejection of H0 Non-rejection of H0

H0 true

HA true

Nonsignificant result(1- )

Type II error at rate

Significant result(1-)

Type I error at rate

Page 13: Power  and Sample Size

Power• The probability of rejection of a

false null-hypothesis depends on: –the significance criterion ()–the sample size (N) –the effect size (Δ)

“The probability of detecting a given effect size in a population from a sample of size N, using significance criterion ”

Page 14: Power  and Sample Size

T

alpha 0.05

Sampling distribution if HA were true

Sampling distribution if H0 were true

POWER: 1 -

Standard Case

Non-centrality parameter

Page 15: Power  and Sample Size

T

alpha 0.05

Sampling distribution if HA were true

Sampling distribution if H0 were true

POWER: 1 - ↑

Increased effect size

Non-centrality parameter

Page 16: Power  and Sample Size

Impact of more conservative

T

alpha 0.01Sampling distribution if HA were true

Sampling distribution if H0 were true

POWER: 1 - ↓

Non-centrality parameter

Page 17: Power  and Sample Size

Impact of less conservative

T

alpha 0.10Sampling distribution if HA were true

Sampling distribution if H0 were true

POWER: 1 - ↑

Non-centrality parameter

Page 18: Power  and Sample Size

T

alpha 0.05

Sampling distribution if HA were true

Sampling distribution if H0 were true

POWER: 1 - ↑

Increased sample size

Non-centrality parameter

Page 19: Power  and Sample Size

Effects on Power Recap

• Larger Effect Size

• Larger Sample Size

• Alpha Level shifts <Beware the False Positive!!!>

• Type of Data:– Binary, Ordinal, Continuous

• Multivariate analysis

• Empirical significance/permutation

Page 20: Power  and Sample Size

When To Do Power Calculations?

• Generally study planning stages of study

• Occasionally with negative result

• No need if significance is achieved

• Computed to determine chances of success