Power Analysis in Grant Writing

Jill Harkavy-Friedman, Ph.D.

Is there a difference?Is there a difference?

ObservedYes

ObservedNo

ExpectedYes True Positive

Type II Error

βExpected

NoType I Error

α True Negative

POWER IS KNOWLEDGEPOWER IS KNOWLEDGE

Power: How likely are you to detect an effect?

Sample Size: nHow many people will you need?

Effect Size: σ (e.g., R2)How much of a difference are you trying to detect?

Significance Level (Type I Error): αHow much of a risk are you willing to take of saying there is a difference when there none?

Type II Error: β or (1- α)

How much risk there is of saying that there is no difference when there is a difference.

Sample Size: nSample Size: n

Depends on Nature of the question & statistic approach

Group differences: m1-m2=0 Correlations: r=0 Regression: R2=0

Feasibility Economic, staffing, recruitment

Power needed

Effect Size: Effect Size: σσ

Amount of difference want to detect

Based on previous literature: average SD Based on pilot data: SD Based on size of difference

Small: d=.20Medium: d=.50Large: d=.80

Statistical difference ≠ Clinical difference

3 4 5 60.0

2.5

5.0

7.5

10.0

Count

11% 34%2.5% 34% 11% 2.5%

Normal Curve and Distribution of Sample

Means

Significance Level (Type I Error): Significance Level (Type I Error): αα

The larger the sample size the greater the power

The larger the effect size the greater the power

The larger the significance level the greater the power

Power and Sample Size Table at Power and Sample Size Table at αα 1=.05 1=.05

ES .30 .40 .50 .60

Power

.70 108 62 41 29

.75 123 70 46 33

.80 140 80 52 37

.85 163 93 61 43

.90 193 110 72 51

.95 243 138 90 63

.99 353 200 129 91

Data Analysis comes first - Data Analysis comes first - power secondpower second

Determine your hypotheses

Determine your analyses

Determine the parameters for analysis by hypothesis (i.e., power, ES, α)

Conduct power analysis

Power analysis will require:Power analysis will require:

Type of analysisSample sizeEffect sizeSignificance levelNumber of groups or factorsPlug in the numbers

ConsiderationsConsiderations

What is your question?

What type of data do you have?

What are your hypotheses?

What are your resources?

Clinical vs. statistical significance

How will you present your data?

What you would like the news headline to be?

SAGES Research Committee, August 2006 SAGES Research Committee, August 2006 RE: Suggestions to Assist with Completing RE: Suggestions to Assist with Completing a Winning Applicationa Winning Application

Power Analysis: In order to minimize the reporting of false-negative data, a

power analysis should be performed for sample size determination. Power is the capability of a study to detect a difference if the difference really exists. A type II error occurs when a true difference exists between study populations but there are insufficient numbers of subjects to detect this difference.

Any grant submitted without one of the items below will not be eligible for review.

Power analysis. Please provide the following data: alpha and beta, sample size needed in each group, what difference is expected. (Example: "A power analysis was performed with a beta of .20 and an alpha of 05. Assuming that a 10% difference exists between patient and control groups, 150 subjects will be needed in each arm. Thus the study would provide an 80% chance that a difference would be detected if one exists.")

If a power analysis in not appropriate for the submitted project, a statement should be included explaining why a power analysis is not appropriate for the study.

Consultation with a statistician is recommended. However, there are many statistical software programs available

What to do when you need What to do when you need more powermore power

Increase sample size

Reduce number of variables

Show your data graphically

Power AnalysisPower Analysis

With n=400, alpha2 = .05, and a medium effect size (.30) the power will be >.99 for analyses of variance and .96 for zero order correlations120. The power for a regression analysis that includes 11 variables (i.e. sex, ethnicity, positive symptoms, negative symptoms, aggression, impulsivity, depression, premorbid adjustment, gene marker, family history and substance abuse) with n=400, alpha=.05 and a medium effect size (R2=.10) will be greater than .90. We do not anticipate that all 11 variables will contribute significantly to the model. With 6 variables, a more likely model, the power will be > .90. For the exploratory regression analyses conducted within the group of attempters (n=200) the power will be at least .80 to detect a medium effect size (R2=.10). For correlational studies among the genetic and biochemical measures (approximate n=100) the power will be .80 for a medium effect size (R2=.10) and .95 for a larger medium effect size (R2=.25). The increased sample size will now provide the power necessary to consider attempters and nonattempters separately. Analyses examining single attempters, multiple attempters and nonattempters (approximate sample sizes: 120, 80, 200) will still maintain adequate power (power>.70).

Grant application by Jill M. Harkavy-Friedman, PhD

Free Power and Sample Size Free Power and Sample Size CalculationCalculation

Cohen J. Statistical Power Analysis for the Behavioral Sciences (2nd edition). Hillsdale, New Jersey: Lawrence Erlbaum Associates, Publishers, 1988 http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSampleSi

ze

G*Power 2 http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/

(limited)

G*3 http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/