Power Analysis in Grant Writing
Jill Harkavy-Friedman, Ph.D.
Is there a difference?Is there a difference?
ObservedYes
ObservedNo
ExpectedYes True Positive
Type II Error
βExpected
NoType I Error
α True Negative
POWER IS KNOWLEDGEPOWER IS KNOWLEDGE
Power: How likely are you to detect an effect?
Sample Size: nHow many people will you need?
Effect Size: σ (e.g., R2)How much of a difference are you trying to detect?
Significance Level (Type I Error): αHow much of a risk are you willing to take of saying there is a difference when there none?
Type II Error: β or (1- α)
How much risk there is of saying that there is no difference when there is a difference.
Sample Size: nSample Size: n
Depends on Nature of the question & statistic approach
Group differences: m1-m2=0 Correlations: r=0 Regression: R2=0
Feasibility Economic, staffing, recruitment
Power needed
Effect Size: Effect Size: σσ
Amount of difference want to detect
Based on previous literature: average SD Based on pilot data: SD Based on size of difference
Small: d=.20Medium: d=.50Large: d=.80
Statistical difference ≠ Clinical difference
3 4 5 60.0
2.5
5.0
7.5
10.0
Count
11% 34%2.5% 34% 11% 2.5%
Normal Curve and Distribution of Sample
Means
Significance Level (Type I Error): Significance Level (Type I Error): αα
The larger the sample size the greater the power
The larger the effect size the greater the power
The larger the significance level the greater the power
Power and Sample Size Table at Power and Sample Size Table at αα 1=.05 1=.05
ES .30 .40 .50 .60
Power
.70 108 62 41 29
.75 123 70 46 33
.80 140 80 52 37
.85 163 93 61 43
.90 193 110 72 51
.95 243 138 90 63
.99 353 200 129 91
Data Analysis comes first - Data Analysis comes first - power secondpower second
Determine your hypotheses
Determine your analyses
Determine the parameters for analysis by hypothesis (i.e., power, ES, α)
Conduct power analysis
Power analysis will require:Power analysis will require:
Type of analysisSample sizeEffect sizeSignificance levelNumber of groups or factorsPlug in the numbers
ConsiderationsConsiderations
What is your question?
What type of data do you have?
What are your hypotheses?
What are your resources?
Clinical vs. statistical significance
How will you present your data?
What you would like the news headline to be?
SAGES Research Committee, August 2006 SAGES Research Committee, August 2006 RE: Suggestions to Assist with Completing RE: Suggestions to Assist with Completing a Winning Applicationa Winning Application
Power Analysis: In order to minimize the reporting of false-negative data, a
power analysis should be performed for sample size determination. Power is the capability of a study to detect a difference if the difference really exists. A type II error occurs when a true difference exists between study populations but there are insufficient numbers of subjects to detect this difference.
Any grant submitted without one of the items below will not be eligible for review.
Power analysis. Please provide the following data: alpha and beta, sample size needed in each group, what difference is expected. (Example: "A power analysis was performed with a beta of .20 and an alpha of 05. Assuming that a 10% difference exists between patient and control groups, 150 subjects will be needed in each arm. Thus the study would provide an 80% chance that a difference would be detected if one exists.")
If a power analysis in not appropriate for the submitted project, a statement should be included explaining why a power analysis is not appropriate for the study.
Consultation with a statistician is recommended. However, there are many statistical software programs available
What to do when you need What to do when you need more powermore power
Increase sample size
Reduce number of variables
Show your data graphically
Power AnalysisPower Analysis
With n=400, alpha2 = .05, and a medium effect size (.30) the power will be >.99 for analyses of variance and .96 for zero order correlations120. The power for a regression analysis that includes 11 variables (i.e. sex, ethnicity, positive symptoms, negative symptoms, aggression, impulsivity, depression, premorbid adjustment, gene marker, family history and substance abuse) with n=400, alpha=.05 and a medium effect size (R2=.10) will be greater than .90. We do not anticipate that all 11 variables will contribute significantly to the model. With 6 variables, a more likely model, the power will be > .90. For the exploratory regression analyses conducted within the group of attempters (n=200) the power will be at least .80 to detect a medium effect size (R2=.10). For correlational studies among the genetic and biochemical measures (approximate n=100) the power will be .80 for a medium effect size (R2=.10) and .95 for a larger medium effect size (R2=.25). The increased sample size will now provide the power necessary to consider attempters and nonattempters separately. Analyses examining single attempters, multiple attempters and nonattempters (approximate sample sizes: 120, 80, 200) will still maintain adequate power (power>.70).
Grant application by Jill M. Harkavy-Friedman, PhD
Free Power and Sample Size Free Power and Sample Size CalculationCalculation
Cohen J. Statistical Power Analysis for the Behavioral Sciences (2nd edition). Hillsdale, New Jersey: Lawrence Erlbaum Associates, Publishers, 1988 http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSampleSi
ze
G*Power 2 http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/
(limited)
G*3 http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/
Top Related