Targeted Maximum Likelihood Learning of Scientific Causal Questions Mark J. van der Laan Division of...

Targeted Maximum Targeted Maximum Likelihood Learning of Likelihood Learning of

Scientific Causal Scientific Causal QuestionsQuestions

Mark J. van der LaanMark J. van der LaanDivision of BiostatisticsDivision of Biostatistics

U.C. BerkeleyU.C. Berkeley

JSM July 31, 2007, Salt Lake CityJSM July 31, 2007, Salt Lake Citywww.bepres.com/ucbbiostatwww.bepres.com/ucbbiostatwww.stat.berkeley.edu/~laanwww.stat.berkeley.edu/~laan

Targeted Maximum LikelihoodTargeted Maximum LikelihoodEstimation Flow ChartEstimation Flow Chart

Inputs

Target feature map: Ψ( )

User Dataset

The model is a set of possible probability distributions

of the data

Target Featurebetter estimates are closer to ψ(PTRUE)

Target feature values

Initial P-estimator of theprobability distribution of the data: P̂

P̂

TRUEP

ˆΨ(P*)

Ψ(PTRUE)

ˆΨ(P) Targeted feature estimator

True valueof the target feature

Initial feature estimator

Targeted P-estimator of the probability distributionof the dataO(1), O(2),

… O(n)

Observations

True probability distribution

ModelP*ˆ

Philosophy of the Philosophy of the Targeted P EstimatorTargeted P Estimator

Find element P* in the model which givesFind element P* in the model which givesLarge bias reduction for target featureLarge bias reduction for target feature, e.g., by requiring , e.g., by requiring

that it solves the efficient influence curve equationthat it solves the efficient influence curve equation

i=1i=1DD**(P)(O(P)(Oii)=0 in P)=0 in P

Small increase of log-likelihoodSmall increase of log-likelihood relative to the initial P relative to the initial P estimator (This usually results in a small increase in estimator (This usually results in a small increase in variance and preserves the overall quality of the initial P variance and preserves the overall quality of the initial P estimator)estimator)

An iterative targeted maximum likelihood procedure can An iterative targeted maximum likelihood procedure can be used to construct a targeted P estimator (described be used to construct a targeted P estimator (described later)later)

^

An example of targeted MLE An example of targeted MLE for a survival probabilityfor a survival probability

The transformed distribution The transformed distribution ˆ solves the solves the efficient influence curve equationefficient influence curve equation– The area under the curve of to the right of The area under the curve of to the right of

28 (the28 (the target feature target feature) equals the actual ) equals the actual proportion of observations >28 in our sampleproportion of observations >28 in our sample

pTRUE

actual probability distribution

function

Target feature: Survival at 28 yearsRed striped area under the red curve

Initial feature estimate: Green stripedarea under the green curve

Targeted feature estimate: Blue striped area under the blue curve.

Survival time

P*ˆ

00 10 20 30 40

P*ˆ

28

P̂

P*

density of P* – targeted P estimatorˆ

density of P – initial P estimatorˆ

The iterative Targeted MLEThe iterative Targeted MLE

Identify a strategy for “stretching” the function P so that a small “stretch” Identify a strategy for “stretching” the function P so that a small “stretch” yields the maximum change in the yields the maximum change in the target featuretarget feature. Mathematically, this is . Mathematically, this is achieved by constructing a path P(achieved by constructing a path P( with free parameter with free parameter through P whose through P whose score at score at = 0 equals the efficient influence curve at P. = 0 equals the efficient influence curve at P.

Given this optimal “stretching strategy”, we must determine the optimum Given this optimal “stretching strategy”, we must determine the optimum amount of stretch, amount of stretch, OPTOPT. This value is obtained by maximizing the likelihood of . This value is obtained by maximizing the likelihood of

the dataset over the dataset over ..

Applying the optimal amount of stretch Applying the optimal amount of stretch OPTOPT to P using our optimal stretching to P using our optimal stretching

function P(function P( yields a new probability distribution P yields a new probability distribution P11, which is called the , which is called the first first

step targeted maximum likelihood estimatorstep targeted maximum likelihood estimator..

PP11 can be substituted for P in the above process, producing an estimate P can be substituted for P in the above process, producing an estimate P22. .

This process continues until the incremental “stretch” is essentially zero.This process continues until the incremental “stretch” is essentially zero.

The last probability distribution generated is P*, which solves the efficient The last probability distribution generated is P*, which solves the efficient influence curve equation, thereby achieving the desired bias reduction with a influence curve equation, thereby achieving the desired bias reduction with a small increase in likelihood relative to P.small increase in likelihood relative to P.

In many cases, the convergence occurs in one step.In many cases, the convergence occurs in one step.

The iterative targeted MLE is double robust locally efficient.The iterative targeted MLE is double robust locally efficient.

^

^^

^^ ^

^ ^ ^

^

^

The iterative Targeted MLEThe iterative Targeted MLEIdentify a strategy for “stretching” the initial P so that a small “stretch” Identify a strategy for “stretching” the initial P so that a small “stretch” yields the maximum change in the yields the maximum change in the target featuretarget feature. Mathematically, this . Mathematically, this is achieved by constructing a path P(is achieved by constructing a path P( with free parameter with free parameter through through P whose score at P whose score at = 0 equals the efficient influence curve. = 0 equals the efficient influence curve.

Given this optimal “stretching strategy”, we must determine the Given this optimal “stretching strategy”, we must determine the optimum amount of stretch, optimum amount of stretch, OPTOPT. This value is obtained by maximizing . This value is obtained by maximizing

the likelihood of the dataset over the likelihood of the dataset over ..

Applying the optimal amount of stretch Applying the optimal amount of stretch OPTOPT to P using our optimal to P using our optimal

stretching function P(stretching function P( yields a new probability distribution, which is yields a new probability distribution, which is called the called the first step targeted maximum likelihood estimatorfirst step targeted maximum likelihood estimator..

This process continues until the incremental “stretch” is essentially This process continues until the incremental “stretch” is essentially zero.zero.

The last probability distribution generated is P*, which solves the The last probability distribution generated is P*, which solves the efficient influence curve equation, thereby achieving the desired bias efficient influence curve equation, thereby achieving the desired bias reduction with a small increase in likelihood relative to P.reduction with a small increase in likelihood relative to P.

In many cases, the convergence occurs in one step.In many cases, the convergence occurs in one step.

The iterative targeted MLE is double robust locally efficient.The iterative targeted MLE is double robust locally efficient.

^

^^

^

^

^

^ ^ ^

^

^

This process continues until the incremental This process continues until the incremental “stretch” is essentially zero.“stretch” is essentially zero.The last probability distribution generated is P*, The last probability distribution generated is P*, which solves the efficient influence curve which solves the efficient influence curve equation, thereby achieving the desired bias equation, thereby achieving the desired bias reduction with a small increase in likelihood reduction with a small increase in likelihood relative to P.relative to P.In many cases, the convergence occurs in one In many cases, the convergence occurs in one step.step.The iterative targeted MLE is double robust The iterative targeted MLE is double robust locally efficient in causal inference/censored locally efficient in causal inference/censored data applications.data applications.

An example of iterating with An example of iterating with targeted MLE to estimate a mediantargeted MLE to estimate a median

Starting with the initial P estimator PStarting with the initial P estimator P00, determine , determine optimal “stretching function” and “amount of optimal “stretching function” and “amount of stretch”, producing a new P estimator Pstretch”, producing a new P estimator P11

Continue repeating until further stretching is Continue repeating until further stretching is essentially zeroessentially zero

pTRUE

actual probability distribution

function

Survival time00 10 20 40Median for PTRUE

ˆˆ

ˆ

ˆ ˆ

p1ˆp2ˆ

pk-1ˆ

…

ˆ

pk = density of P* – targeted P estimatorˆ ˆ

p – density of P – initial P estimatorˆ ˆ

Targeted MLE for finding a medianTargeted MLE for finding a median

DataData is n survival times O is n survival times O11,…,O,…,Onn with common probability density p with common probability density p00

ModelModel for p for p00 is nonparametric is nonparametric Target featureTarget feature is median of p is median of p00

Initial P estimatorInitial P estimator is an (say) estimator p is an (say) estimator pnn (e.g., kernel density (e.g., kernel density estimator, or estimator based on working model such as the normal estimator, or estimator based on working model such as the normal distributions)distributions)FluctuationFluctuation p pnn(()=c()=c(,p,pnn)Exp()Exp( D(p D(pnn))p))pnn, where , where

D(pD(pnn)=I(O)=I(O·· Median(p Median(pnn))-0.5 ))-0.5 is theis the efficient influence curve efficient influence curve of median at p of median at pnn

Let Let nn11 be MLE, and set p be MLE, and set pnn

11=p=pnn((nn11), which is the ), which is the first step targeted first step targeted

MLE: MLE: this is a new curve in which the median is moved in the direction this is a new curve in which the median is moved in the direction of the empirical median.of the empirical median.Iterate until convergenceIterate until convergence In the limit, we have that the update In the limit, we have that the update ppnn

** has a has a median equal to the empirical median, i.e. the value at which 50% of median equal to the empirical median, i.e. the value at which 50% of data points are smaller than that valuedata points are smaller than that value

Targeted MLE for Causal EffectTargeted MLE for Causal EffectDose-Response CurveDose-Response Curve

O=(W,A,Y=Y(A)) O=(W,A,Y=Y(A)) drawn from probability distributiondrawn from probability distribution P PTrueTrue,, W baseline covariates, W baseline covariates, A dose of drug, Y outcome, Y(a) counterfactual dose-specific outcomesA dose of drug, Y outcome, Y(a) counterfactual dose-specific outcomesNo unmeasured confounders so that we have No unmeasured confounders so that we have Missing at RandomMissing at RandomModel for PModel for PTrueTrue is nonparametric is nonparametricE(Y(a)|V)E(Y(a)|V) dose response curve by strata V of a user supplied choice of effect dose response curve by strata V of a user supplied choice of effect modifier V modifier V WWTarget featureTarget feature is a weighted least squares projection of the dose response curve is a weighted least squares projection of the dose response curve on the working model on the working model m(a,V|m(a,V|)) where the weight function is denoted with where the weight function is denoted with h(A,V)h(A,V)Initial P estimatorInitial P estimator is (say) logistic regression fit of binary outcome Y on A,W is (say) logistic regression fit of binary outcome Y on A,WFirst step targeted MLEFirst step targeted MLE is obtained by adding to the logistic regression fit a is obtained by adding to the logistic regression fit a covariate extension covariate extension h(A,V)/g(A|W) h(A,V)/g(A|W) / / m(a,V| m(a,V|)) and computing the MLE of the and computing the MLE of the coefficient coefficient If the working model is linear in parameter If the working model is linear in parameter , then the iterative targeted MLE , then the iterative targeted MLE converges in one stepconverges in one stepNote: Note: In a randomized trial the targeted MLE typically converges in ZERO steps.In a randomized trial the targeted MLE typically converges in ZERO steps.

OutlineOutline Multiple Testing for variable importance in predictionMultiple Testing for variable importance in prediction Overview of Multiple TestingOverview of Multiple Testing Previous proposals of joint null distribution in resampling based Previous proposals of joint null distribution in resampling based

multiple testing: Westfall and Young (1994), Pollard, van der Laan multiple testing: Westfall and Young (1994), Pollard, van der Laan (2003), Dudoit, van der Laan, Pollard (2004).(2003), Dudoit, van der Laan, Pollard (2004).

Quantile Transformed joint null distribution: van der Laan, Hubbard Quantile Transformed joint null distribution: van der Laan, Hubbard 2005.2005.

Simulations.Simulations. Methods controlling tail probability of the proportion of false Methods controlling tail probability of the proportion of false

positives.positives.Augmentation Method: van der Laan, Dudoit, Pollard (2003)Augmentation Method: van der Laan, Dudoit, Pollard (2003)Empirical Bayes Resampling based Method: van der Laan, Birkner, Empirical Bayes Resampling based Method: van der Laan, Birkner, Hubbard (2005).Hubbard (2005).

Data Applications.Data Applications.Pathway Testing: Birkner, Hubbard, van der Laan (2005).Pathway Testing: Birkner, Hubbard, van der Laan (2005).ConclusionConclusion

Multiple Testing in PredictionMultiple Testing in Prediction

Suppose we wish to estimate and test for the importance Suppose we wish to estimate and test for the importance of each variable for predicting an outcome from a set of of each variable for predicting an outcome from a set of variables.variables.Current approach involves fitting a data adaptive Current approach involves fitting a data adaptive regression and measuring the importance of a variable in regression and measuring the importance of a variable in the obtained fit.the obtained fit.We propose to define variable importance as a (pathwise We propose to define variable importance as a (pathwise differentiable) parameter, and directly estimate it with differentiable) parameter, and directly estimate it with targeted maximum likelihood methodologytargeted maximum likelihood methodology This allows us to test for the importance of each variable This allows us to test for the importance of each variable separately and carry out multiple testing procedures.separately and carry out multiple testing procedures.

Example: HIV resistance Example: HIV resistance mutationsmutations

Goal: Rank a set of genetic mutations based on Goal: Rank a set of genetic mutations based on their importance for determining an outcometheir importance for determining an outcome– MutationsMutations (A)(A) in the HIV protease enzyme in the HIV protease enzyme

Measured by sequencingMeasured by sequencing

– Outcome (Y)Outcome (Y) = change in viral load 12 weeks after = change in viral load 12 weeks after starting new regimen containing saquinavirstarting new regimen containing saquinavir

– ConfoundersConfounders (W) = Other mutations, history of patient (W) = Other mutations, history of patient

How important is each mutation for viral How important is each mutation for viral resistanceresistance to this specific protease inhibitor to this specific protease inhibitor drug? drug? 00=E E(Y|A=1,W)-E(Y|A=0,W)=E E(Y|A=1,W)-E(Y|A=0,W)– Inform genotypic scoring systemsInform genotypic scoring systems

Targeted Maximum LikelihoodTargeted Maximum Likelihood

In regression case, implementation just involves In regression case, implementation just involves adding a covariate adding a covariate h(A,W)h(A,W) to the regression to the regression modelmodel

Requires estimating g(A|W)Requires estimating g(A|W)– E.g. distribution of each mutation given covariatesE.g. distribution of each mutation given covariates

RobustRobust: Estimate of : Estimate of ψψ00 is consistent if either is consistent if either– g(A|W) is estimated consistentlyg(A|W) is estimated consistently– E(Y|A,W) is estimated consistentlyE(Y|A,W) is estimated consistently

)|Pr()|( where,)|0(

)0(

)|1(

)1(),( WaAWag

Wg

AI

Wg

AIWAh

Mutation Rankings Based on Mutation Rankings Based on Variable ImportanceVariable Importance

Current ScoreCurrent Score MutationMutation VIMVIM VIM p-valueVIM p-value CrudeCrude Crude p-valueCrude p-value

3535 90M90M 0.700.70 0.000.00 0.760.76 0.000.00

4040 48VM48VM 0.790.79 0.000.00 1.071.07 0.000.00

00 30N30N -0.78-0.78 0.000.00 -1.06-1.06 0.000.00

1010 82AFST82AFST 0.460.46 0.010.01 0.350.35 0.030.03

1010 54VA54VA 0.460.46 0.010.01 0.310.31 0.110.11

1010 73CSTA73CSTA 0.670.67 0.030.03 0.800.80 0.000.00

22 20IMRTVL20IMRTVL 0.320.32 0.070.07 0.260.26 0.180.18

11 36ILVTA36ILVTA 0.280.28 0.100.10 0.270.27 0.120.12

22 10FIRVY10FIRVY 0.270.27 0.130.13 0.480.48 0.000.00

55 88DTG88DTG -0.23-0.23 0.240.24 -0.50-0.50 0.330.33

22 71TVI71TVI 0.180.18 0.290.29 0.140.14 0.370.37

55 32I32I -0.18-0.18 0.580.58 -0.20-0.20 0.550.55

22 63P63P 0.060.06 0.770.77 0.110.11 0.560.56

55 46ILV46ILV 0.130.13 0.980.98 0.270.27 0.100.10

Hypothesis Testing IngredientsHypothesis Testing Ingredients

Data (XData (X11,…,X,…,Xnn))

HypothesesHypotheses

Test StatisticsTest Statistics

Type I ErrorType I Error

Null DistributionNull DistributionMarginal (p-values) or Marginal (p-values) or

Joint distribution of the test statisticsJoint distribution of the test statistics

Rejection RegionRejection Region

Adjusted p-valuesAdjusted p-values

Type I Error RatesType I Error Rates

FWERFWER: Control the probability of at least one : Control the probability of at least one Type I error (VType I error (Vnn): ): P(VP(Vn n > 0) > 0) ··

gFWERgFWER: Control the probability of at least k : Control the probability of at least k Type I errors (VType I errors (Vnn): ): P(VP(Vn n > k) > k) ··

TPPFPTPPFP: Control the proportion of Type I errors : Control the proportion of Type I errors (V(Vnn) to total rejections (R) to total rejections (Rnn) at a user defined level ) at a user defined level q: q: P(VP(Vnn/R/Rn n > q) > q) ··

FDRFDR: Control the expectation of the proportion of : Control the expectation of the proportion of Type I errors to total rejections: Type I errors to total rejections: E(VE(Vnn/R/Rnn) ) ··

QUANTILE TRANSFORMED QUANTILE TRANSFORMED JOINT NULL DISTRIBUTIONJOINT NULL DISTRIBUTION

Let Let QQ0j0j be a marginal null distribution so that be a marginal null distribution so that

for jfor j22 S S00

QQ0j0j-1-1QQnjnj(x)(x)¸̧ x x

where where QQnjnj is the j-th marginal distribution of is the j-th marginal distribution of

the true distribution the true distribution QQnn(P)(P) of the test statistic of the test statistic

vector vector TTnn..

QUANTILE TRANSFORMED QUANTILE TRANSFORMED JOINT NULL DISTRUTIONJOINT NULL DISTRUTION

We propose as null distribution the We propose as null distribution the distribution distribution QQ0n0n of of

TTnn**(j)=Q(j)=Q0j0j

-1-1QQnjnj(T(Tnn(j)), j=1,…,J(j)), j=1,…,J

This joint null distribution This joint null distribution QQ0n0n(P)(P) does indeed does indeed

satisfy the wished multivariate asymptotic satisfy the wished multivariate asymptotic domination condition in (Dudoit, van der domination condition in (Dudoit, van der Laan, Pollard, 2004).Laan, Pollard, 2004).

We estimate this null distribution We estimate this null distribution QQ0n0n(P)(P) with with

the bootstrap analogue:the bootstrap analogue:

TTnn##(j)=Q(j)=Q0j0j

-1-1QQnjnj##(T(Tnn

##(j))(j))

where # denotes the analogue based on where # denotes the analogue based on bootstrap sample bootstrap sample OO11

##,..,O,..,Onn## of an of an

approximation approximation PPnn of the true distribution of the true distribution PP. .

BOOTSTRAP QUANTILE-BOOTSTRAP QUANTILE-TRANSFORMED JOINT NULL TRANSFORMED JOINT NULL

DISTRIBUTIONDISTRIBUTION

Description of SimulationDescription of Simulation

– 100 subjects each with one random 100 subjects each with one random XX (say a SNP’s) uniform over 0, 1 or 2. (say a SNP’s) uniform over 0, 1 or 2.

– For each subject, 100 binary For each subject, 100 binary YY’s, (’s, (YY11,...Y,...Y100100) generated from a model such that:) generated from a model such that:first 95 are independent of first 95 are independent of XXLast 5 are associated with Last 5 are associated with XXAll All YY’s correlated using random effects model’s correlated using random effects model

– 100 hypotheses of interest where the null is the independence of 100 hypotheses of interest where the null is the independence of X X andand Y Yi i ..

– Test statistic is Pearson’s Test statistic is Pearson’s 22 test where the null distribution is test where the null distribution is 22 with 2 df. with 2 df.

– In this case, In this case, YY00 is the outcome if, counter to fact, the subject had received is the outcome if, counter to fact, the subject had received A=0A=0..

– Want to contrast the rate of miscarriage in groups defined by Want to contrast the rate of miscarriage in groups defined by V,R,AV,R,A if among these women, if among these women, one removed decaffeinated coffee during pregnancy.one removed decaffeinated coffee during pregnancy.

Figure 1: Density of null distributions: null-centered, rescaled bootstrap,Figure 1: Density of null distributions: null-centered, rescaled bootstrap,quantile-transformed and the theoretical. A is over entire range, B is thequantile-transformed and the theoretical. A is over entire range, B is the

right tail.right tail.

Description of Simulation, cont.Description of Simulation, cont.

– Simulated data 1000 timesSimulated data 1000 times

– Performed the following MTP’s to control FWER at 5%.Performed the following MTP’s to control FWER at 5%.BonferroniBonferroniNull centered, re-scaled bootstrap (NCRB) – based on 5000 bootstrapsNull centered, re-scaled bootstrap (NCRB) – based on 5000 bootstrapsQuantile-Function Based Null Distribution (QFBND)Quantile-Function Based Null Distribution (QFBND)

– ResultsResultsNCRB anti-conservative (inaccurate)NCRB anti-conservative (inaccurate)Bonferroni very conservative (actual FWER is 0.005)Bonferroni very conservative (actual FWER is 0.005)QFBND is both accurate (FWER 0.04) and powerful (10 times the QFBND is both accurate (FWER 0.04) and powerful (10 times the power of Bonferroni).power of Bonferroni).

SMALL SAMPLE SIMULATIONSMALL SAMPLE SIMULATION

2 populations. 2 populations.

Sample nSample njj p-dim vectors from population j, p-dim vectors from population j,

j=1,2.j=1,2.

Wish to test for difference in means for each Wish to test for difference in means for each of p components.of p components.

Parameters for population j: Parameters for population j: jj, , jj, , jj..

hh00 is number of true nulls is number of true nulls

ADJUSTED P VALUESADJUSTED P VALUES

Empirical Bayes/Resampling Empirical Bayes/Resampling TPPFP MethodTPPFP Method

We devised a resampling based multiple testing We devised a resampling based multiple testing procedure, asymptotically controlling (e.g.) the procedure, asymptotically controlling (e.g.) the proportion of false positives to total rejections. proportion of false positives to total rejections. This procedure involves:This procedure involves:

– Randomly sampling a guessed (conservative) set Randomly sampling a guessed (conservative) set of true null hypotheses: e.g. of true null hypotheses: e.g. HH00(j)~Bernoulli (j)~Bernoulli (Pr(H(Pr(H00(j)=1|T(j)=1|Tjj)=p)=p00ff00(T(Tjj)/f(T)/f(Tjj) )) ) based based on the Empirical Bayes model: on the Empirical Bayes model: TTjj|H|H00=1 ~f=1 ~f0 0 TTjj~f ~f pp00=P(H=P(H00(j)=1)(j)=1) (p0=1 conservative) (p0=1 conservative)

– Our bootstrap quantile joint null distribution of test Our bootstrap quantile joint null distribution of test statistics.statistics.

REMARK REGARDING MIXTURE REMARK REGARDING MIXTURE MODEL PROPOSALMODEL PROPOSAL

Under overall null min(1,fUnder overall null min(1,f00(T(Tnn(j))/f(T(j))/f(Tnn(j)) ) does (j)) ) does

not converge to 1 as n converges to infinity, not converge to 1 as n converges to infinity, since the overall density f needs to be estimated. since the overall density f needs to be estimated. However, if number of tests converge to infinity, However, if number of tests converge to infinity, then this ratio will approximate 1.then this ratio will approximate 1.

This latter fact probably explains why, even This latter fact probably explains why, even under the overall null, we observe a good under the overall null, we observe a good practical performance in our simulations.practical performance in our simulations.

Emp. BayesTPPFP MethodEmp. BayesTPPFP Method1.1. Grab a column from the null distribution Grab a column from the null distribution

of length M. of length M.

2.2. Draw a length M binary vector corresponding Draw a length M binary vector corresponding to Sto S0n0n..

3.3. For a vector of c values calculate:For a vector of c values calculate:

4.4. Repeat 1. and 2. 10,000 times and average Repeat 1. and 2. 10,000 times and average over iterations.over iterations.

5.5. Choose the c value where P(rChoose the c value where P(rnn(c) > q)(c) > q)·· ..

nT~

Examples/SimulationsExamples/Simulations

Bacterial Microarray ExampleBacterial Microarray Example

Airborne bacterial levels in specific cities over a Airborne bacterial levels in specific cities over a span of several weeks are being collected and span of several weeks are being collected and compared.compared.

A specific Affymetrics array was constructed to A specific Affymetrics array was constructed to quantify the actual bacterial levels in these air quantify the actual bacterial levels in these air samples. samples.

We will be comparing the average (over 17 We will be comparing the average (over 17 weeks) strain-specific intensity in San Antonio weeks) strain-specific intensity in San Antonio versus Austin, Texas.versus Austin, Texas.

420 Airborne Bacterial Levels420 Airborne Bacterial Levels17 time points 17 time points

San Antonio vs AustinSan Antonio vs Austin

ProcedureProcedure

Number RejectedNumber Rejected

BonferroniBonferroni

FWEFWE = 0.05= 0.05 55

= 0.10= 0.10 66

AugmentationAugmentation

TPPFPTPPFP = 0.05= 0.05 1111

= 0.10= 0.10 1414

E.Bayes/BootstrapE.Bayes/Bootstrap

TPPFPTPPFP = 0.05= 0.05 1313

= 0.10= 0.10 2121

Protein Data ExampleProtein Data Example

We are interested in analyzing mass-spectrometry data We are interested in analyzing mass-spectrometry data to determine specific mass-to-charge ratios (m/z) which to determine specific mass-to-charge ratios (m/z) which significantly differ in mean intensity between two types of significantly differ in mean intensity between two types of leukemia, ALL and AML. leukemia, ALL and AML.

The data structure consists of two replicates each for 7 The data structure consists of two replicates each for 7 samples of AML and 13 samples of ALL.samples of AML and 13 samples of ALL.

The data has undergone preprocessing to correct for The data has undergone preprocessing to correct for baseline spectral shifts.baseline spectral shifts.

Mass Spectrometry DataMass Spectrometry Data

204 Protein Levels:204 Protein Levels:AML (7) vs ALL (13)AML (7) vs ALL (13)

ProcedureProcedure

Number RejectedNumber Rejected

BonferroniBonferroni

FWEFWE = 0.05= 0.05 00

= 0.10= 0.10 11

AugmentationAugmentation

TPPFPTPPFP = 0.05= 0.05 11

= 0.10= 0.10 11

E.Bayes/BootstrapE.Bayes/Bootstrap

TPPFPTPPFP = 0.05= 0.05 33

= 0.10= 0.10 33

CGH Arrays and Tumors CGH Arrays and Tumors in Micein Mice

11 Comparative genomic hybridization (CGH) arrays 11 Comparative genomic hybridization (CGH) arrays from cancer tumors of 11 mice.from cancer tumors of 11 mice.

DNA from test cells is directly compared to DNA from test cells is directly compared to DNA from normal cells using bacterial DNA from normal cells using bacterial artificial chromosomes (BACs), which are artificial chromosomes (BACs), which are small DNA fragments placed on an array. small DNA fragments placed on an array.

With CGH: With CGH: – differentially labeled test [tumor] and reference [healthy] differentially labeled test [tumor] and reference [healthy]

DNA are co-hybridized to the array. DNA are co-hybridized to the array. – Fluorescence ratios on each spot of the array are Fluorescence ratios on each spot of the array are

calculated. calculated. – The location of each BAC in the genome is known and The location of each BAC in the genome is known and

thus the ratios can be compiled into a genome-wide copy thus the ratios can be compiled into a genome-wide copy number profilenumber profile

Plot of Adjusted p-values for 3 procedures Plot of Adjusted p-values for 3 procedures vs. Rank of BAC (ranked by magnitude of vs. Rank of BAC (ranked by magnitude of

T-statistic)T-statistic)

Pathway TestingPathway TestingBiologists are often interested in testing the relationship Biologists are often interested in testing the relationship between a collection of genes or mutations and a between a collection of genes or mutations and a specific outcome.specific outcome.

For example, imagine the situation with 10 potential For example, imagine the situation with 10 potential mutations and an outcome of cancer/no cancer. mutations and an outcome of cancer/no cancer.

We propose using the Residual Sum of Squares (RSS) We propose using the Residual Sum of Squares (RSS) or Likelihood Ratio (LR) as a test statistic for the model, or Likelihood Ratio (LR) as a test statistic for the model, after fitting the data with a data adaptive regression after fitting the data with a data adaptive regression algorithm. algorithm.

The null distribution is obtained under the permutation The null distribution is obtained under the permutation distribution. distribution.

SimulationsSimulations

Underlying Model (10 total Xs) : ln(P/(1-P)) = Underlying Model (10 total Xs) : ln(P/(1-P)) = 00 + + 11XX11XX22..

MethodMethod Power (rejections/500 simulations)Power (rejections/500 simulations)

PolyclassPolyclass 0.7620.762

Regression XRegression X11 + X + X22 0.2040.204

Regression XRegression X11 + X + X22 + … X + … X1010 0.140.14

Forward/Backward Selection Forward/Backward Selection 0.1540.154

Global Test (Pathway Testing)Global Test (Pathway Testing) 0.1840.184

FWERFWER 0.0620.062

COMBINING PERMUTATION COMBINING PERMUTATION DISTRIBUTION WITH QUANTILE DISTRIBUTION WITH QUANTILE

NULL DISTRIBUTIONNULL DISTRIBUTIONFor a test of independence, the For a test of independence, the permutation distribution is the preferred permutation distribution is the preferred choice of marginal null distribution, due to choice of marginal null distribution, due to its finite sample control.its finite sample control.We can construct a quantile transformed We can construct a quantile transformed joint null distribution whose marginals joint null distribution whose marginals equal these permutation distributions, and equal these permutation distributions, and use this distribution to control any wished use this distribution to control any wished type I error rate. type I error rate.

ConclusionsConclusions

Quantile function transformed bootstrap null distribution for test-statistics is Quantile function transformed bootstrap null distribution for test-statistics is generally valid and powerful in practice.generally valid and powerful in practice.Powerful Emp Bayes/Bootstrap Based method sharply controlling proportion Powerful Emp Bayes/Bootstrap Based method sharply controlling proportion of false positives among rejections.of false positives among rejections.Combining general bootstrap quantile null distribution for test statistics with Combining general bootstrap quantile null distribution for test statistics with random guess of true nulls provides general method for obtaining powerful random guess of true nulls provides general method for obtaining powerful (joint) multiple testing procedures (alternative to step down/up methods).(joint) multiple testing procedures (alternative to step down/up methods).Combining data adaptive regression with testing and permutation Combining data adaptive regression with testing and permutation distribution provides powerful test for independence between collection of distribution provides powerful test for independence between collection of variables and outcome.variables and outcome.Combining permutation marginal distribution with quantile transformed joint Combining permutation marginal distribution with quantile transformed joint bootstrap null distribution provides powerful valid null distribution if the null bootstrap null distribution provides powerful valid null distribution if the null hypotheses are tests of independence.hypotheses are tests of independence.Targeted ML estimation of variable importance in prediction allows multiple Targeted ML estimation of variable importance in prediction allows multiple testing (and inference) of variable importance for each variable.testing (and inference) of variable importance for each variable.


Suppose we wish to estimate and test for the importance Suppose we wish to estimate and test for the importance of each variable for predicting an outcome from a set of of each variable for predicting an outcome from a set of variables.variables.Current approach involves fitting a data adaptive Current approach involves fitting a data adaptive regression and measuring the importance of a variable in regression and measuring the importance of a variable in the obtained fit.the obtained fit.We propose to define variable importance as a (pathwise We propose to define variable importance as a (pathwise differentiable) parameter, and directly estimate it with differentiable) parameter, and directly estimate it with general estimating function methodologygeneral estimating function methodology This allows us to test for the importance of each variable This allows us to test for the importance of each variable separately and carry out multiple testing procedures.separately and carry out multiple testing procedures.

Application in HIV Sequence Application in HIV Sequence AnalysisAnalysis

336 patients for which we measure sequence of 336 patients for which we measure sequence of HIV virus, and replication capacity of virus.HIV virus, and replication capacity of virus.The PRO positions 4-99 and RT positions 38-The PRO positions 4-99 and RT positions 38-222 are used, resulting in a total of 282 222 are used, resulting in a total of 282 positions, which are coded as a binary covariate.positions, which are coded as a binary covariate.We wish to test for the importance of each We wish to test for the importance of each mutation.mutation.Running a data adaptive regression algorithm Running a data adaptive regression algorithm resulted in resulted in

Algorithm: max-T Single-Step Approach Algorithm: max-T Single-Step Approach (FWER)(FWER)

The maxT procedure is a JOINT procedure used to control FWER. The maxT procedure is a JOINT procedure used to control FWER.

Apply the bootstrap method (B=10,000 bootstrap samples) to obtain Apply the bootstrap method (B=10,000 bootstrap samples) to obtain the bootstrap distribution of test statistics (M x B matrix). the bootstrap distribution of test statistics (M x B matrix).

Mean-center at null value to obtain the wished null distributionMean-center at null value to obtain the wished null distribution

Chose the maximum value over each column, therefore resulting in Chose the maximum value over each column, therefore resulting in a vector of 10,000 maximum values.a vector of 10,000 maximum values.

Use as common cut-off value for all test statistics the (1-Use as common cut-off value for all test statistics the (1-) quantile ) quantile of these numbers.of these numbers.

FPRP and FDRFPRP and FDR

P0=Prob(Null is true)P0=Prob(Null is true)T=test statisticT=test statisticProb(T>c| Null is true)=S0(c)Prob(T>c| Null is true)=S0(c)Prob(T>c)=S(c) Prob(T>c)=S(c)

(=P0*S0(c)+(1-P0)S1(c)) (=P0*S0(c)+(1-P0)S1(c))FPRP(c)=Prob(Null is true|T>c) FPRP(c)=Prob(Null is true|T>c)

= P0*S0(c)/S(c)= P0*S0(c)/S(c)Fact: If FPRP(c(j))< alpha for a list of Fact: If FPRP(c(j))< alpha for a list of independent tests statistics T(j), then independent tests statistics T(j), then FDR=E(V/R)< alpha.FDR=E(V/R)< alpha.

Proof:Proof:

VVnn(c)=(c)=jj I(T I(Tnn(j)>c(j)>cjj,H,H00(j)=1)(j)=1)

RRnn(c)=(c)=jj I(T I(Tnn(j)>c(j)>cjj))

Take conditional expectation of VTake conditional expectation of Vnn(c), (c),

given (Tgiven (Tnn(j)>c(j)>cjj) for all j, to obtain ) for all j, to obtain

I(TI(T(j)>c(j))*FPRP(c(j)) (j)>c(j))*FPRP(c(j))

Thus, if FPRP(cThus, if FPRP(cjj))·· , then , then

E(VE(Vnn(c)/R(c)/Rnn(c))(c))··

Empirical Bayes FDR and BH-FDREmpirical Bayes FDR and BH-FDR

1) Assume common mixture model for all test-1) Assume common mixture model for all test-statistics, 2) fit FPRP() (i.e., marginal null statistics, 2) fit FPRP() (i.e., marginal null distribution S0 and true distribution S) from data, distribution S0 and true distribution S) from data, and 3) reject the null hypothesis if FPRP at the and 3) reject the null hypothesis if FPRP at the value of the observed test statistic is smaller value of the observed test statistic is smaller than alpha (Storey et al. late 90’s).than alpha (Storey et al. late 90’s).

The above method controls FDR at level alpha, The above method controls FDR at level alpha, and is equivalent with frequentist Benjamini-and is equivalent with frequentist Benjamini-Hochberg FDR method (1995).Hochberg FDR method (1995).

What is our test-statistic?What is our test-statistic?

The test-statistic of interest is:The test-statistic of interest is: TTnn = = difdif – 0 – 0

difdif

Where Where difdif is the mean difference over the 17 is the mean difference over the 17

weeks and weeks and difdif is respective the standard error. is respective the standard error.

We applied several methods to this analysis: We applied several methods to this analysis: Bonferroni, Joint-bootstrap null distribution Bonferroni, Joint-bootstrap null distribution method & Augmentation, and TPPFP (q=0.1).method & Augmentation, and TPPFP (q=0.1).

What is our test statistic?What is our test statistic?We are interested in testing the difference in the mean We are interested in testing the difference in the mean intensities of the AML versus the ALL samples at each of intensities of the AML versus the ALL samples at each of the 204 m/z ratios.the 204 m/z ratios.

The test-statistic of interest is:The test-statistic of interest is:

TTnn= = AMLAML - - ALLALL

AML/ALLAML/ALL

Where Where AMLAML is the mean difference of AML samples, is the mean difference of AML samples, ALLALL is the mean difference of ALL samples, and is the mean difference of ALL samples, and AML/ALLAML/ALL is the is the pooled standard error of AML and ALL.pooled standard error of AML and ALL.

We applied several methods to this analysis: Bonferroni, We applied several methods to this analysis: Bonferroni, Joint-bootstrap null distribution method & Augmentation, Joint-bootstrap null distribution method & Augmentation, and TPPFP (q=0.1).and TPPFP (q=0.1).

Extra SlidesExtra Slides

Multiple Testing ProceduresMultiple Testing Procedures– Order genes by p-value based on t-Order genes by p-value based on t-

statistic (the natural null here is the means statistic (the natural null here is the means for each row are 0 implying no mean for each row are 0 implying no mean difference in copy number for a particular difference in copy number for a particular BAC).BAC).

– Compare Benjamini and Hochbergs FDR Compare Benjamini and Hochbergs FDR and Bonferonni’s FWER adjusted p-and Bonferonni’s FWER adjusted p-values to those based on the re-sampling values to those based on the re-sampling TPPFP method (in this case, set q = TPPFP method (in this case, set q = 0.10).0.10).

Test StatisticsTest Statistics

A test statistic is written as:A test statistic is written as:

TTnn= = ((nn - - 00))

n n

Where Where nn is the standard error, is the standard error, nn is the is the

parameter of interest, and parameter of interest, and 00 is the null is the null

value of the parameter.value of the parameter.

HypothesesHypotheses

Hypotheses are created as one-sided or two Hypotheses are created as one-sided or two sided.sided.

A one-sided hypothesis:A one-sided hypothesis:

HH00(m)=I((m)=I(nn==00), m=1,…,M.), m=1,…,M.

A two-sided hypothesis:A two-sided hypothesis:

HH00(m)=I((m)=I(nn ·· 00), m=1,…,M.), m=1,…,M.

Type I & II ErrorsType I & II Errors

Type I errorsType I errors corresponds to making a false corresponds to making a false positive.positive.

Type II errors (Type II errors ()) corresponds to making a false corresponds to making a false negative.negative.

TheThe Power Power is defined as 1- is defined as 1-

Multiple Testing Procedures are interested in Multiple Testing Procedures are interested in simultaneously minimizing the Type I error rate simultaneously minimizing the Type I error rate while maximizing power.while maximizing power.

Null DistributionNull Distribution

The null distribution is the distribution to which The null distribution is the distribution to which the original test statistics are compared and the original test statistics are compared and subsequently rejected or accepted as null subsequently rejected or accepted as null hypotheses.hypotheses.

Multiple Testing Procedures are based on either Multiple Testing Procedures are based on either MarginalMarginal or or JointJoint Null Distributions. Null Distributions.

Marginal Null DistributionsMarginal Null Distributions are based on the are based on the marginal distribution of the test statistics.marginal distribution of the test statistics.

Joint Null DistributionsJoint Null Distributions are based on the joint are based on the joint distribution of the test statistics.distribution of the test statistics.

Rejection RegionsRejection Regions

Multiple Testing Procedures use the null Multiple Testing Procedures use the null distribution to create rejection regions for distribution to create rejection regions for the test statistics. the test statistics.

These regions are constructed to control These regions are constructed to control the Type I error rate.the Type I error rate.

They are based on the null distribution, the They are based on the null distribution, the test statistics, and the level test statistics, and the level . .

Single-Step & StepwiseSingle-Step & Stepwise

Single-stepSingle-step procedures assess each null procedures assess each null hypothesis using a rejection region which hypothesis using a rejection region which is independent of the tests of other is independent of the tests of other hypotheses.hypotheses.

StepwiseStepwise procedures construct rejection procedures construct rejection regions based on the acceptance/rejection regions based on the acceptance/rejection of other hypotheses. They are applied to of other hypotheses. They are applied to smaller nested subsets of tests (e.g. Step-smaller nested subsets of tests (e.g. Step-down procedures).down procedures).

Adjusted p-valuesAdjusted p-values

Adjusted p-values are constructed as Adjusted p-values are constructed as summary measures for the test statistics.summary measures for the test statistics.

We can think of the adjusted p-value We can think of the adjusted p-value p(m)p(m) as the nominal level as the nominal level at which test at which test statistic statistic T(m)T(m) would have just been would have just been rejected. rejected.

Multiple Testing ProceduresMultiple Testing Procedures

Many of the Multiple Testing Procedures are Many of the Multiple Testing Procedures are constructed with various assumptions regarding constructed with various assumptions regarding the dependence structure of the underlying test the dependence structure of the underlying test statistics. statistics.

We will now describe a procedure which controls We will now describe a procedure which controls a variety of Type I error rates and uses a null a variety of Type I error rates and uses a null distribution based on the joint distribution of the distribution based on the joint distribution of the test statistics (Pollard and van der Laan (2003)), test statistics (Pollard and van der Laan (2003)), with with no underlying dependence assumptionsno underlying dependence assumptions..

Null Distribution Null Distribution (Pollard & van der Laan (2003))(Pollard & van der Laan (2003))

This approach is interested in Type I error control under the true data This approach is interested in Type I error control under the true data generating distribution, as opposed to the data generating null generating distribution, as opposed to the data generating null distribution, which does not always provide control under the true distribution, which does not always provide control under the true underlying distribution (e.g. Westfall & Young).underlying distribution (e.g. Westfall & Young).

We want to use the null distribution to derive rejection regions for the We want to use the null distribution to derive rejection regions for the test statistics such that the Type I error rate is (asymptotically) test statistics such that the Type I error rate is (asymptotically) controlled at desired level controlled at desired level . .

In practice, the true distribution In practice, the true distribution QQnn=Q=Qnn(P)(P), for the test statistics T, for the test statistics Tnn, is , is unknown and replaced by a null distribution unknown and replaced by a null distribution QQ00 (or estimate, (or estimate, QQ0n0n). ).

The proposed null distribution QThe proposed null distribution Q00 is the asymptotic distribution of the is the asymptotic distribution of the vector of null value shifted and scaled test statistics, which provides vector of null value shifted and scaled test statistics, which provides the desired asymptotic control of the Type I error rate. the desired asymptotic control of the Type I error rate.

t-statistics:t-statistics: For the test of single-parameter null hypotheses using t- For the test of single-parameter null hypotheses using t-statistics the null distribution Qstatistics the null distribution Q00 is an M--variate Gaussian is an M--variate Gaussian distribution. distribution.

QQ00 = Q = Q00(P) (P) ´́ N(0, N(0,**(P)).(P)).

gFWER AugmentationgFWER Augmentation

gFWER Augmentation set:gFWER Augmentation set: The next k The next k hypotheses with smallest FWER adjusted hypotheses with smallest FWER adjusted p-values.p-values.

The adjusted p-values:The adjusted p-values:

TPPFP AugmentationTPPFP AugmentationTPPFP Augmentation set:TPPFP Augmentation set: The next hypotheses The next hypotheses with the smallest FWER adjusted p-values where with the smallest FWER adjusted p-values where one keeps rejecting null hypotheses until the ratio of one keeps rejecting null hypotheses until the ratio of additional rejections to the total number of rejections additional rejections to the total number of rejections reaches the allowed proportion q of false positives.reaches the allowed proportion q of false positives.

The adjusted p-values:The adjusted p-values:

TPPFP TechniqueTPPFP TechniqueThe TPPFP Technique was created as a less conservative and The TPPFP Technique was created as a less conservative and more powerful method of controlling the tail probability of the more powerful method of controlling the tail probability of the proportion of false positives.proportion of false positives.

This technique is based on constructing a distribution of the set of This technique is based on constructing a distribution of the set of null hypotheses null hypotheses SS0n0n, as well as a distribution under the null , as well as a distribution under the null hypothesis (hypothesis (TTnn). We are interested in controlling the random ). We are interested in controlling the random variable variable rrnn(c)(c)..

The distribution under the null is the identical null distribution used in The distribution under the null is the identical null distribution used in Pollard and van der Laan (2003): mean centered joint distribution of Pollard and van der Laan (2003): mean centered joint distribution of test-statistics.test-statistics.

Constructing SConstructing S0n0n

SS0n0n is defined by drawing a null or alternative status for each of the is defined by drawing a null or alternative status for each of the test statistics. The model defining the distribution of test statistics. The model defining the distribution of SS0n0n assumes assumes TTnn(m) (m) »» p p00ff00 + (1-p + (1-p00)f)f11, a mixture of a null density f, a mixture of a null density f00 and alternative and alternative density fdensity f11..

The posterior probability, defined as the probability that TThe posterior probability, defined as the probability that Tnn(m) came (m) came from a true null, Hfrom a true null, H0m0m, given its observed value: , given its observed value:

P(B(m)=0|TP(B(m)=0|Tnn(m)) = p(m)) = p0 0 ff00(T(Tnn(m))(m)) f(Tf(Tnn(m))(m))

Given TGiven Tnn, we can draw the random set S, we can draw the random set S0n0n from: from: SS0n0n = ( j:C(j) = 1), C(j) = ( j:C(j) = 1), C(j) »» Bernoulli(min(1,p Bernoulli(min(1,p00ff00(T(Tnn(m)/f(T(m)/f(Tnn(m)))).(m)))).

Note: We estimated f(TNote: We estimated f(Tnn(m)) using a kernel smoother on a (m)) using a kernel smoother on a bootstrapped set on Tbootstrapped set on Tnn(m), f(m), f00 »» N(0,1), and p N(0,1), and p00=1.=1.

Augmentation MethodsAugmentation Methods

Given adjusted p-values from a FWER Given adjusted p-values from a FWER controlling procedure, one can easily control controlling procedure, one can easily control gFWER or TPPFP.gFWER or TPPFP.

gFWER:gFWER: Add the next Add the next kk most significant most significant hypotheses to the set of rejections from the hypotheses to the set of rejections from the FWER procedure.FWER procedure.

TPPFP:TPPFP: Add the next Add the next (q/1-q)r(q/1-q)r00 most most significant hypotheses to the set of rejections significant hypotheses to the set of rejections from the FWER procedure. from the FWER procedure.

Application in HIV Sequence Application in HIV Sequence AnalysisAnalysis

336 patients for which we measure sequence of 336 patients for which we measure sequence of HIV virus, and replication capacity of virus.HIV virus, and replication capacity of virus.The PRO positions 4-99 and RT positions 38-The PRO positions 4-99 and RT positions 38-222 are used, resulting in a total of 282 222 are used, resulting in a total of 282 positions, which are coded as a binary covariate.positions, which are coded as a binary covariate.We wish to test for the importance of each We wish to test for the importance of each mutation.mutation.Running a data adaptive regression algorithm Running a data adaptive regression algorithm resulted in resulted in

Multiple Testing ProceduresMultiple Testing ProceduresProcedureProcedure Error RateError Rate StepStep Null DistributionNull Distribution

BonferroniBonferroni FWERFWER SSSS MarginalMarginal

Holm, HochbergHolm, Hochberg FWERFWER SD SUSD SU MarginalMarginal

Lehmann & Romano*Lehmann & Romano* gFWER, gFWER, TPPFPTPPFP

SS, SS, SDSD

MarginalMarginal

Genovese, WassermanGenovese, Wasserman TPPFPTPPFP MarginalMarginal

Benjamini & Hochberg*Benjamini & Hochberg* FDRFDR SUSU MarginalMarginal

Benjamini & YekutieliBenjamini & Yekutieli FDRFDR SUSU MarginalMarginal

Westfall & Young*Westfall & Young* FWERFWER SS, SS, SDSD

JointJoint

Pollard & van der LaanPollard & van der Laan

Dudoit et al.Dudoit et al.

van der Laan et al. ‘svan der Laan et al. ‘s

gFWER gFWER

gFWERgFWER

FWE, TPPFPFWE, TPPFP

SS, SS,

SSSS

SDSD

JointJoint

Notes for preparing presentationNotes for preparing presentation

Joint null distributionJoint null distributionEstimation of set of nulls, mixture model.Estimation of set of nulls, mixture model.Under overall null this mixture estimate is Under overall null this mixture estimate is inconsistent, but if the number of tests inconsistent, but if the number of tests increrasses the bias will go to zero. That might increrasses the bias will go to zero. That might explain why still robust and good performance in explain why still robust and good performance in simulations of houston.simulations of houston.If one is not a null then it is a consistent estimate If one is not a null then it is a consistent estimate sincce posterior prob B_n=0 is estmated as sincce posterior prob B_n=0 is estmated as f_0(T_n)/f(T_n), f shifts to right, so if T_n from f_0(T_n)/f(T_n), f shifts to right, so if T_n from null then this converges to infinity and thus is set null then this converges to infinity and thus is set to 1to 1

Targeted Maximum Likelihood Learning of Scientific Causal Questions Mark J. van der Laan Division of...

Documents

Transcript of Targeted Maximum Likelihood Learning of Scientific Causal Questions Mark J. van der Laan Division of...