Using primary care databases to evaluate drug benefits and harms: are the results replicable and...

Using primary care databases toevaluate drug benefits and harms:are the results valid andreplicable?

David A. Springate, University of ManchesterCentres for Primary Care/Biostatistics

Outline

1. Primary Care Database (PCD) study validity

2. PCD Replications

– Statins and Ischaemic heart disease– β-blockers and Cancer

3. Lessons to be learned

PCD studies are all the rage. . .Number of UK PCDpublications is rapidlyincreasing

1990 1995 2000 2005 2010

050

100

150

PCD articles in pubmed

year

Num

ber

of a

rtic

les

There is global interest in UK PCDresearch

Institutions affiliated with UK PCD publications

xx

x x

xxxxxxx

x x x

x

xx

x

x

x

xxx

xx

xxxx xxxxx

xxx

xxxxxx

xxxx

x

xxx

x

x xx xx

x

xx

x

xxxxxxx

xxxx x

xxx

x xxx

x

xxxxx

xxxxxx

x

xx

xxx

xxxxx

x xxx

x

xxx

xxx

x

x

xx

xx

x

x

x

xx xx

x

xxx

x x

xx

xx

xx

xx

xx x

xx

xxxxx

xx

xxx

x

xx

x

xxx

x xx

xx

xxx

x

xx

x

xxxxxx x

x

xxx

xxxxxx

xxx

x

x

xx

xxxxxxxx

x

xxxxxxxx

xxxx

xx

x

x

xx x

xxxx

x

xxxx

x xxxxxx

xxxxxx

xxxxx

x

x

x

xxxx

xxxxx

xx

xxx

xx

xx

x

x

x

xxx

xxxxxxxx x

x

x

xx

x

x

x

x

xx

xxx

x x

xx

x

xxx

x

x

xxxx

xxxxxx

x

x xxxxx x

xxxxx

xxxx

x

x

xxxxxxxx xxx

x

xxxxx

x

x xx xxx xx

xx xxxxx

x

xxxxx

x

xxxxxxx

xxx

x

xxx

x

xx

xx

x

x

xxxx

xxx

xxxxx

xxxxx

xx

x

xx x

xxx

xxxxxxx

x

x

xxxx

x xxxxx

x

xx

xx

x

x

x

x

x

x

x

x xxxxxx xxxx xx

x

xxxx

xx

x

x xx x

xx

x

xx

x

x

xxx

xx

xxx

xxxxx

x

x

x

xx

xxx x

xx

xxx

xxxx

xxxx

xx

x

x

xxxx

x

xx

xxxx

x xxxx

xx

xxxxxxxxxxx

xxxx

x

xxxx

xx

x

xxxx xx

xx x

x

x

x xx

x

xxx

x

xx

xxxxxx xxx

xx

x

x

x

xxxx

xx

xxxxx

xx

xxx

xxx

xxx

xx x

x

xxxxx

x

xx xxxx

xxxxxxx

x

x

x

xx

x

x

xx

x

xxx

xx

x

xxxxxx x

xx

xxx

xx

xxxxxxxxx

x

x

xxxxxx xx

x

x x x

xxxx

xx

xx

xxx

xxx

x xx

xxx

xxx

x

x

xxx

xxx

x

x

x

x

xxxxx

x

x x

xxxxxxxxxxxxxxx x

xxxx

xxx

xxxx x

x

x

xxxx

xx

x

xx

xxxx

xx

x xx

xxx

xxx

x

x

x xxxxxxx xxxx

x

xxxx

x

x

xxx

x xx

xxxxx

x

x

x

x

x

x

x

xxxxxxxxx

x

xxx

xxx

xxx

x

xx xxxxxx

x

xxxxxxxx xxxxx

x

xxxx

xxxxxxxx

x

x

xx

xxx

x

x xxx

xxxx

xx

xx

x

x

x

xxx

x x

xxxx

x

x

xxxx

xx

xxxx

x

xxx x

x

xxx x

x

x x

xx

xx

xxx

xxx

xxxxxxxxx x

xxx

xxx

xxxxxxx

xx

xxxx

xxxx

xxxx

xx

x xxx

x

xxx

x

xx

xx

x

xx

x xx

x xxx x

xxxx x

xx

xx

x

x

x

xxxxxxx

xxx

xxx

xx

x

x

xx

xx

xx

x

x

xx

x

xxxxxxx

x

xxxxxxxxxxx xxxxxxxxxxxxxx xx

xxx

xxx

xxxxxxxxxxxxxxxxxxxxxx xxxxx

Uses of Primary care databases. . .

• Prevalence / incidence studies

• Associations between conditions

• Harms and risks of treatments

• Comparative effectiveness

• RCT comparisions / replications (and replacements?)

BUT, There are still concerns about thevalidity of PCD-based studies. . .

Threat Refs

Data quality Herrett 2009,

Khan 2010, Jordan 2004

Data completeness Marston 2010, Delaney 2007,

Collins 2010

Confounding Tannen 2008, Lewis 2007

Clinical coding www.ClinicalCodes.org

PCD Replications“Non-reproducible singleoccurrences are of nosignificance to science.”

—– Karl Popper (1959)

An approach to validity that askswhether flaws and differences inthe data make any difference tothe ultimate conclusions ratherthan looking at validity andcompleteness of the underlyingindividual data

http://xkcd.com/242

Replicating studies in another, independentPCD

• Agreement implies that conclusions are not dependenton data source

• Some factors could influence both (such asconfounding by indication)

• First completely independent PCD replication (but seeVinogradova (BMJ 2013): relationship betweenbisphosphate exposure and cancer in QResearch andCPRD)

Replications were performed in CPRD

• Largest UK primary care database (CPRD-GOLD)

• ˜ 14 million patients

• ˜ 650 practices across the UK

• Uses the Vision GP computer system

Criteria for replication

1. Effectiveness studies

2. Different GP computer system from CPRD (notVision)

3. No practice overlap with CPRD

4. Representative coverage

5. Primary Care Database (Not integratedprimary/secondary/pharmacy)

PCD replications

Hippisley-Cox and Coupland (2006) Effect of statins on themortality of patients with ischaemic heart disease: populationbased cohort study with nested case-control analysis. Heart92:752-758 (QResearch)

Shah, Carey et al. (2011) Does β-adrenoceptor blockertherapy improve cancer survival? Findings from apopulation-based retrospective cohort study

Br J Clin Pharmacol 72:157-161 (DIN-LINK)

We then compare

• summary statistics

• mortality rates

• model coefficients and standard errors

Effect of statins on the mortality of patientswith ischaemic heart disease (QResearch)

Objective To measure the effect of statins on mortality forcommunity based patients with IHD

Design Cohort survival analysis and nested case-control

Setting 1.18 million patients in 89 practices

Subjects Patients with first diagnosis of IHD betweenJanuary 1996 and December 2003

Outcomes - Cohort: Adjusted hazard ratios (+/-95%CI) for all-cause mortality

- Case-control: Odds ratio (+/-95%CI) forcurrent use of statins, previous use andduration of use

Effect of statins on the mortality of patientswith ischaemic heart diseaseSummary statistics

Measure Analysis CPRD Qresearch

Number of practices Cohort 661 89Number of patients Cohort 91589 13029

Cases 15591 2266Controls 62356 9064

Median age Cases 80 80Controls 79 80

Percent female Cases 45.5% 44.3%Controls 45.5% 44.3%

Median followup (months) Cases 22.1 20.3Controls 22.5 21

Percent on statins Cases 17% 19.6%Controls 23.6 25.4%

Mortality rates for patients with IHD

Age Comorbidity Sex

0

100

200

300

400

Tota

l0−

44

45−5

4

55−6

4

65−7

4

75−8

4

85−9

4

95_p

lus

No dia

bete

s

Diabet

es

No hy

perte

nsion

Hyper

tens

ion

No CCF

CCF

Fem

aleM

ale

Patient group

Mor

talit

y ra

te (

per

1000

per

son

year

s)

PCD

CPRD

QResearch

Survival analyses for patients on Statins

QResearch

Adjusted HR 0.47 (0.41 to 0.53)

CPRD

0 2 4 6 8

Time since diagnosis of IHD (Years)

Sur

viva

l

0.00

0.25

0.50

0.75

1.00

Adjusted HR 0.43 (0.40 to 0.46)

Case-control analysis: Odds ratios for effectsof Statins on mortality in IHD patients

Odds are relative topatients not on statins.Dotted line represents1:1 odds

Previously on statins Currently on statins

0.0

0.5

1.0

1.5

2.0

All Sta

tins

Atorv

asta

tin

Ceriva

statin

Fluvas

tatin

Prava

statin

Simva

statin

All Sta

tins

Atorv

asta

tin

Ceriva

statin

Fluvas

tatin

Prava

statin

Simva

statin

Statin type

Odd

s ra

tio PCD

CPRD

Qresearch

Misleading pooled odds ratios

Combining effects of e.g. two drugs of the same BNFchapter

Group Y N Odds ratio

Drug 1 Cases 600 2003/3 = 1

Drug 1 Controls 75 25

Drug 2 Cases 10 300.333/1 = 0.333

Drug 2 Controls 30 30

Pooled Cases 610 2302.65/1.91 = 1.39

Pooled Controls 105 55

Case-control analysis: Adjusted OR forduration of use of statins on survival

Odds are relative topatients not on statins.Dotted line represents1:1 odds

0.00

0.25

0.50

0.75

1.00

0−12

13−2

4

25−3

6

37−4

8

49−6

0>6

0

Duration (months)

Adj

uste

d O

dds

ratio

PCD

CPRD

Qresearch

Summary — Statins study

1. Strikingly similar results in the two studies, despitedifferent GP computer systems (Vision vs EMIS)

2. As expected, narrower confidence intervals due tolarger study

3. Original study was well designed (Matching,appropriate analyses etc.)

4. Given the results, pooling of “all statins” isquestionable

Does β-adrenoceptor blocker therapy improvecancer survival?(DIN-LINK)

Objective To examine the effect of β-blocker treatment oncancer survival

Design Survival analyses for 9 cancer types

Setting 3462 cancer patients on β-blocker or otherantihypertensive therapy

Subjects Patients 40-85 with first cancer diagnosisbetween 1997 and 2006

Outcomes - Adjusted hazard ratios (+/- 95%CI) forall-cause mortality in each cancer type

- Pooled hazard ratio (random effects)

Comparison of summary statisticsCohort size:

CPRD 11316

DIN-LINK 3462

BPLM = Blood pressure

lowering medicines

Deaths Alive at 1 year Diagnosed 1997−2001 Diagnosed 2002−2006

0

25

50

75

100

Beta−

block

ers

Other

BPLM

Beta−

block

ers

Other

BPLM

Beta−

block

ers

Other

BPLM

Beta−

block

ers

Other

BPLM

Drug combination

Per

cent

age

of p

atie

nts

PCD

CPRD

DIN−LINK

Comparison of patient samples by cancer site

BPLM = Blood pressure

lowering medicines

0

10

20

30

40

brea

st

colon lun

g

oeso

phag

us

ovar

ian

panc

reas

pros

tate

rena

l

stom

ach

Cancer site

Per

cent

age

of p

atie

nts

PCD

CPRD

DIN−LINK

Survival analyses: comparison of hazard ratios

Cancer typebreastcolonlungoesophagusovarianpancreasprostaterenalstomach

Overall

Hazard ratio1.09 (0.8, 1.49)1 (0.77, 1.3)1.12 (0.89, 1.41)1.05 (0.69, 1.6)1.14 (0.63, 2.06)1.88 (1.09, 3.25)1.54 (1.13, 2.09)1.14 (0.52, 2.52)1.44 (0.76, 2.74)

1.18 (1.04, 1.33)

% weight14.8520.9325.78.174.274.9115.192.373.61

100

0 1 2 3 4

all Beta−blockers vs controls DIN−LINK

Cancer typebreastcolonlungoesophagusovarianpancreasprostaterenalstomach

Overall

Hazard ratio1.19 (1.03, 1.37)0.85 (0.74, 0.97)1.04 (0.91, 1.19)1.27 (1.01, 1.59)1.05 (0.74, 1.5)0.94 (0.74, 1.21)1.03 (0.92, 1.15)0.46 (0.26, 0.83)1.03 (0.78, 1.36)

1.01 (0.91, 1.13)

% weight14.8715.1115.3610.76.329.7616.272.898.71

100

0 1 2 3 4

all Beta−blockers vs controlsCPRD

Summary — β-blocker study

1. Different individual cancer HR’s and overallconclusions

2. Important differences in some cohort statistics

3. Differences remain after correcting to give the samepatient:practice

4. Differences remain after reducing the size of theCPRD study

5. Databases appear to be demographically similar (Careyet al. 2004)

WHY?

”an experiment is reproducable until anotherlaboratory tries to repeat it.” — Alexander Kohn

• Artifact of differences in GP computer systems?DIN-LINK uses Torex/iSoft systems (SeeKontopantelis et al 2013)

• Analysis methods?

– No matching - potential confounding– No clustering by practice– Limited control for covariates– Is meta-analysis the most appropriate method

(Assumes independence)?

• Data quality?

Conclusions / recommendations

These replications add to the evidencethat PCD results are valid

• PCD Replication is hard!

– Methods details are inadequate forreplication

– Clinical codes not provided withthe original article

– Relies on active cooperation ofauthors of original studies

– Even then, ambiguity can remain

• Publish full methods (in onlineappendix?)

• Publish full clinical code lists...

ClinicalCodes.org

Thanks. . .

Research team

David ReevesEvan KontopantelisIvan OlierDarren Ashcroft

Authors of the originalstudies

Iain Carey (St. GeorgesUniversity, London)Carol Coupland(University ofNottingham)

Contact: [email protected]

Using primary care databases to evaluate drug benefits and harms: are the results replicable and...

Health & Medicine

Transcript of Using primary care databases to evaluate drug benefits and harms: are the results replicable and...