Challenges and Advances in Large-scale DFT …on-demand.gputechconf.com/gtc/2014/webinar/gtc... ·...

72
Heather J. Kulik Assistant Professor, ChemE, MIT April 22, 2014

Transcript of Challenges and Advances in Large-scale DFT …on-demand.gputechconf.com/gtc/2014/webinar/gtc... ·...

Heather J. Kulik Assistant Professor,

ChemE, MIT April 22, 2014

 "  "

 "  "

energy! health!

catalysis!

Grand challenge: how do we harness and control energy to make useful products?!Computation allows us to understand known processes, predict and design new pathways.""

−2

2m∇2 +V (r )

#

$%

&

'(ψ(r ) = Eψ(r )

many-body Schrodinger equation!

Vext"

N,ρ"

E,!Ψ"the real!spatial density from single particle orbitals!

external potential!

The DFT reformulation:!

many-body wavefunction!

a “zoo” of XC functionals!

Kinetic energy!

Coulomb repulsion!

Pieces of our DF:!

Exchange- correlation (XC)

functional!

Number of atoms!

Classical N log(N)"

Empirical N2"

Semi-empirical N3"

DFT N3"

Correlated N5-N7"

ExactN!"

Accuracy!

1 10 100 1,000 10,000"

chemical accuracy!

relative rates!

may find TS!

Also, sampling!!before!

Terachem!DFT- O(N1.8)!

then (mid-2000s):!Beowulf clusters!

now:!GPU clusters!

DFT on a handful of atoms (three to ~100)!

DFT or better on three thousand atoms!!

TeraChem: see http://petachem.com"

1! 100! 10000! 1000000!

110!

168!

350!

900!

time (s)!

# at

oms!

CPU!GPU!

Novel architecture & GPU-optimized algorithms:!

I.S. Ufimtsev and T. J. Martinez J. Chem. Theory Comput. 5, 1004 (2009)."

183x"

62x"

33x"

13x"

(SS|SS), (SS|SP), (SS|PP), … , (DD|DD)

Reordering 2e integrals by type:"

Coulomb"repulsion"

µν | λσ( ) = χµ r1( )χν r1( ) 1r2 − r1

χλ r2( )χσ r2( )dr1 dr2∫∫

I.S. Ufimtsev and T.J. Martinez J. Chem. Theory Comput., 4, 222 (2008)."

Only need high accuracy DP for largest integrals"

DP

SP

µν | λσ( ) ≤ µν | µν( )1/2 λσ | λσ( )1/2Reordering 2e integrals by size:"

I.S. Ufimtsev and T. J. Martinez J. Chem. Theory Comput. 5, 1004 (2009)."

System size & complexity!

Getting the necessary physics!

Unsystematic errors!Energetics !

!!!!!

(self-interaction)!

Charge transfer!O OO O O O O O O

1 2 3 4

Bond rearrangement! Non-adiabatic processes!

relativistic effects, dispersion, and so on…!

Heterogeneity! Conditions!

•  Studying proteins with quantum mechanics!

"•  Mechanochemical depolymerization""•  Enzyme catalysis with a methyltransferase"

Why we are interested:!Force fields (MM) usually for proteins."BUT limitations remain:"1)  Charge transfer!2)  Bond rearrangement!3)  Polarization!

All are key for catalysis in enzymes!!"

Open questions: Can QM…!do well in cases for which force fields are optimized: prototypical structures? !

QM!MM!

QM/MM?!

Protein!only!

Less!than!30%!similarity!

No!ligands!or!modified!residues!

1!En:ty/chain!

5>35!aa!

q≤!±2!

70k"

15k"

6.7k"

4.7k"

413"

58"

Our protein test set selection method!

H.J. Kulik, N. Luehr, I.S. Ufimtsev, and T.J. Martinez JPCB 116, 12501 (2012)."

_ _ _

_

_ _ _ _

____ _ _

_ ___ _

_

_ _ ___

____ _ _

__ _ _

_ _ _ __

GAV L I MFWP S T CYNQ DEKRH020406080

100

AA

Fre

q. Human Genome_Total PDB_

non-polar polar chargedYes! Good correspondence to total PDB in primary structure:!

Underestimates: His, Cys." Overestimates: Gly/Ala, Trp."""His! Gly! Ala! Trp!Cys!

helix sheet none

non-polar polar charged

Secondary structure!

Sort of… We sample some helical and beta sheet secondary structure motifs:!

Our small peptides have much higher abundance of loop or disordered regions than is common in globular (i. e. large, natural) proteins. "

1AQG! 1CEK! 1EMZ! 1J4M! 1LB0! 1LB7! 1LBJ! 1LCX! 1LVQ!

1LVR! 1LVZ! 1MZI! 1O53! 1ODP! 1PJD! 1QLO! 1RIJ! 1T2Y!

1UAO! 1V46! 1Y03! 1Y49! 1YJP! 1YT6! 2AP7! 2CEH! 2CSA!

2E4E! 2EVQ! 2FBU! 2FXY! 2FXZ! 2I9M! 2JOF! 2JTA! 2JXF!

2K58! 2K59! 2KJM! 2KNP! 2KUX! 2KVX! 2NX6! 2NX7! 2OL9!

2ONW! 2OQ9! 2PJV! 2PV6! 2RLJ! 2RMW! 2RPS! 3E4H! 3FTK!

3FTR! 3FVA! 3NJW! 3NVG!

58 proteins from Protein Data Bank "5-35 residues in length

+2

q

-2

Normally treated with force fields, now can characterize whole proteins with DFT.

•  RHF, B3LYP, ωPBEh, BLYP functionals"•  STO-3G, 3-21g and 6-31g localized basis

sets"•  gas phase, PCM, and MM water-solvated

results"•  Optimized structures with 1) Amber ff03 force

field in AMBER or 2) DFT/RHF in TeraChem"

RHF!

BLYP!

B3LYP!

WPBEH!ω"

0% " " 20% " "40% " 60% " " 80% 100% """

convergence problems"converged"

This

talk"

convergence problems"converged"

Expt."

C.M. Isborn, N. Luehr, I.S. Ufimtsev, and T.J. Martinez JCTC 8 5092 (2012). "

MM QM0.00.40.81.21.6

MM QM Expt.0%5%

10%15%20%

MM QM Expt.02468

10

MM QM Expt.0%25%50%75%

100%Clashes Favor. Rama.

Poor RotamerRMSD

MM QM0.00.40.81.21.6

MM QM Expt.0%5%

10%15%20%

MM QM Expt.02468

10

MM QM Expt.0%25%50%75%

100%Clashes Favor. Rama.

Poor RotamerRMSD

MM QM0.00.40.81.21.6

MM QM Expt.0%5%

10%15%20%

MM QM Expt.02468

10

MM QM Expt.0%25%50%75%

100%Clashes Favor. Rama.

Poor RotamerRMSD

>0.4 Å!

Also from unexpected connectivity."

MM QM0.00.40.81.21.6

MM QM Expt.0%5%

10%15%20%

MM QM Expt.02468

10

MM QM Expt.0%25%50%75%

100%Clashes Favor. Rama.

Poor RotamerRMSD

Ramachandran!

Method! Cα-RMSD! Clash/1000! Poor Rot! Good Rama!

AMBER! 0.61! 3! 9%! 80%!RHF/STO-3G" 0.70" 40" 15%" 75%"RHF/3-21G" 0.68" 14" 11%" 86%"RHF/6-31G" 0.72" 8" 9%" 86%"Experiment! --! 9! 19%! 80%!

Reasoning: RMSD is Cα positioning only – not a significant basis-dependence, others more sensitive to treatment of O, N, S, etc. "

Beyond minimal basis set needed to describe sidechains and secondary structure with RHF:"

Method! Cα-RMSD! Clash/1000! Poor Rot! Good Rama!

AMBER! 0.61! 3! 9%! 80%!ωPBEh/MINI" 0.71" 75" 24%" 69%"ωPBEh/STO-3G" 0.77" 72" 19%" 71%"ωPBEh/3-21G" 0.69" 21" 15%" 81%"ωPBEh/6-31G" 0.63" 9" 13%" 85%"Experiment! --! 9! 19%! 80%!

More significant basis set dependence with ωPBEh than with RHF:"

Method! Cα-RMSD! Clash/1000! Poor Rot! Good Rama!

AMBER! 0.61! 3! 9%! 80%!RHF/MINI" 0.69" 45" 18%" 80%"RHF/MINI-D" 0.67" 44" 18%" 78%"Experiment! --! 9! 19%! 80%!

Inclusion of Grimme’s D3 empirical dispersion does not change outcome:"

Reasoning: peptides are too small, not enough ternary structure for dispersion to matter."

Best described by MM: prototypical structures

PDB ID: 1ODP

Best described by QM: less ordered structures

PDB ID: 3FTR

PDB ID: 1RIJ

PDB ID: 2RPS

PDB ID: 2I9M

QM!MM!

QM/MM?!

Disorder = 12Nres

unassigned−ss

Nres

+14NSS−int

NSS

+14Nres

atypical

Nres

Unhealthy residues"

Interruptions in secondary structure"

Residues with no/disordered secondary structure type "

Many possible definitions. One which covers key descriptors of disorder:"

Chen et al Acta. Crystall. D. (2010)."

clashing!

rotamers!

Ramachandran!

>0.4 Å!

RelativeHealth =HealthMM −HealthQM

HealthExpt

Molprobity scores compared for each protein: negative value means MM is better, positive means QM is better."

H.J. Kulik, N. Luehr, I.S. Ufimtsev, and T.J. Martinez JPCB 116, 12501 (2012)."

!

ZF = neutralized N and C termini."MMH2O = solvated in MM water."

Selected set of 20 “worst offender” proteins from original 58. Some of the clashing problem is fixed with neutralized termini but not with solvation."

Sidechain positioning is greatly improved but protons are still transferring"

ZF = neutralized N and C termini."MMH2O = solvated in MM water."

•  Studying proteins with quantum mechanics"

"•  Mechanochemical depolymerization!"•  Enzyme catalysis with a methyltransferase"

OPA: o-phthalaldehyde PPA: poly-o-phthalaldehyde

hydrolysis of end caps!

capping!

Uncapped: Tc=-50 °C" "Capped: Tc>100 °C"

Previously: remove endcap with chemical reaction/light: depolymerization.""Will mechanical bond scission in middle cause depolymerization? "

PPA90

PPA26

Polymers above MWmin undergo mechanical bond scission. ""26 kDa < PPA MWmin < 90 kDa!

Experimental conditions:!Dissolved in THF"Low-entanglement ~ 1mg/mL"NaOH to prevent acidic degradation"under Argon @ -15 °C""Pulsed ultrasound"-0.5s on/1.0 s off, 8.7 W/cm2"

"Gel filtration to identify product MWs.""

A

B -"+"

M.T. Ong, J. Leiding, H. Tao, A. M. Virshup, and T. J. Martinez JACS (2009)."

where Nattach is the number of APs (two in the following) and ni isa unit vector directed from the ith AP to its corresponding PP:

ni )ri

fix - ri

|rifix - ri|

(2)

The positions of the APs and PPs are denoted as ri and rifix,

respectively. The total force is then given as the vector sum of theab initio internal forces and the external force:

Ftotal ) Fab initio + Fext (3)

Here, we choose idealized fixed pulling points which areconsistent with forces that would act on the CB molecule embeddedin a polymer which was being elongated.

We have carried out AISMD simulations using the completeactive space self-consistent field (CASSCF) method9 with second-order perturbation theory corrections,10 i.e., CAS(4/4)-PT2, and the6-31G** basis set in a modified version of the AIMS-MOLPRO

code.11 Twenty trajectories were followed for each of the cis andtrans attachment cases. Representative results with an applied force

Figure 2. Force-modified potential energy surface for cyclobutene ring opening. Left panel shows the FMPES obtained by cis-pulling. Right panel showsthe FMPES obtained by trans-pulling. The blue spheres and arrows in the inset denote the fixed PPs and molecular APs, respectively. When there is noapplied force, the ring opening occurs through a conrotatory pathway, as predicted by the WH rules. At large applied forces, the ring opening occurs througha disrotatory pathway for cis-pulling (left panel) but through a conrotatory pathway for trans-pulling (right panel).

Figure 3. Optimized MEPs for disrotatory ring opening under a range ofexternal forces and cis-pulling. Superpositions of the reactant, transitionstate, and product geometries under a range of external forces are shownbelow (color scheme matches the one used in plotting the MEPs).

Figure 4. MEP for conrotatory and disrotatory ring opening in BCB withexternal force correction for a variety of applied forces. The cis-pullingcase is shown in the upper panel, and the trans-pulling case is shown inthe lower panel. Qualitative features are similar to those seen in CB.

6378 J. AM. CHEM. SOC. 9 VOL. 131, NO. 18, 2009

C O M M U N I C A T I O N S

MOLECULE fixed pulling point!(PP)!

attachment point!(AP)!

Fi!

Fext = Fi

riPP − ri

AP

riPP − ri

APi

AP

C.E. Diesendruck, G.I. Peterson, H. J. Kulik, J. A. Kaitz, B. D. Mar, P. A. May, S. R. White, T. J. Martinez, A.J. Boydston, and J. S. Moore Nature Chemistry (in press 2014)."

dimer!41 atoms!

trimer!57 atoms!

tetramer!73 atoms!

UB3LYP/6-31g calculations""Tetramer has 73 atoms.""2ps with 0.25 fs timestep @ 300K""Wigner initial conditions""Calculations on tetramer take 1-3 days:"

"8000 steps,""10s-6000s/timestep"

!

B3LYP&

Frame&7& Frame&12& Frame&44& Frame&45&HO

MO&

LUMO&

Mecha

nism

&

O OO O O

OO OO O

+ OO OO O

+

Occ:!2.00!

Occ:!0.00! Occ:!0.00! Occ:!0.00!

Occ:!2.00! Occ:!2.00! Occ:!1.00!

Occ:!1.00!

""

•  Studying proteins with quantum mechanics"

"•  Mechanochemical depolymerization""•  Enzyme catalysis with a

methyltransferase!

Cyclophilin A!

Non-local and dynamic"

?!

Local and static"

Chymotrypsin!J.S. Fraser,et al., Nature (2009).!J. Fastrez and A. R. Fersht, Biochemistry (1973).!

SAM!

catechol!

Mg2+"

Y68!E6!

W38"W143"

K144"

Human soluble form, 221 residues, ~3400 atoms."

1)  Remote residues influence catalysis."

2)  Methyl transfer is ubiquitous"

3)  Enzyme in humans (all tissues)"

4)  V108M polymorph key indicator of mental function"

5)  Target for antipsychotics and Parkinson’s"

1! 7! 10!

Model # Ats. Time (s) Reactants 63 7 Key res., Rct. 631 193 Key res., Rct. 995 554 Whole protein 3419 6233

J. Zhang and J. P. Klinman, JACS (2011)."HJK, J. Zhang, J. P. Klinman and T. J. Martinez (in preparation 2014)."

QM/MM models!

10111213141516

E a (kca

l/mol

)

0 10 20 30 40 50 60# QM Residues

08162432

Cov

. Cut

s

-4-3-2-101

Cha

rge

EaQM chargeCovalent cuts

+K144 +Y68 +E6+W38, W143

!

Chromophore excitation energies in PYP"

C.M. Isborn, N. Luehr, I.S. Ufimtsev, and T. J. Martinez JCTC 8 5092 (2012). "

GPUs help us to apply DFT to larger and more varied systems. ""TeraChem has been designed from the ground up to exploit scaling over GPU cores.""Our first results suggest practical DFT can fail in unusual ways. Big systems are hard to study!""However there’s a wide open frontier of work that can be done once DFT on a thousand atoms is routine.!

Lots of fun stuff ahead: http://hjklol.mit.edu"Acknowledgements:!""""""Funding: Burroughs Wellcome Fund """""""

My group at MIT!Tim Ioannidis"John La (UROP)"Dr. Niladri Patra"Natasha Seelam"Lisi Xie"

Todd J. Martinez!Martinez Group at Stanford!Prof. Christine Isborn (UC Merced)"Fang Liu"Brendan Mar"Dr. Lee-Ping Wang"Judith Klinman!Klinman Group at Berkeley!Jianyu Zhang"

TEST DRIVE K40 GPU - WORLD’S FASTEST GPU

Upload and run your own codes by remotely accessing a cluster

The GPU Test Drive is awesome! We were able to benchmark, gain valuable insight and significant performance improvement. A very big thank you for the opportunity.

” Richard Heyns, CEO of brytlyt, UK

www.nvidia.com/GPUTestDrive

UPCOMING GTC EXPRESS WEBINARS

April 23: CUDA 6 Features Overview

May 1: CUDA 6: Unified Memory

May 7: CUDA 6: Drop-in Performance Optimized Libraries

May 13: An Overview of AMBER 14 - Creating the World's Fastest Molecular Dynamics Software

Package

May 14: CUDA 6: Performance Overview

June 3: The Next Steps for Folding@home

www.gputechconf.com/gtcexpress

NVIDIA GLOBAL IMPACT AWARD

•  $150,000 annual award

•  Categories include: disease research, automotive safety, weather prediction

•  Submission deadline: Dec. 12, 2014

•  Winner announced at GTC 2015

Recognizing groundbreaking work with GPUs in tackling key social and humanitarian problems

impact.nvidia.com