Post on 20-May-2018
Accelerating Shallow Water Flow and Mass Transport Using Lattice Boltzmann Methods on GPUs
Kevin Tubbs
Dell | Global HPC Solutions Engineering
HPC Engineering
Background and Overview
3
Role of CFD
• Improve Environmental Predictions
• Used to Improve Decision and Policy Making
• Test and Evaluate Preventative Methods
HPC Engineering
Dell Global HPC Solutions Engineering
4 Confidential
Performance Analysis & Characterization - Node level - Cluster level
Power Consumption and Energy Analysis
Benchmark and Application Analysis - Industry Standard Benchmarks & Apps: HPL, NPB, NAMD - Focus Area Specific: CFD, Climate & Weather, Molecular Dynamics, etc.
HPC Engineering
Why Accelereyes ? Programmability
Faster
Slower
Time - Consuming Easy-to-use
Writing Kernels
Using Libraries
Instruction Set Optimization
Compiler Directives
HPC Engineering
Lattice Boltzmann Method Overview:
• Modeling Shallow Water and Mass Transport
Shallow Water Equation
( )1 ( )eqf f f ft τ
∂+ ⋅∇ = − −
∂c
LB Numerical Formulation
Advection Dispersion Equation for Solute Concentration
( ) 0i
i
huht x
∂∂+ =
∂ ∂
( ) ( )jij C
j j
hu ChC CD h Ft x x x
∂ ∂ ∂ ∂+ = + ∂ ∂ ∂ ∂
( ) ( )22( )2
i ji ii
i i j j
hu uhu huhg Ft x x x x
ν∂∂ ∂ ∂
+ = − + + ∂ ∂ ∂ ∂ ∂
HPC Engineering
• Boltzmann equation
• Particle distribution function
• EDF
• LBM Grid
Background: LBM
D2Q9 LBM
( , , )f tx e
( )( ) ( )
( ) ( )
0,0 0
1 1cos ,sin , 1, 2,3, 4
4 4
1 12 cos ,sin , 5,6,7,8
4 4
e
e
α
α
α π α πα
α π α πα
= − − = =
− − =
e
( )1 ( )eqf f f ft τ
∂+ ⋅∇ = − −
∂e
4 7 8
3
6 2 5
1 0
( )22
02 2 4 20
3 1 , 02 2
eq eq eqscf h f h fe e e e
ααα α α
α
ω α>
⋅⋅ ⋅= + + − > = −
∑u eu e u u
( )22
2 2 4 20
3 1 , 02 2
eq eq eqs hc h h hg C g hC ge e he he
αα αα α α α
α
ω α>
⋅⋅ ⋅= + + − > = −
∑u eu e u u
( , , )g tx e
HPC Engineering
• Zeroth Moment
• First Moment
• LBM evolves in a two-step process – Step 1. Collision
– Step 2. Streaming
Background: LBM (cont.)
h fαα=∑i ihu e fα αα=∑
( ) ( ) ( ) ( )( )
( )2
1ˆ, , , ,
,6
eq
i i
f t f t f t f t
t e F te
α α α α
α
τ= − −
∆+
x x x x
x
( ) ( )ˆ, ,f t t t f tα α α+ ∆ + ∆ =x e x
hC gαα=∑i iChu e gα αα=∑
HPC Engineering
Domain: 12 m x 12 m Dam Break Initial Conditions
Upstream: water depth 5 m Downstream: water depth 1m
BC: Free – Slip at walls for flow
2D – Dam Break: Backward Facing Step
t = 0.5 s
HPC Engineering
Domain: 200 m x 200 m Dam Break Initial Conditions
Upstream: water depth 10m Downstream: water depth 5m
BC: Free – Slip at walls for flow, Impermeable for concentration
2D Partial Dam Break
Water Depth Surface t = 7.2 s Velocity Vector Field t = 7.2 s
HPC Engineering
Problem Setup Domain: Infinite Shallow Water
Pool Constant Water Depth 1 m
2D – Continuous Release Mass Transport
( )( ) 11 2 12 1 2s Aτ τ
−= + −
Velocity Anisotropic Dispersion
( )2
1 2ij
sijA
hDc
t τ=∆ −
xx LD h κ= u yy TD h κ= u / 10L Tκ κ =
HPC Engineering
Domain: 800m x 200m Dam Break with
Transport Initial Conditions Upstream: water depth 10m Downstream: water depth 5m BC: Free – Slip at walls for flow
o Time Snap shots t = 10 s t = 30 s t = 60 s t = 90 s
2D – Dam Break with Mass Transport
HPC Engineering
Domain: 800 m x 200 m Dam Break with Transport Initial
Conditions Upstream: water depth 10m , Concentration 0.7 Downstream: water depth 5m, Concentration 0.2 BC: Free – Slip at walls for flow,
Impermeable for concentration
2D – Dam Break with Mass Transport
( ) i jij L ij L T
z
u u h gD k k k
Cδ
= + −
uu
5.93Lk =
0.23Tk =
o Time Snap shots t = 10 s t = 30 s t = 60 s t = 90 s
HPC Engineering
Summary
• LBM is a flexible algorithm for various CFD simulations
• Tools from Accelereyes allow rapid development and validation of LBM based CFD algorithms
• The GPU accelerated LBM algorithm achieved maximum of 24X speedup on GPU based architectures.
• The GPU parallel performance depends on size of simulation.
HPC Engineering
Resources
• Blogs
• Whitepapers
• HPC Advisor Online
• www.dell.com/gpu • www.dell.com/hpc • www.hpcatdell.com • www.DellHPCSolutions.com • http://www.hpcadvisorycouncil.com/best_practices.php
HPC Engineering
Related Work
Publications Tubbs, K. R., and F. T.-C. Tsai, GPU Accelerated Lattice Boltzmann Model for Shallow Water Flow and Mass Transport. Int. J. Numer. Meth. Eng (2010) DOI: 10.1002/nme.3066 http://blog.accelereyes.com/blog/2010/12/01/lattice_boltzmann_model/ Tubbs, K. R., and F. T.-C. Tsai, Multilayer Shallow Water Flow using Lattice Boltzmann Model with High Performance Computing. Advances in Water Resources, 32(12), 1767-1776, 2009. Presentations Tubbs, K. R., and F. T.-C. Tsai, A GPU Accelerated Lattice Boltzmann Model for Shallow Water Flow and Mass Transport, Sixth International Conference for Mesoscopic Methods in Engineering and Science (ICMMES), Guangzhou City, Guangdong Province, China, July 13-17, 2009. Proceedings Tubbs, K. R., and F. T.-C.. Tsai. Simulation of Multilayer Shallow Water Fluid Flow Using Lattice Boltzmann Modeling and High Performance Computing. ASCE/EWRI World Water & Environmental Resources Conference, Kansas City, Missouri, May 17-21, 2009. Tubbs, K. R., and F. T.-C.. Tsai, C. White, G. Allen. Lattice Boltzmann Modeling Using High Performance Computing for Shallow Water Fluid Flow and Mass Transport, American Geophysical Union 2008 Fall Meeting, San Francisco, CA, Dec 15-19, 2008. http://www.accelereyes.com/success_story/shallow_water_fluid_flow.php
Publications, Proceedings & Presentations: