Barycentric Differential

Partial differential = relationship between two coordinate systemschange in barycentric coordinate related to change in screen space

-1-

Screen space (x, y) Barycentric space (β, γ)

𝝏 𝜷𝝏 𝒙…

𝜕 𝛽𝜕 𝑦

𝜕 𝛽𝜕 𝑥

Detecting Edge

-2-

If any test of four corner is failed, it is edge!Sampling more rays for the case

Edge Detection Example

Geometric edge detection, not visual edge Recklessly modeled object

-3-

Image Comparison (0-depth)

-4-

1X non-sample 4x proposed 4X super-sampling

Image Comparison (4-depth)

-5-

1X non-sample 4x proposed 4X super-sampling

Performance Comparison

-6-

1x 4x ours 4x SS 8x ours 8x SS0

5000000

10000000

15000000

20000000

25000000

# of rays (0 depth)

PrimaryShadow

1x 4x ours 4x SS 8x ours 8x SS0

5000000

10000000

15000000

20000000

25000000

30000000

35000000

# of rays (4depth)

PrimaryShadowReflectionRefraction

1x 4x-ours 4x-ss 8x-ours 8x-ss

0-depth 6.54 11 25.68 16.69 50.2

4-depth 11.05 18.82 42 29.3 82.13

• Rendering Time

• # of Rays

Todos

논문명 제출처 상태

A Latency-efficient Stencil/depth Buffer System for  Mobile 3-D Rendering Processors

IEICE Express On review

An Effective Depth Data Memory System Using an Escape Count Buffer for 3D Rendering Processors 

IEICE Express Accepted

무게 중심 좌표계의 편미분을 이용한 광선추적기법의 텍스처 MIPMAP

레벨 선택 알고리즘게임학회 논문지 계획 중

-7-

논문 작성 계획– Adaptive antialiasing using ray differentials

• 추가 실험 필요 , PBRT 적용 학위 논문 작성

2011/06Jeongsoo Park

The Graduate SchoolYonsei University

Department of Computer Science

The Design of Acceleration Algorithm for Real-time Ray Tracing

Agenda

Introduction– Benefit of Ray Tracing– Why today’s issue?– Contributions

Background & Related Work– Basics of Ray Tracing– Acceleration technique

Ray Differentials for Adaptive Level-of-Detail– Derivation of ray differential– Ray-diff for texture filtering– Ray-diff for adaptive antialiasing

Probability Heuristics for Ray Tracing Acceleration– Traditional SAH– Surface Probability Heuristics

Conclusion & Future Work

-9-

Introduction

Application of computer graphics is getting wider

Quality of computer graphics is getting real and faster

-10-

Toy story, 1995, 11 hours Crysis, 2011, 30 ms

Game Movie Medical AR/VR

Introduction (cont.)

Ray tracing is capable of high degree of realistic image– Support reflection, refraction, caustics naturally– Calculate physically-correct lighting effect– Simulate interference between primitives in the scene

-11-

Background (cont.)

Ray tracing is basically “SEARCH” algorithm– Rasterization sorts the primitive into 2-D grid of buckets(pixels)

-12-

Processing PRIMITIVES to SORT onto image plane

Z-buffered rasterization

Processing RAYS to SEARCH contributing objects

Ray tracing

Ray differential unit for antialiasing

Introduction– Ray tracing is regular sampling texture aliasing– Solution• Stochastic strategy

More rays per pixel, EXPENSIVE (Distribution ray tracing, Path tracing)• Filtering strategy

Filtering out the area within a pixel or using pre-filtered texture How to know minification ratio

-13-

Ray differential unit for antialiasing (cont.)

Related works– Distance based [BMRT1996]• Ray is linearly diverging – twice the distance, twice the difference• Used in RenderMan [Apodaca1999]• Only valid for PRIMARY RAY• Not support reflection/refraction of ray

-14-

Ray differential unit for antialiasing (cont.)

Tracing ray differential– Partial derivatives of ray’s starting point (P) and directions (D)– Difference of P and D in accordance with that of x and y.

-15-

𝜕 𝑥

𝑃 (𝑥 , 𝑦)

𝑃 (𝑥+𝜕𝑥 , 𝑦 )

𝐷(𝑥 , 𝑦 )

𝐷(𝑥+𝜕 𝑥 , 𝑦 )𝝏 𝑷 ′ ′𝝏𝒙

𝝏 𝑫 ′ ′𝝏 𝒙

𝝏 𝑷 ′𝝏 𝒙

𝝏 𝑫 ′𝝏 𝒙

Ray differential unit for antialiasing (cont.)

For primary ray

-16-

View

Right

Up

Differentiating

Ray differential unit for antialiasing (cont.)

For reflection ray

-17-

Differentiating

ND Phit

Drefl

Ray differential unit for antialiasing (cont.)

For refraction ray

-18-

Differentiating

ND Phit

Drefr

Ray differential unit for antialiasing (cont.)

Proposed differential unit operates in three modes (raytype)

-19-

Ray differential unit for antialiasing (cont.)

Proposed differential unit operates in three modes– Similar pipeline process ( 1DIV= 2MUL)

-20-

Primary ray

DOT3 (1)

DIV (1)

DOT3(2)

SQRT (1)

MUL (3)

MUL (12)

ADD (6)

MUL (1)

MUL (1)

P1

P2

P3

P4

P5

P6

P7

P8

P9

P10

P11

P12

P13

Reflection ray

DOT3 (1)

MUL (1)

MUL (3)

MUL (1)

MUL (3)

ADD (6)

DOT3(1)

Refraction ray

DIV (1)

MUL (1)

MUL (1)

ADD (1)

MUL (2)

MUL (12)

ADD (6)

DOT3 (1)

DOT3(2)

Ray differential unit for antialiasing (cont.)

Application of ray differential (Texture filtering)– Calculate length of projected area with• R1= , R2 =

– Level of texture• LOD = [Heckbert1983]

-21-

Ray differential unit for antialiasing (cont.)

Image comparison– Distance method excessively blurs (blue)

,or does not filter (red)

-22-

Distance Method Ray Differential MethodBase Level (l=0 always)

Base Level (l=0 always) Distance Method Proposed method

Ray differential unit for antialiasing (cont.)

Image comparison (cont.)– Distance method could not support convergence/divergence of ray– Proposed method selects appropriate level of filtered texture

-23-

Base Level (l=0 always) Distance Method Proposed method

Surface Probability Heuristics

Introduction– If ray tracing is a SEARCH algorithm, pre-calculated indexing is desirable– Acceleration structure is index of geometry distribution• Spatial - GRID[Klimaszewski1997], KD-tree[Fussel1998]• Object - BVH[Goldsmith1987]

– KD-tree is known as the best AS for rendering speed [Havran2000]• Redundant primitive reference

-24-

Surface Probability Heuristics (cont.)

Related works– Object median[Goldsmith1987]

– Spatial median• Bad for skewed distribution

– Surface Area Heuristics [Mac-donald1990]• Split minimizing a cost function• Empty space cut, but sweeping

– Hybrid approach[Shevtsov2007]• Upper-level : median• Lower-level : SAH

-25-

☞Proposed division scheme is enhancement of SAH

Surface Probability Heuristics (cont.)

Surface Area Heuristics– Probability that arbitrary ray goes through space VL

=

– Then, cost of traversing this node

-26-

KT = Cost of visiting this nodeKI = cost of intersection testTL|R = Number of primitives

V

 

 

 

 

 

   VL VR

 

 

 

 

 

p

  

 

    

Surface Probability Heuristics (cont.) Proposed scheme

– Calculate all probabilities of inbound and outbound sub-surface

– Probability of ray come IN with ‘a’

– Probability of ray go OUT with ‘b’

– Probability of ray hit left

…

– Probability of hit ‘both’ (left to right)

)…

-27-

a e

b f

c g

d h

i j

development figure

   

a

b

e

f

j

☞

Surface Probability Heuristics (cont.)

Experimental result– ~ 82.3% reduction in tree size

– ~69.3% reduction in primitive redundancy

-28-

kitchen sponza0

200000400000600000800000

10000001200000140000016000001800000

# prim. index

SAHProposed

kitchen sponza0

100000

200000

300000

400000

500000

600000

700000

Tree size

SAHProposed

Surface Probability Heuristics (cont.)

Experimental result– ~ 15.8% reduction in tree size

– 3% overhead in rendering performance

-29-

kitchen sponza020406080

100120140160180200

# traversal

SAHProposed

kitchen sponza0

20

40

60

80

100

120

rendering time

SAHProposed

Conclusions

This research focuses on the research of algorithm design for memory reduction and visual improvement for real-time ray trac-ing – Effective filtering kernel estimator by ray differential– Hardware design that switch it operating mode in accordance with type of

ray– Advanced heuristics that reduces required memory for acceleration struc-

ture

-30-

BACKUP

-31-

The Design of Acceleration Al-gorithm for Real-time Ray Trac-

ing

Jeong-soo Park

2011/05/31Media System Lab.Yonsei Univ.

* 목차

1. Introduction– Benefit of Ray Tracing

– Why today’s issue?

– Contributions

2. Background & Related Work– Basics of Ray Tracing

– Technical Categorization

• RT execution platform

• Acceleration technique

3. Ray Differentials for Adaptive Level-of-Detail– Derivation of ray differential

– Ray-diff for texture filtering

– Ray-diff for adaptive antialiasing

4. Probability Heuristics for Ray Tracing Acceleration

5. Simulation Environment for Graphics Acceleration Hardware

6. Conclusion & Future Work

Introduction

* Benefit of Ray Tracing

• High degree of realistic image

– Support reflection, refraction, caustics naturally

– Calculate physically-correct lighting effect

– Simulate interference between primitives in the scene

35

Ray tracing is one of global illumination algorithm to synthesize high degree realistic images as well as to

process 3D scene more efficiently than Z-buffer algorithm

Image of nVidia. :Optix : INTERACTIVE RAY TRACING ON NVIDIA QUADRO PROFESSIONAL GRAPHICS SOLUTIONS,” developer.nvidia.com, 2009

* Barrier of Ray Tracing

• Computation

– All operations are based on real

numbers

– High amount of computations

• For example:– Image resolution : 1280 x 720,

Scene size : 100K primitives, 10

propagations

– 1280 x 720 x 100K x 10 = 858G

times intersection test and shading

36

However ray tracing has several barriers to achieve in real-time performance, which are its types and amount of

computations.

objectresolution

R N

propagation

D

Color O(#pixel * #obj * #propagation)

* Why Today’s Issue

• Hardware performance growth

– Raw performance of CPU/GPU in-

creased

• GPU as massive parallel architec-

ture

– More parallelism exploited

• Even CPU starts to expand parallel-

ism

• Ray tracing algorithm is highly

parallel

37

Even so, there are several trends that realizes real-time performance of ray tracing.

*Nvidia developer center

Parallel architect`ure in commodity PC*

*Larrabee architecture

*Intel, “Larrabee: A Many-Core x86 Architecture for Visual Computing,” SIGGRAPH 2008

* Why Today’s issue

• Industrial demands for advanced, realistic graphics

– In addition to be getting complex, it becomes more real-time

38

Even so, there are several trends that realizes real-time performance of ray tracing.

Expendable(1999)Quake 3(1999) Half-Life 2(2004)

Real-time Rendering (Game)Crysis 2(2011)

A Bugs Life(1998)Toy Story(1995) Car(2006)Finding Nemo(2003)

TextureMapping

BumpMapping HDRI

RayTracing

High Quality Rendering (Movie)

Background & Related Work

* Basics of Ray Tracing

• Ray tracing processes RAYS to SEARCH corresponding primitives

– Basically searching algorithm

– Z-buffered algorithm processes primitives to sort its projective position

Ray tracing algorithm is essentially searching algorithm that finds the primitive that contributes to the screen

Processing PRIMITIVES to SORT onto image plane

Processing RAYS to SEARCH contributing objects

Z-buffered rasterization Ray tracing

40

* Acceleration Algorithms

There are several works for accelerating ray tracing to real-time performance.

41

• Ray tracing Acceleration works

– Acceleration structure : pre-indexing primitives with auxiliary data structure

– Core acceleration : optimize efficiency of RT core algorithm (traversal & intersection

test)

– Execution platform : exploiting hardware resources efficiently Search Algo-

rithmRay tracing

Acceleration Structure

• Grid[Kalojanov11], BVH[Lauterbach09], KD-tree, • Fast build/update[Kalojanov09, Pantaleoni10]

Core Acceleration • MLRTA[Reshtov05], Packet[Boulos08] • Fast IST[Havel09]

Platform • CPU/GPU• GPU[Parker10, Aila09], Multi-core[Wald10, Seiler08]

• GPU build [Lauterbach09]

Ray Differentials for Adaptive

LOD

* Introduction

• Regular sampling of ray tracing aliasing problem

• Solution

– Stochastic sampling

• More rays per pixel, EXPENSIVE– Eg.) Distribution ray tracing, Path tracing

– Texture filtering (like rasterization)

• Filtering texels in a pixel, or choose from prefiltered texture map (MIPMAP)

• Filtering kernel is ray’s footprint on texture space

How to know ray’s footprint along ray traversal

Like other rendering algorithm, ray tracing is another regular sampling technique. It also suffers from aliasing

problem.

43

* Related works

• BMRT[Gritz96]

– Estimating difference based on ray

traversal distance

– Only valid for primary ray

– Could not support convergence/di-

vergence of ray ( lens effect )

• Tracing ray differential[Igehy99]

– Derivatives of world coordinate po-

sition

– Used for geometry subdivision in

offline renderer [Christensen03,

Christensen06]

Several works has been announced to eliminate aliasing problem in texture mapping

44

* Tracing Ray Differential

• Essentially tracing derivatives of ray’s starting point (P) and its direction (D)

– Keep track of whether ray is expanding or shrinking

45

𝜕 𝑥

𝑃 (𝑥 , 𝑦)

𝑃 (𝑥+𝜕𝑥 , 𝑦 )

𝝏 𝑷 (𝒙 , 𝒚 )

𝝏 𝑫 (𝒙 , 𝒚 )𝐷(𝑥 , 𝑦 )

𝐷(𝑥+𝜕 𝑥 , 𝑦 )

* Derivation of Ray Differential

Ray generation Ray differential

Primary ray

Reflection Ray

Refraction Ray

Barycentric Coor-dinate

Ray differential is partial derivatives of ray parameters of x, y. Hence ray differentials for each types of ray

could be found by differentiating ray generation equation.

46

* Ray differentials for texture filtering

• Longer edge (R1 or R2) is selected

for filtering kernel size

• For Mipmapping, appropriate level

of mipmap is calculated as:

47

size =

𝒍𝒆𝒗𝒆𝒍=(𝒊𝒏𝒕 ) log𝟐(𝒔𝒊𝒛𝒆)

* Ray differential pipeline design

• Ray differential unit needs 3 DOT3, 17 MUL, 10 ADD, 1 DIV, 1 SQRT

48

At ray generation stage, differential unit calculates ray differential according to type of ray.

Primary ray

DOT3 (1)

DIV (1)

DOT3(2)

SQRT (1)

MUL (3)MUL (12)

ADD (6)

MUL (1)

MUL (1)

P1

P2

P3

P4

P5

P6

P7

P8

P9

P10

P11

P12

P13

P14

P15

Reflection ray

DOT3 (1)

MUL (1)

MUL (3)

MUL (1)MUL (3)

ADD (6)

DOT3(1)

Refraction ray

DIV (1)

MUL (1)

MUL (1)

ADD (1)

MUL (2)MUL (12)

ADD (6)

DOT3 (1)

DOT3(1)

* Result

49

Scene comparison (no filtering – distance based – ray differential )

BMRT Method Ray Differential MethodBase Level (l=0 always)

Base Level (l=0 always) BMRT Method Ray Differential Method

*

Base Level (l=0 always) BMRT Method Ray Differential Method

* Result

51

Scene comparison (no filtering – distance based – ray differential )

Base Level (l=0 always) BMRT Method Ray Differential Method

Conclusion & Future work

* Conclusion & Future works

• Ray differential could estimated size of ray footprint as the ray propagates

• By differentiating function of world-to-texture space transformation, it is possible

to calculate footprint on the texture

• Texture space footprint could be used to compute:

– Filtering kernel size

– Level of mipmap

• Adaptive supersampling by differential scheme

– Transform ray differential barycentric differential

53

Backup

* Benefit of Ray Tracing

• Benefit – High Image Quality

• Support reflection, refraction, caus-

tics naturally

• Calculate lighting effect globally

– Efficient processing

• Logarithmic behavior to scene

complexity

• Only process the primitive that con-

tributes to final image

55

Ray tracing is one of global illumination algorithm to synthesize high degree realistic images as well as to

process 3D scene more efficiently than Z-buffer algorithm

*Daniel Pohl, “Ray Tracing and Gaming,” PC perspective articles from Intel, Jan. 2008

Z-buffer algorithm

Ray Tracing

ComputationBehavior of graphics algorithm*

*

성능

사실성

56

현재 그래픽스 기술의 문제점

•지역 조명 기반으로 인한 사실감의 한계• 각 물체와 광원과의 관계만 고려

• Eg. 1’ 에 드리워지는 3’ 의 그림자 표현 불가•인접한 물체간의 위치 관계를 표현하지 못함

• 특정 물체를 처리할 때 다른 물체의 위치를 모름• Eg. 4 에 비친 1 의 영상을 표현 못함

•성능효율이 낮음• 비가시 영역에 대해 불필요한 렌더링작업 수행

• Eg. 3 에 의해 가려지는 2 의 영역까지 렌더링수행• 사실감 개선을 위한 중복 렌더링

• Eg. 5 를 다시 렌더링하여 1 에 대한 반사를 표현•물체의 수에 선형 복잡도를 가짐

• 장면을 구성하는 모든 물체 (N 개 ) 에 대해 렌더링 수행 (Lin-ear complexity: O(N) )

• 그래픽 application 의 영상 복잡도는 급격한 증가추세

5’

3

2

1

Graphics Pipeline

2

1’

3’

N 5

4

* Result

57

Scene comparison (no filtering – distance based – ray differential )

BMRT Method Ray Differential MethodBase Level (l=0 always)

Base Level (l=0 always) BMRT Method Ray Differential Method

* Global Illumination 기법의 장점

58

성능

사실성•실감있는 직접 조명 효과 재현

• 광선의 연쇄추적을 통해 실제와 유사한 빛의 반사 , 굴절 , 그림자를 재현 (simulation)

• Eg. 1 에서 광원에 대해 그림자 ray 추적 , 4 에서 1에 대한 반사 ray 추적

•간접조명 효과 재현 • 광원에서 생성되는 간접조명효과 지원

• Eg. 3 에 의한 간접조명 효과로 1 의 색상이 바뀜

•효율적인 렌더링 처리• 물체가 아닌 생성한 광선을 시스템에서 처리• 광선 역추적으로 현 시점에서 가시영역만 렌더링 수행

•계산 복잡도가 낮음• 공간상에 배치된 각 물체를 tree 구조화하여 물체 수 (N) 에

대해 Log(N) 의 복잡도를 가짐• 장면 복잡도 증가에 효과적으로 대응

1

23

1

Shadow ray

Eye ray

Reflection ray

Refraction ray

4

* Edge detection using ray differential

• For each sub-pixel, check conditions:

Barycentric coordinate differential could be used to estimate hit-point of neighbor rays

𝜕𝑃𝜕 𝑥

𝜕𝑃𝜕 𝑦

3 4

1 2

* Why today’s issue

• Hardware performance growth

– Raw performance of CPU/GPU in-

creased

• GPU as massive parallel architec-

ture

– More parallelism exploited

• Even CPU starts to expand parallel-

ism

• Ray tracing algorithm is highly

parallel

Even so, there are several trends that realizes real-time performance of ray tracing.

CE/ 표준

CE/ 표준

CE/ 표준Gflops growth

CPUGPU

Gtx 480

Gtx 200

G92

G80

Core2 XeonI7 980

* 목차

1. Introduction

– Benefit of Ray Tracing

– Why today’s issue?

– Contributions

2. Background & Related Work

– Basics of Ray Tracing

– Technical Categorization

• RT execution platform

• Acceleration technique

3. Ray Differentials for Adaptive LOD

– Derivation of ray differential

– Ray-diff for texture filtering

– Ray-diff for adaptive antialiasing

4. New Heuristics for Accelerating RT

5. Simulation Environment for Advanced Computer Graphics

6. Conclusion & Future Work

* 목차

• Introduction

• Background & Related Work

• Ray Differentials for Adaptive LOD

• New Heuristics for Accelerating RT

• Simulation Environment for Advanced Computer Graphics

• Conclusion & Future Work

* Result

63

Scene comparison (no filtering – distance based – ray differential )