Bridging Shape and Reflectance

Presented by: Jongwoo Lim, Feb 18 2003

Bridging Shape and Reflectance – p.1/23

Rendering vs Inverse-Rendering

Geometry

Reflectance Property

Lighting

Radiance MapsRendering

Geometry

Radiance Maps

Lighting

Reflectance PropertyInverse Rendering

Bridging Shape and Reflectance – p.2/23

Parametric BRDF’s

L = B · IRadiance = BRDF · Irradiance

Torrence-Sparrow model

BTS(θi, θo, α|ρd, ρs, σ) = ρd cos θi+ρs

cos θo

e−α2/2σ2

Ward model (isotropic version)

BW (θi, θo, δ|ρd, ρs, σ) =ρd

π+

ρs

4πσ2√

cos θi cos θo

e− tan2 δ/σ2

Bridging Shape and Reflectance – p.3/23

Renderings with Ward BRDF

Isotropic Anisotropic

from Simon Premoze’s homepage (http://www.cs.utah.edu/ premoze/brdf/)

Bridging Shape and Reflectance – p.4/23

BRDF Estimation

We need to estimate BRDF at every point on the object.

Knowns Unknowns

Radiance L

Irradiance I =⇒ BRDF parameters

Surface normal n (ρd, ρs, σ)

Light source direction s

Viewing direction v

Bridging Shape and Reflectance – p.5/23

Difficulties in Estimation

Estimation of specular parameters is difficult :

• The specular lobe is highly peaked

- it is difficult to observe specular points in images

• The specular term is not linear

- it requires many samples to estimate the parameter

Bridging Shape and Reflectance – p.6/23

Reflection Component Separation

We can infer the contribution of diffuse and specular part.

M = [MR MG MB ]

=

cos θi1 K(θo1, α1)

......

cos θiNK(θoN

, αN )

ρD,R ρD,G ρD,B

ρS,R ρS,G ρS,B

= [ GD GS ]

KtD

KtS

= G K

55 65 75 85 95 105image frame

0.0

50.0

100.0

150.0

inte

nsity

redgreenblue

55 65 75 85 95 105image frame

0.0

50.0

100.0

150.0

inte

nsity

diffuse reddiffuse greendiffuse bluespecular red

specular bluespecular green

Bridging Shape and Reflectance – p.7/23

Illumination Condition

Direct Illumination

• all reflected lights are from light sources

• only one reflection : light source - surface - camera

Global Illumination

• reflecting surfaces also work as light sources

• multiple reflections until being observed

Bridging Shape and Reflectance – p.8/23

Inverse Radiosity (Lambertian)

Assume Lambertian surfaces

Li = Ei + ρi

∑

j

Lj Fij

Li radiance (radiosity)

Ei emission (light source)

ρi albedo (BRDF)

Fij form factor between patches

ρi = (Li − Ei) / (∑

j

Lj Fij)

pi

j

F

L

ij

i

A

C

Ei

Lj

Bridging Shape and Reflectance – p.9/23

Inverse Radiosity (Parametric BRDF)

radiance = emission + diffuse + specular

LCvPi= ECvPi

+

+ ρd

∑

j

LPiAjFPiAj

+ ρs

∑

j

LPiAjKCvPiAj

pi

j

L L

L

P

C P

i j

A

A

v

C A

i

jk

C

C

v

k

Bridging Shape and Reflectance – p.10/23

Inverse Radiosity (Parametric BRDF)

Radiance from one point can vary with viewing directions.

LPiAj= LCkAj

− SCkAj+ SPiAj

= LCkAj+ ∆SCkPiAj

Iteratively estimate ∆S with initial guess ∆S = 0

Bridging Shape and Reflectance – p.11/23

Inverse Global Illumination Algorithm

Assume constant BRDF over each patch

• Detect specular highlights on surfaces (geometrically)

• Choose sample points inside & around each highlight

• Build links between sample points and patches

• Assign L0 with average radiance value, and ∆S = 0

• Iterate

• Update L using ∆S of each link

• Optimize each surface’s BRDF parameters

• Estimate ∆S with new BRDF parameters

Bridging Shape and Reflectance – p.12/23

Monte-Carlo Sampling

Ck

Aj

QPiAj

QCk Aj

Cv

iP

PiAjL

One-bounce approximation is enough for this purpose.

Bridging Shape and Reflectance – p.13/23

Diffuse Albedo Map

Assume constant specular property over each patch

ρd(x) = πDiffuse(x)

Irradiance(x)

Diffuse(x) = L(x) − Specular(x)

Give lower weight on sample

• which has large specularity

• whose viewing angle is grazing the surface

Bridging Shape and Reflectance – p.14/23

Geometry

So far, we assumed that the geometry is already given.

• input by hand

• measure from the object

• laser range finder, stereo vision, shape from X, . . .

color camera

light stripe range Þnder

robotic arm

light source

object

Bridging Shape and Reflectance – p.15/23

Geometry Acquisition Procedure

1. Surface acquisition from each range image

2. Alignment of range images

3. Retrieving 3D representation

• Merging based on a volumetric representation

• Isosurface extraction

(a) range image acquisition (b) alignment (c) merging (d) isosurface extraction

Bridging Shape and Reflectance – p.16/23

Surface Normal Estimation

Why is the surface normal important?

BRDF is evaluated at each surface points locally.For accurate estimation, precise θ, α are required.

The eigenvector of the covariance matrix of neighbor points

n = Null

(

∑

neighbor

(x − x)(x− x)t

) principal axis

Bridging Shape and Reflectance – p.17/23

Experiment Setup : Conference Room

Bridging Shape and Reflectance – p.18/23

Result : Conference Room

(a) Initial hierarchical polygon mesh, with radiances assigned from images.

(b) Synthetic rendering of recovered properties under original illumination.Bridging Shape and Reflectance – p.19/23

Result : Conference Room

(c) Synthetic rendering of room under novel illumination.

(d) Synthetic rendering of room with seven virtual objects added.Bridging Shape and Reflectance – p.20/23

Result : Whiteboard

Bridging Shape and Reflectance – p.21/23

Result : Mug

Synthesized object images

frame 50input synthesized frame 80input synthesized

Bridging Shape and Reflectance – p.22/23

Reference

• Y. Sato, M. D. Wheeler, K. Ikeuchi, Object Shape and ReflectaneModeling from Observation, SIGGRAPH, 1997

• Y. Yu, P. Debevec, J. Malik, T. Haukins, Inverse GlobalIllumination: Recovering Reflectance Models for Real Scenesfrom Photographs, SIGGRAPH, 1999

• P. Debevec, J. Malik, Recovering High Dynamic Range RadianceMaps from Photographs, SIGGRAPH, 1997

• P. Debevec, Rendering Synthetic Objects into Real Scenes:Bridging Traditional Image-based Graphics with GlobalIllumination and High Dynamic Range Photography, SIGGRAPH,1998

Bridging Shape and Reflectance – p.23/23