Lighting and Photometric Stereocseweb.ucsd.edu/classes/wi06/cse252a/lec7.pdfPhotometric Stereo with...

1

CS252A, Winter 2006 Computer Vision I

Lighting andPhotometric Stereo

Computer Vision ICSE252ALecture 7


Photometric Stereo

• HW2 will be on web later today


Last lecture in a nutshell

CSE 252A, Winter 2006βα

δδπ

δδ coscos

'4'3

2

AAL

zd

APE ⎟

⎠⎞

⎜⎝⎛== Image Irradiance

Radiometry of thin lensesRadiometry of thin lenses

2

'coscos

'⎟⎠⎞

⎜⎝⎛=

zz

AA

βα

δδ

( )3

2

2

2

cos4cos/

cos4

απααπ

⎟⎠⎞

⎜⎝⎛==Ω

zd

zd

βαδπβδδ coscos4

cos 32

ALzdALP ⎟⎠⎞

⎜⎝⎛=Ω=

LzdE

⎥⎥⎦

⎤

⎢⎢⎣

⎡⎟⎠⎞

⎜⎝⎛= απ 4

2

cos4

E: Image irradianceL: emitted radianced : Lens diameterZ: depthα: Angle of patch from optical axis

( ) ( )22 cos/cos

cos/'cos'

αβδ

ααδδω

zA

zA

==

δA

δA’

CSE 252A, Winter 2006

BRDFBRDFWith assumptions in previous slide• Bi-directional Reflectance

Distribution Function ρ(θin, φin ; θout, φout)

•• Ratio of incident irradiance to Ratio of incident irradiance to emitted radianceemitted radiance

• Function of– Incoming light direction:

θin , φin– Outgoing light direction:

θout , φout

n̂(θin,φin)

(θout,φout)

( ) ( )( ) ωθφθφθφθφθρ

dxLxLx

ininini

outoutooutoutinin cos,;

,;,;,; =


2


Light sources, shading, shadowsLight sources, shading, shadows• Lighting

– Nearby• Point Sources• Line sources• Area sources• Function on 4-D space

– Distant• Point sources• Function on sphere• Spherical Harmonics

• Shadows


Standard nearby point source modelStandard nearby point source model

• N is the surface normal• ρ is diffuse (Lambertian)

albedo• S is source vector - a

vector from x to the source, whose length is the intensity term

– works because a dot-product is basically a cosine

( ) ( ) ( )( ) ⎟

⎟⎠

⎞⎜⎜⎝

⎛ •2xr

xSxNxdρ

NS


Line sources Line sources

radiosity due to line source varies with inverse distance, if the source is long enough


Area sourcesArea sources• Examples: diffuser boxes,

white walls.• The radiosity at a point due to

an area source is obtained by adding up the contribution over the section of view hemisphere subtended by the source

– change variables and add up over the source

• See Forsyth & Ponce or a graphics text for details.


Standard distant point source modelStandard distant point source model

• Assume that all points in the scene are close to each other with respect to the distance to the source. Then the source direction doesn’t vary much over the scene, and the distance doesn’t vary much either. We can roll the constants together to get:

( ) ( )( )SxNxd •ρN

S


Illumination in terms of Spherical Illumination in terms of Spherical HarmonicsHarmonics

m=-1m=-2 m=0 m=1 m=2

l=0

l=1

l=2

.

, ( , )l mY θ ϕ

xy z

xy yz 23 1z − zx 2 2x y−

1

Band

, ,0

( , ) ( , )l

l m l ml m l

L L Yθ φ θ φ∞

= =−

=∑ ∑

Express lighting L as sum spherical harmonics Yl,m(θ,Φ)

3


Shadows cast by a point sourceShadows cast by a point source

• A point that can’t see the source is in shadow• For point sources, the geometry is simple

Cast Shadow

Attached Shadow


Area Source Shadows

1. Fully illuminated2. Penumbra3. Umbra (shadow)

CS252A, Winter 2006 Computer Vision IFigure from “Mutual Illumination,” by D.A. Forsyth and A.P. Zisserman, Proc. CVPR, 1989, copyright 1989 IEEE

At the top, geometry of a gutter with triangular cross-section; below, predicted radiositysolutions, scaled to lie on top of each other, for different albedos of the geometry. When albedo is close to zero, shading follows a local model; when it is close to one, there are substantial reflexes.


Irradiance observed in an image of this geometry for a real white gutter.

Figure from “Mutual Illumination,” by D.A. Forsyth and A.P. Zisserman, Proc. CVPR, 1989, copyright 1989 IEEE


Photometric Stereo


Shading reveals 3-D surface geometry

4


• Shape-from-shading: Use just one image to recover shape. Requires knowledge of light source direction and BRDF everywhere. Too restrictive to be useful.

• Photometric stereo: Single viewpoint, multiple images under different lighting.

1. Arbitrary known BRDF, known lighting2. Lambertian BRDF, known lighting3. Lambertian BRDF, unknown lighting.


An example of photometric stereo

Input ImagesInput Images

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

Photometric Stereo:General BRDF

and Reflectance Map


Coordinate system

x

y

f(x,y)

Surface: s(x,y) =(x,y, f(x,y))Tangent vectors:

⎟⎟⎠

⎞⎜⎜⎝

⎛∂∂

=∂

∂

⎟⎠⎞

⎜⎝⎛

∂∂

=∂

∂

yf

yyxs

xf

xyxs

,1,0),(

,0,1),(Normal vector

⎟⎟⎠

⎞⎜⎜⎝

⎛−

∂∂

∂∂

=

×=∂∂

×∂∂

=

1,,yf

xf

ssys

xs

yxn


Coordinate system

x

y

f(x,y)

Normal vectorT

yf

xf

ys

xs

⎟⎟⎠

⎞⎜⎜⎝

⎛−

∂∂

∂∂

=∂∂

×∂∂

= 1,,nyfq

xfp

∂∂

=∂∂

= ,

Gradient Space: (p,q)

n

( )Tqpqp

1,,1

1ˆ22

−++

=n

5


Image Formation

For a given point A on the surface, the image irradiance E(x,y) is a function of

1. The BRDF at A 2. The surface normal at A3. The direction of the light source

ns

.

a

E(x,y)

A


Reflectance Map

Let the BRDF be the same at all points on the surface, and let the light direction s be a constant.

1. Then image irradiance is a function of only the direction of the surface normal.

2. In gradient space, we have E(p,q).

ns

a

E(x,y)


Example Reflectance Map: Lambertian surface

For lighting from front

E(p,q)


Light Source Direction, expressed in gradient space.


Reflectance Map of Lambertian Surface

What does the intensity (Irradiance) of one pixel in one image tell us?It constrains normal to a curve

E.g., Normal lies on this curve


Two Light SourcesTwo reflectance maps

Emeasured2

Emeasured1

A third image would disambiguate match

6


Reflectance Map of Lambertian + Specular Surface


Photometric stereo: Step1

1. Acquire three images with known light source direction.

2. Using known source direction & BRDF, construct reflectance map for each direction.

3. For each pixel location (x,y), find (p,q) as the intersection of the three curves.

4. This is the surface normal at pixel (x,y). Over image, this is normal field.


Normal Field


Plastic Baby Doll: Normal Field


Next step:Go from normal field to surface


Recovering the surface f(x,y)Many methods: Simplest approach1. From estimate n =(nx,ny,nz), p=nx/nz, q=ny/nz2. Integrate p=df/dx along a row (x,0) to get f(x,0)3. Then integrate q=df/dy along each column

starting with value of the first row

f(x,0)

7


What might go wrong?

• Height z(x,y) is obtained by integration along a curve from (x0, y0).

• If one integrates the derivative field along any closed curve, on expects to get back to the starting value.

• Might not happen because of noisy estimates of (p,q)

∫ ++=),(

),(00

00

)(),(),(yx

yx

qdypdxyxzyxz


What might go wrong?

yf

xxf

y ∂∂

∂∂

=∂∂

∂∂

xq

yp

∂∂

=∂∂

Integrability. If f(x,y) is the height function, we expect that

In terms of estimated gradient space (p,q), this means:

But since p and q were estimated indpendentlyat each point as intersection of curves on three reflectance maps, equality is not going to exactly hold


Horn’s Method[ “Robot Vision, B.K.P. Horn, 1986 ]

• Formulate estimation of surface height z(x,y) from gradient field by minimizing cost functional:

where (p,q) are estimated components of the gradient while zx and zy are partial derivatives of best fit surface

• Solved using calculus of variations – iterative updating

• z(x,y) can be discrete or represented in terms of basis functions.

• Integrability is naturally satisfied.

dxdyqzpz yx22

Image

)()( −+−∫∫


We stopped here, but rest of slides are included for those looking to get a head start on the next assignment.


II. Photometeric Stereo:Lambertian Surface,

Known Lighting


Lambertian Surface

At image location (u,v), the intensity of a pixel x(u,v) is:

e(u,v) = [a(u,v) n(u,v)] · [s0s ]= b(u,v) · s

where• a(u,v) is the albedo of the surface projecting to (u,v).• n(u,v) is the direction of the surface normal.• s0 is the light source intensity.• s is the direction to the light source.

n̂ŝ

^ ^

a

e(u,v)

8


Lambertian Photometric stereo• If the light sources s1, s2, and s3 are known, then

we can recover b from as few as three images. (Photometric Stereo: Silver 80, Woodham81).

[e1 e2 e3 ] = bT[s1 s2 s3 ]

• i.e., we measure e1, e2, and e3 and we know s1, s2, and s3. We can then solve for b by solving a linear system.

• Normal is: n = b/|b|, albedo is: |b|[ ][ ] 1321 −= 321T sssb eee


What if we have more than 3 Images?Linear Least Squares

[e1 e2 e3 ] = bT[s1 s2 s3 ]

Rewrite as e = Sb

wheree is n by 1b is 3 by 1S is n by 3

Let the residual ber=e-Sb

Squaring this: r2 = rTr = (e-Sb)T (e-Sb)

= eTe - 2bTSTe + bTSTSb

(r2)b=0 - zero derivative is a necessary condition for a minimum, or-2STe+2STSb=0;

Solving for b gives

b= (STS)-1STe


Input Images


Recovered albedo


Recovered normal field


Surface recovered by integration

9


Lambertian Photometric Stereo


Reconstruction with albedo map


Without the albedo map


Another person


No Albedo map


III. Photometric Stereo with unknown lighting and Lambertian surfaces

Covered in Illumination cone slides

Lighting and Photometric Stereocseweb.ucsd.edu/classes/wi06/cse252a/lec7.pdfPhotometric Stereo with...

Documents

Transcript of Lighting and Photometric Stereocseweb.ucsd.edu/classes/wi06/cse252a/lec7.pdfPhotometric Stereo with...