Computer Vision

23
Computer Vision Stereo Vision

description

Computer Vision. Stereo Vision. Pinhole Camera. Perspective Projection. Stereo Vision. Two cameras. Known camera positions. Recover depth. scene point. p. p’. image plane. optical center. Correspondences. p. p’. Matrix form of cross product. a =a x i +a y j +a z k. - PowerPoint PPT Presentation

Transcript of Computer Vision

Page 1: Computer Vision

Computer Vision

Stereo Vision

Page 2: Computer Vision

Bahadir K. Gunturk 2

Pinhole Camera

Page 3: Computer Vision

Bahadir K. Gunturk 3

Perspective Projection

' ' 'x y fx y z

Page 4: Computer Vision

Bahadir K. Gunturk 4

Stereo Vision

Two cameras. Known camera positions. Recover depth.

scene pointscene point

optical centeroptical center

image planeimage plane

p p’

Page 5: Computer Vision

Bahadir K. Gunturk 5

Correspondences

p p’

Page 6: Computer Vision

Bahadir K. Gunturk 6

Matrix form of cross product

20

0 0

y z z y z

z x x z z x

x y y z y x

a b a b a aa b a b a b a a b a b

a b a b a a

( ) 0( ) 0

a a bb a b

a×b=|a||b|sin(η)u a=axi+ayj+azk b=bxi+byj+bzk

Page 7: Computer Vision

Bahadir K. Gunturk 7

The Essential Matrix

( , ,1)' ( ', ',1)

T

T

p u vp u v

' 0Tp Ep

Essential matrix

Page 8: Computer Vision

Bahadir K. Gunturk 8

Stereo Constraints

X1

Y1

Z1O1

Image plane

Focal plane

M

p p’Y2

X2

Z2O2

Epipolar Line

Epipole

Page 9: Computer Vision

Bahadir K. Gunturk 9

A Simple Stereo System

Zw=0

LEFT CAMERA

Left image:reference

Right image:target

RIGHT CAMERA

Elevation Zw

disparity

Depth Z

baseline

Page 10: Computer Vision

Bahadir K. Gunturk 10

Stereo View

Left View Right View

Disparity

Page 11: Computer Vision

Bahadir K. Gunturk 11

Stereo Disparity The separation between two matching objects

is called the stereo disparity.

Page 12: Computer Vision

Bahadir K. Gunturk 12

Parallel Cameras

ZT

fZxxTlr

OOll OOrr

PP

ppll pprr

TT

ZZxxll xxrr

ff

T is the stereo baseline

rlxx

TfZ

rlxxd Disparity:

Page 13: Computer Vision

Bahadir K. Gunturk 13

Finding Correspondences

Page 14: Computer Vision

Bahadir K. Gunturk 14

Correlation Approach

For Each point (xl, yl) in the left image, define a window centered at the point

(xl, yl)LEFT IMAGE

Page 15: Computer Vision

Bahadir K. Gunturk 15

Correlation Approach

… search its corresponding point within a search region in the right image

(xl, yl)RIGHT IMAGE

Page 16: Computer Vision

Bahadir K. Gunturk 16

Correlation Approach

… the disparity (dx, dy) is the displacement when the correlation is maximum

(xl, yl)dx(xr, yr)RIGHT IMAGE

Page 17: Computer Vision

Bahadir K. Gunturk 17

Comparing Windows ==??

ff gg

MostMostpopularpopular

Page 18: Computer Vision

Bahadir K. Gunturk 18

Maximize Cross correlation

Minimize Sum of Squared Differences

Comparing Windows

Page 19: Computer Vision

Bahadir K. Gunturk 19

Correspondence Difficulties Why is the correspondence problem difficult?

Some points in each image will have no corresponding points in the other image.(1) the cameras might have different fields of view.(2) due to occlusion.

A stereo system must be able to determine the image parts that should not be matched.

Page 20: Computer Vision

Bahadir K. Gunturk 20

Structured Light Structured lighting

Feature-based methods are not applicable when the objects have smooth surfaces (i.e., sparse disparity maps make surface reconstruction difficult).

Patterns of light are projected onto the surface of objects, creating interesting points even in regions which would be otherwise smooth.

Finding and matching such points is simplified by knowing the geometry of the projected patterns.

Page 21: Computer Vision

Bahadir K. Gunturk 21

Stereo results

Ground truthScene

Data from University of Tsukuba

(Seitz)

Page 22: Computer Vision

Bahadir K. Gunturk 22

Results with window correlation

Estimated depth of field(a fixed-size window)

Ground truth

(Seitz)

Page 23: Computer Vision

Bahadir K. Gunturk 23

Results with better method

A state of the art methodBoykov et al., Fast Approximate Energy Minimization via Graph Cuts,

International Conference on Computer Vision, September 1999.

Ground truth

(Seitz)