O

xy

z

xbdb

dt

γ

nv

θ

xt

np

hp

ng

α Hf

ground plane

image plane(inverse) gravity

ground plane orientation

ground plane height

object vertical orientation

real world heightobject depthcamera center

focal length

object pitch and roll angles

object landmarks

ground plane

inverse gravity

√

vertical surface candidate 1

vertical surface candidate 2

vertical surface candidate 1

vertical surface candidate 2

ground plane

vertical surface candidateinverse gravity

object candidate

√

√

√

√

√

X

Local 3D context

Global 3D context

Best hypothesis

(inverse) gravity

ground plane orientation valid

valid invalid (#1)

invalid (#2)

invalid (#1)invalid

(#1)

incompatible (#3)

incompatible (#4)

ground plane

hiv hjv

vv

)()( jikhj

khi

k vvnv

)(kv

k vnv

hiv hjv

)()( jikhj

khi

k vvnv

(a)

(b)

object maskback-projected mask

occlusion mask

After object is detected:

closer object

farther object

closer object

farther object

occlusion mask of the farther object

intersection region of the two object masks

w

l

β

pn~

ph~

ground plane

vertical surface candidate

object candidate

inverse gravity

supporting plane 1

supporting plane 2

23

45 6

1

Generate object/surface

candidates

Propose global 3D geometry

hypotheses by object/surface

candidates

Evaluate each hypothesis by

inferring validity of object/surface

candidates

Select the best hypothesis and its associated validity of object/surface

candidates

O

xy

z

xbdb

dt

γ

nv

θ

xt

np

hp

ng

α Hf

ground plane

image plane(inverse) gravity

ground plane orientation

ground plane height

object vertical orientation

real world heightobject depthcamera center

focal length

object pitch and roll angles

object landmarks