Ø w-block(S-SVM):

Ø λ-block(loss-augmentedparametricenergyminimization):

LearningtoCombineMid-levelCuesforObjectProposalGenerationTomLee,Sanja Fidler,SvenDickinson

Ø AnovelParametricMin-Loss(PML)structuredlearningframeworkforparametricenergyfunctions.

Ø PMLlearnstopredictmultipleoutputsusinganovellossfunction.

Ø PMLbridgesthegapbetweenlearningandinferenceforparametricenergyfunctions.

Ø PMLisapplicabletoanydomainthatusesparametricenergyfunctions.

Contributions

Ø Objectproposalsreduceanexhaustivesetofhypothesestoafewplausiblecandidatesegments.

Ø Objectproposalsareoftenpredictionsfromparametricenergyfunctions (CPMC[2]etc.)

Ø Parametricenergyfunctionscanencoderelevantbottom-upgroupingcues[4].

Ø Butnopreviousapproachexistsforlearningtopredictmultipleoutputswithparametricenergyfunctions.

Motivation

100 200 300 400 500 600 700 800 900 10000.42

0.43

0.44

0.45

0.46

0.47

0.48

number of proposals

ave

rage b

est

ove

rlap

Segment Overlap by #Proposals, VOC’12 Vall

τ=1 (S−SVM)

τ=10

τ=20

τ=30

τ=40

τ=50

500 1000 1500 2000 2500 3000 3500 4000 45000.54

0.56

0.58

0.6

0.62

0.64

0.66

0.68

0.7

number of proposals

ave

rag

e b

est

ove

rla

p

Segment Overlap by #Proposals, VOC’12 Val

oursCPMCSelective Search

500 1000 1500 2000 2500 3000 3500 4000 4500 50000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

number of proposals

ave

rage b

est

ove

rlap

Segment Overlap by #Proposals, COCO’14 Val

oursMulticueSuperpixel ClosureMCG

Results

Ø WeachieveresultscomparablewithCPMC[2]andMCG[1]Ø Weoutperformmethodsthatlacklearning,e.g.SelectiveSearch[5]

Ø BiasenergytodifferentlocationsØ Maximumsuperpixel distance

Location- andcolor-baseddiversification Postprocessing

Ø Discardnon-maximumproposalsamongproposalswithhighoverlap.

Ø TrainSVMondeepfeaturestoassignanobjectnessscoretoeachproposal.Ø Biasenergytodifferentforeground-backgroundcolorpairs

Ø Gaussianmixturemodelofsuperpixel colors

Ø Theappearancecuediscouragesdivisionofsimilarcolorsandtextures:

Ø Theclosurecuediscouragesgapsalongboundaries:

Ø Thesymmetrycuediscouragesdivisionofsymmetricparts:

Ø Theenergyisnormalizedbyareabyafactorλ:

Ø Evaluatemultiplepredictedsegmentsagainstonecorrectgroundtruthsegment.

Ø Lossfunctionideallyexpressesa“min”:

Ø Innerlossfunctionmeasurestheerrorofasinglepredictedsegment:

Ø Upperboundforinnerlossfunction(hingeloss):

Ø Upperboundforlossfunction(min-hingeloss[3]):

Ø Regularizedtrainingobjective:Ø Nonnegativeweightsandnonnegative

λ coefficientsguaranteeasmallsetofsolutionsfromparametricmaxflow.

Ø Onepredictionforaspecificλ:

Ø Asetofpredictionsoverarangeofλ:

Parametricenergyfunction

Multiple-outputprediction

y = argminy

E

�(x,y,w)

ˆy(x,w) = argmax

ywT

�

�(x,y)

Y (x,w) = {y�(x,w) : � 2 [�1, 0]}

L(Y ,y) = miny2Y

`(y,y)

`(y,y(g)) =1

|g|X

p

|p|(vp yp = 0

1� vp yp = 1,

H(w) = min�2[�1,0]

h(w,�)

h(w,�) = max

y`(

ˆy,y) +wT�

�(x,

ˆy)�wT�

�(x,y)

minw

1

2||w||2 + C

N

NX

n=1

min�n2[�1,0]

hn(w,�n)

arg min�2[�1,0]

h(w,�)

w = argminw

1

2||w||2 + C

N

NX

n=1

hn(w,�n)

8� 2 [�1, 0],miny

�`(y,y)�wT�

�(x, y)

decreasingλ

Block-coordinatedescent

E

�(x,y) = E

app

(x,y) + E

clo

(x,y) + E

sym

(x,y) + E

�scale

(x,y)

ParametricMin-Losslearning

[1]Arbelaez etal.,CVPR2014. [4]Leeetal.,ACCV2014.[2]Carreira &Sminchisescu,PAMI2012. [5]Uijlings etal.,IJCV2013.[3]Guzman-Riveraetal.,NIPS2012.

Diversification

Learning