MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

31
MAP Estimation in Binary MRFs using Bipartite Multi-Cuts Sashank J. Reddi Sunita Sarawagi Sundar Vishwanathan Indian Institute of Technology, Bombay

description

MAP Estimation in Binary MRFs using Bipartite Multi-Cuts. Sashank J. Reddi Sunita Sarawagi Sundar Vishwanathan Indian Institute of Technology, Bombay. TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A A A A A A. MAP Estimation. θ i (x i ). - PowerPoint PPT Presentation

Transcript of MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Page 1: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Sashank J. Reddi Sunita Sarawagi Sundar Vishwanathan

Indian Institute of Technology, Bombay

Page 2: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

MAP Estimation• Energy Function

• MAP Estimation: Find the labeling which minimizes the energy function– NP Hard in general

xi

xjθij(xi,xj)

θi(xi)

Page 3: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Popular Approximation• Based on this LP relaxation

• Can be solved efficiently using Message Passing algorithms (TRW-S) and Graph cut based algorithms (QPBO)

Page 4: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Tightening Pairwise LP Relaxation• MPLP (David Sontag et.al 2008), Cycle Repairing Algorithm

(Nikos Komodakis et.al 2008)• MPLP uses higher order relaxation

Page 5: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Tightening Pairwise LP Relaxation• MPLP (David Sontag et.al 2008), Cycle Repairing Algorithm

(Nikos Komodakis et.al 2008)• MPLP uses higher order relaxation

Page 6: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

MAP Estimation via Graph Cuts• Construct a specialized graph for the particular energy

function• Minimum cut on this graph minimizes the energy

function• Exact MAP in polynomial time when the energy

function is sub-modular–

Page 7: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Assumption on Parameters• Symmetric, that is and

• Zero-normalized, that is and

• Any Energy function can be transformed in to an equivalent energy function of this form

Page 8: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Graph Construction

00

01

Page 9: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Graph Construction

00

01

i0

i1

Page 10: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Graph Construction

00

01

i0i1

θi(1)

θi(0)

Page 11: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Graph Construction

00

01

i0 j0

θi(1)

θi(0)

θj(1)

θj(0)

θij/2

θij= θij(0,1)+ θij(1,0)- θij(0,0)- θij(1,1) > 0

i1 j1

Page 12: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Graph Construction

00

01

i0 j1

θi(1)

θi(0)

-θij/2

θij= θij(0,1)+ θij(1,0)- θij(0,0)- θij(1,1) < 0

i1 j0

θj(1)

θj(0)

Page 13: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Bipartite Multi-Cut problem• Given an undirected graph with non-negative

edge weights and k ST pairs– Objective: Find the minimum cut with divides the graph into 2 regions

and separates the ST pairs

• LP and SDP Relaxations (Sreyash Kenkre et.al 2006)– LP Relaxation gives approximation– SDP Relaxation gives approximation

• Bipartite Multi-Cut vs Multi-Cut– Additional constraint on the number of regions the graph is cut

Page 14: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

BMC LP• are the ST pairs in the Bipartite Multi-Cut problem • Terminals =

Page 15: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

BMC LP• are the ST pairs in the Bipartite Multi-Cut problem • Terminals = is jt

00 01

de

D00(is)

D00(jt)

k0k1

Dk0(is) Dk0

(jt)

Page 16: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

BMC LP• are the ST pairs in the Bipartite Multi-Cut problem • Terminals =

i0 i1

j0j1

Page 17: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

BMC LP• are the ST pairs in the Bipartite Multi-Cut problem • Terminals =

is jt

00 01

de

D00(is)D00(jt)

Page 18: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

BMC LP• Infeasible to solve it using LP solvers– Large number of constraints

• Combinatorial Algorithm– Solving LP for general graphs may not be easy– Flexibility in construction of the graph– Combinatorial algorithm for a special kind of

construction

Page 19: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Symmetric Graph Construction

ee ww 00

01

i0 j0

θj(1)/2

θj(0)/2θi(0)/2

θij/4

θij= θij(0,1)+ θij(1,0)- θij(0,0)- θij(1,1)

00

01

i1 j1

θj(0)/2

θj(1)/2

θi(0)/2

θi(1)/2

θij/4

θi(1)/2

Page 20: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Symmetric Graph Construction• More ST pairs are added• Trade off is combinatorial algorithm

Page 21: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Approach of the Algorithm

BMC LP

Multicommodity flow LP

MultiCut LP

Page 22: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Multi-Cut LP• denotes all paths between pair of vertices in ST• denotes the set of paths in which contain edge e

• Can be solved approximately i.e ε-approximation for any error parameter ε

Multi – Cut LP Multi -Commodity Flow LP

Page 23: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Symmetric BMC LP (BMC-Sym LP)

• Very similar to Multi-Cut LP except for the symmetric constraints

Page 24: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Equivalence of LPs

Theorem – When the constructed graph is symmetric, BMC LP, BMC-Sym LP and Multi-Cut LP are equivalent

Proof Outline• Any feasible solution of each of the LP can be transformed into

a feasible solution of other LPs without changing the objective value

Page 25: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Combinatorial AlgorithmPrimal Step

• Find shortest path P between

• Update

• Update Complementary path

Dual Step

• Let• Update the flow in the path

by• Update flow in

complementary path

• Converges in• Can be improved to (Fleischer 1999)

Page 26: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Relationship with Cycle Inequalities

• BMC LP is closely related to cycle inequalities– BMC LP with slight modification is equivalent to cycle inequalities

• Relates our work to many recent works solving cycle inequalities

Page 27: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Empirical Results• Data Sets

– Synthetic Problems• Clique Graphical models

– Benchmark Data Set

• Algorithms– TRW-S– BMC

• ε = 0.02

– MPLP• 1000 iterations or up to convergence

Page 28: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Empirical Results

Page 29: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Empirical Results

Convergence Comparison of BMC and MPLP

Page 30: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Empirical Results

Page 31: MAP Estimation in Binary MRFs using Bipartite Multi-Cuts

Conclusion & Future Work• Conclusion– MAP estimation can be reduced to Bipartite Multi-Cut

problem– Algorithms for multi commodity flow can be used for MAP

estimation– BMC LP is closely related to cycle inequalities

• Future Work– Extensions to multi-label graphical models– Combinatorial algorithm for solving SDP relaxation