7/31/2019 590-71_presentation
1/36
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
2/36
Approaches
Image segmentation by Graph Cut
The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps
Transparency
Border Matting Foreground estimation
Results
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
3/36
Approaches
Image segmentation by Graph Cut
The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps
Transparency
Border Matting Foreground estimation
Results
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
4/36
Magic Wand User specifies a point or a region to
compute a region of connected pixels
All the selected pixels within someadjustable tolerance of the color
statistics of the specified region areconsidered foreground
Issue Tolerance level is hard to be defined
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
5/36
Intelligent Scissors User chooses a minimum cost contour
by roughly tracing the objects boundarywith the mouse.
As the mouse moves, the minimum cost
path from the cursor position back to thelast seed point is shown.
If the computed path deviates from thedesired one, additional user-specifiedseed points are necessary.
Issue Many user interactions are often required
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
6/36
Bayes Matting The user specifies a trimap T = {TB, TU,
TF} in which background and foregroundregions TB and TF are marked, and alphavalues are computed over the remainingregion TU.
Models color distributions are calculated
probabilistically to achieve full alphamattes.
Issues Considerable degree of user interaction is
required
Results are not satisfying when TU is too large
Foreground and background color havesimilar color distribution
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
7/36
Knockout 2 Plug-in for Photoshop which is driven
from a user-defined trimap.
Approach is like Bayes matting, and itsresults are sometimes similar
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
8/36
Graph Cut Optimization technique that can be
used in a setting similar to BayesMatting.
Includes trimaps and probabilistic color
models, to achieve robustsegmentation, when foreground andbackground color distributions are notwell separated.
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
9/36
Grab Cut Matting tool to produce
continuous alpha values [0,1] Hard segmentation of the image
(using iterative graph cut) Border matting
Innovations Iterative estimation
Iterative minimization of energy Incomplete labeling
TF = 0, TU = NOT TB
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
10/36
Approaches
Image segmentation by Graph Cut
The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps
Transparency
Border Matting Foreground estimation
Results
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
11/36
Graph Cut [Boykov and Jolly, 2001]
Image is expressed as an array z = (Z1,.,ZN) of gray values
The segmentation of the image is expressed as an array ofopacity values a = (a1, . . . ,aN) at each pixel. Generally 0 a 1,but for hard segmentation an {0,1} with 0 for foreground and1for background
The parameters describe image foreground and backgroundgrey-level distributions, i.e. a pair of histogram of gray values
= {h(z;a), a = 0,1}
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
12/36
Energy Energy function E is defined so that its minimum
corresponds to a good segmentation Background/foreground are coherent parts. E(,,z) = U(,,z) + V(,z) Where
U is Unary Potential
V is Pairwise Potential
Segmentation (transparency values) are
calculated as a global minimum Min Cut algorithm
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
13/36
Unary U : evaluates the fit of opacity distribution a to dataz, given
Pairwise V (Smoothness term)
C is the set of pairs of neighboring pixels
dis refers to the Euclidean distance of pixels Constant > 0, = 50
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
14/36
Approaches
Image segmentation by Graph Cut
The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps
Transparency
Border Matting Foreground estimation
Results
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
15/36
Use of Gaussian Mixture Models (GMMs)instead of color histograms
GMMs One for the foreground, one for the background Each Gaussian Variable consists of K=5 components
N total pixels vector k={k1,,kN}
Each pixel belongs to a GMM component ki {1,,K}
Pixel i ai = {0,1}
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
16/36
The Energy for segmentation
Unary Potential
where D(an, kn, ,zn) = log p(zn|an, kn, ) log(an, kn),
p() is a Gaussian probability distribution
() are mixture weighting coefficients
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
17/36
The parameters of the model are
The smoothness term V is basically unchangedfrom the monochrome except that the contrastterm is computed using Euclidean distance in colorspace
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
18/36
Approaches
Image segmentation by Graph Cut
The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps
Transparency
Border Matting Foreground estimation
Results
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
19/36
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
20/36
Algorithm converges Reduce of user interaction
Simpler initial interaction
(a) The energy for the llama example converges over 12iterations.(b) The GMM in RGB color space (side-view showing R,G) atinitialization(c) after convergence K = 5 mixture components were used forboth background (red) and foreground (blue).
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
21/36
Approaches
Image segmentation by Graph Cut
The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps
Transparency Border Matting Foreground estimation
Results
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
22/36
Incomplete trimaps In place of the full trimap T, the user
needs only to specify thebackground region TB,leaving TF = 0.
Further user editing Marking roughly with a foreground
brush (white) and a backgroundbrush (red) is sufficient to obtain thedesired result (bottom row).
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
23/36
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
24/36
Foreground brush
Background brush
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
25/36
Approaches
Image segmentation by Graph Cut
The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps
Transparency Border Matting Foreground estimation
Results
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
26/36
Hard Segmentation Pixels of an image either belong or not to background and foreground
respectively (alpha values = {0,1})
Soft Segmentation Pixels of an image may partially belong to foreground or background
(alpha values = [0,1])
Full transparency is allowed in a narrow strip around the hardsegmentation boundary.
The goal is to compute the map an
wheren TU.
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
27/36
A dynamic programming (DP) algorithm for estimating athroughout TU.
Aim is to compute (center) and (width)
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
28/36
nergy function using DP over t:
V is a smoothing regularizer 1 = 50
2 = 1000
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
29/36
The data term D is defined as
where N(z; , ) denotes a Gaussian probability density for z with mean andcovariance .
The Gaussian parameters t (a), t (a), a = 0,1 for foreground and backgroundare estimated as the sample mean and covariance from each of the regions Ftand Bt
Regions Ft and Bt defined as Ft = St \TF and Bt St \TB, where St is a squareregion of size LL pixels centered on the segmentation boundary C at t (andwe take L = 41).
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
30/36
Approaches
Image segmentation by Graph Cut
The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps
Transparency Border Matting Foreground estimation
Results
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
31/36
The aim is to estimate foreground pixel colors without colorsbleeding in from the background of the source image
the Bayes matte is applied to obtain an estimate of foregroundcolor fnon a pixel n TU.
, from the neighborhood Ft(n), the pixel color that is mostsimilar to fn is stolen to form the foreground color fn .
, the combined results of border matting, using bothregularized alpha computation and foreground pixel stealing, areillustrated below.
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
32/36
Approaches
Image segmentation by Graph Cut
The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps
Transparency Border Matting Foreground estimation
Results
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
33/36
GrabCut obtains foreground alpha mattes of good quality for
moderately difficult images with a rather modest degree ofuser effort.
Combines hard segmentation by iterative graph-cutoptimization with border matting to deal with blur and mixedpixels on object boundaries.
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
34/36
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
35/36
8/7/2011HY-590.71 Final Presentation
7/31/2019 590-71_presentation
36/36
Top Related