Download - ΗΥ 590-71_presentation

Transcript
  • 7/31/2019 590-71_presentation

    1/36

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    2/36

    Approaches

    Image segmentation by Graph Cut

    The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps

    Transparency

    Border Matting Foreground estimation

    Results

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    3/36

    Approaches

    Image segmentation by Graph Cut

    The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps

    Transparency

    Border Matting Foreground estimation

    Results

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    4/36

    Magic Wand User specifies a point or a region to

    compute a region of connected pixels

    All the selected pixels within someadjustable tolerance of the color

    statistics of the specified region areconsidered foreground

    Issue Tolerance level is hard to be defined

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    5/36

    Intelligent Scissors User chooses a minimum cost contour

    by roughly tracing the objects boundarywith the mouse.

    As the mouse moves, the minimum cost

    path from the cursor position back to thelast seed point is shown.

    If the computed path deviates from thedesired one, additional user-specifiedseed points are necessary.

    Issue Many user interactions are often required

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    6/36

    Bayes Matting The user specifies a trimap T = {TB, TU,

    TF} in which background and foregroundregions TB and TF are marked, and alphavalues are computed over the remainingregion TU.

    Models color distributions are calculated

    probabilistically to achieve full alphamattes.

    Issues Considerable degree of user interaction is

    required

    Results are not satisfying when TU is too large

    Foreground and background color havesimilar color distribution

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    7/36

    Knockout 2 Plug-in for Photoshop which is driven

    from a user-defined trimap.

    Approach is like Bayes matting, and itsresults are sometimes similar

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    8/36

    Graph Cut Optimization technique that can be

    used in a setting similar to BayesMatting.

    Includes trimaps and probabilistic color

    models, to achieve robustsegmentation, when foreground andbackground color distributions are notwell separated.

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    9/36

    Grab Cut Matting tool to produce

    continuous alpha values [0,1] Hard segmentation of the image

    (using iterative graph cut) Border matting

    Innovations Iterative estimation

    Iterative minimization of energy Incomplete labeling

    TF = 0, TU = NOT TB

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    10/36

    Approaches

    Image segmentation by Graph Cut

    The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps

    Transparency

    Border Matting Foreground estimation

    Results

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    11/36

    Graph Cut [Boykov and Jolly, 2001]

    Image is expressed as an array z = (Z1,.,ZN) of gray values

    The segmentation of the image is expressed as an array ofopacity values a = (a1, . . . ,aN) at each pixel. Generally 0 a 1,but for hard segmentation an {0,1} with 0 for foreground and1for background

    The parameters describe image foreground and backgroundgrey-level distributions, i.e. a pair of histogram of gray values

    = {h(z;a), a = 0,1}

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    12/36

    Energy Energy function E is defined so that its minimum

    corresponds to a good segmentation Background/foreground are coherent parts. E(,,z) = U(,,z) + V(,z) Where

    U is Unary Potential

    V is Pairwise Potential

    Segmentation (transparency values) are

    calculated as a global minimum Min Cut algorithm

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    13/36

    Unary U : evaluates the fit of opacity distribution a to dataz, given

    Pairwise V (Smoothness term)

    C is the set of pairs of neighboring pixels

    dis refers to the Euclidean distance of pixels Constant > 0, = 50

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    14/36

    Approaches

    Image segmentation by Graph Cut

    The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps

    Transparency

    Border Matting Foreground estimation

    Results

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    15/36

    Use of Gaussian Mixture Models (GMMs)instead of color histograms

    GMMs One for the foreground, one for the background Each Gaussian Variable consists of K=5 components

    N total pixels vector k={k1,,kN}

    Each pixel belongs to a GMM component ki {1,,K}

    Pixel i ai = {0,1}

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    16/36

    The Energy for segmentation

    Unary Potential

    where D(an, kn, ,zn) = log p(zn|an, kn, ) log(an, kn),

    p() is a Gaussian probability distribution

    () are mixture weighting coefficients

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    17/36

    The parameters of the model are

    The smoothness term V is basically unchangedfrom the monochrome except that the contrastterm is computed using Euclidean distance in colorspace

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    18/36

    Approaches

    Image segmentation by Graph Cut

    The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps

    Transparency

    Border Matting Foreground estimation

    Results

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    19/36

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    20/36

    Algorithm converges Reduce of user interaction

    Simpler initial interaction

    (a) The energy for the llama example converges over 12iterations.(b) The GMM in RGB color space (side-view showing R,G) atinitialization(c) after convergence K = 5 mixture components were used forboth background (red) and foreground (blue).

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    21/36

    Approaches

    Image segmentation by Graph Cut

    The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps

    Transparency Border Matting Foreground estimation

    Results

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    22/36

    Incomplete trimaps In place of the full trimap T, the user

    needs only to specify thebackground region TB,leaving TF = 0.

    Further user editing Marking roughly with a foreground

    brush (white) and a backgroundbrush (red) is sufficient to obtain thedesired result (bottom row).

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    23/36

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    24/36

    Foreground brush

    Background brush

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    25/36

    Approaches

    Image segmentation by Graph Cut

    The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps

    Transparency Border Matting Foreground estimation

    Results

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    26/36

    Hard Segmentation Pixels of an image either belong or not to background and foreground

    respectively (alpha values = {0,1})

    Soft Segmentation Pixels of an image may partially belong to foreground or background

    (alpha values = [0,1])

    Full transparency is allowed in a narrow strip around the hardsegmentation boundary.

    The goal is to compute the map an

    wheren TU.

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    27/36

    A dynamic programming (DP) algorithm for estimating athroughout TU.

    Aim is to compute (center) and (width)

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    28/36

    nergy function using DP over t:

    V is a smoothing regularizer 1 = 50

    2 = 1000

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    29/36

    The data term D is defined as

    where N(z; , ) denotes a Gaussian probability density for z with mean andcovariance .

    The Gaussian parameters t (a), t (a), a = 0,1 for foreground and backgroundare estimated as the sample mean and covariance from each of the regions Ftand Bt

    Regions Ft and Bt defined as Ft = St \TF and Bt St \TB, where St is a squareregion of size LL pixels centered on the segmentation boundary C at t (andwe take L = 41).

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    30/36

    Approaches

    Image segmentation by Graph Cut

    The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps

    Transparency Border Matting Foreground estimation

    Results

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    31/36

    The aim is to estimate foreground pixel colors without colorsbleeding in from the background of the source image

    the Bayes matte is applied to obtain an estimate of foregroundcolor fnon a pixel n TU.

    , from the neighborhood Ft(n), the pixel color that is mostsimilar to fn is stolen to form the foreground color fn .

    , the combined results of border matting, using bothregularized alpha computation and foreground pixel stealing, areillustrated below.

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    32/36

    Approaches

    Image segmentation by Graph Cut

    The GrabCut segmentation algorithm Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps

    Transparency Border Matting Foreground estimation

    Results

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    33/36

    GrabCut obtains foreground alpha mattes of good quality for

    moderately difficult images with a rather modest degree ofuser effort.

    Combines hard segmentation by iterative graph-cutoptimization with border matting to deal with blur and mixedpixels on object boundaries.

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    34/36

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    35/36

    8/7/2011HY-590.71 Final Presentation

  • 7/31/2019 590-71_presentation

    36/36