Background Subtraction - II - Supercomputer …venky/SE263/slides/BGM2.pdfBackground Subtraction -...

Background Subtraction - II

R. Venkatesh Babu

SE 263 R. Venkatesh Babu

Running Gaussian Average

• Pfinder(Wren, Azarbayejani, Darrell, Pentland, 1997):– fitting one Gaussian distribution (µ,σ)over the

histogram: this gives the background PDF

– background PDF update: running average:

– In test | F -µ| > Th, Th can be chosen as kσ

– It does not cope with multimodal backgrounds()


Need for Adaptive Systems

• Consider the pixel process at (x0,y0)


Mixture of Gaussian(Stauffer and Grimson, CVPR’99, PAMI’00)

• This model copes with multimodal background distributions.

• Issues:

– the number of modes is arbitrarily pre-defined (usually from 3 to 5)

– how to initialize the Gaussians?

– how to update them over time?


Mixture Model

• Training:

• Model the pixel intensity variation as:

– The weight of a particular Gaussian is a measure of how good it represents the background.


Updation

• If none of the K distributions match the current pixel value:

– Replace the least probable dist with a dist with the current value as its mean, having high variance and low prior weight

• Update Prior Weights:

Where, ɑ is the learning rate and Mk,t =1 for matched Gaussian model and zerofor unmatched Gaussian


MoG: Algorithm• All weights ωi are updated (updated and/or normalised) at

every new frame

• At every new frame, some of the Gaussians “match” the current value (those at a distance < 2.5 σi ): for them, µi,σi are updated by the running average:

Where, learning factor


Background Models

• The mixture of Gaussians actually models both the foreground and the background: how to pick only the distributions modeling the background?: – All distributions are ranked according to their ωi /σi and the first ‘b’ are

chosen as “background”


Example


OpenCV Gaussian Mixture Based Background subtraction

• Class to be used: BackgroundSubtractorMOG

• Background model update function: Computes a foreground mask from the given input frame “image” and returns it in “fgmask”.operator()(InputArray image, OutputArray fgmask, double learningRate=0);

• Background computing function: Computes the current background and returns it in “backgroundImage”getBackgroundImage(OutputArray backgroundImage);

• The class implements the algorithm described in P. KadewTraKuPong and R. Bowden, An improved adaptive background mixture model for real-time tracking with shadow detection, Proc. 2nd European Workshop on Advanced Video-Based Surveillance Systems, 2001:http://personal.ee.surrey.ac.uk/Personal/R.Bowden/publications/avbs01/avbs01.pdf

• Other background modelling classes available in OpenCV: BackgroundSubtractor, BackgroundSubtractorMOG2.More info: http://opencv.itseez.com/modules/video/doc/motion_analysis_and_object_tracking.html#backgroundsubtractor http://opencv.itseez.com/modules/video/doc/motion_analysis_and_object_tracking.html#backgroundsubtractormog http://opencv.itseez.com/modules/video/doc/motion_analysis_and_object_tracking.html#backgroundsubtractormog2

http://personal.ee.surrey.ac.uk/Personal/R.Bowden/publications/avbs01/avbs01.pdf

http://personal.ee.surrey.ac.uk/Personal/R.Bowden/publications/avbs01/avbs01.pdf

http://opencv.itseez.com/modules/video/doc/motion_analysis_and_object_tracking.html







OpenCV Code Snippet(C++)……cv::Mat NewFrame, FGMask, BG; //Image matrices for newframe, foreground mask and the Background.

cv::VideoCapture VideoFeed(“test.avi"); //Create class instance to handle video capture.BackgroundSubtractorMOG BGModel; //Create class instance for Background modelling.

cv::namedWindow(“Input Video"); //Create Windows.cv::namedWindow(“ForeGround Mask");cv::namedWindow(“BackGround");

while ( VideoFeed.isOpened()) // Returns true if video capturing has been initialized already.{

if(VideoFeed.grab()) // Grabs the next frame from video file or capturing device and return true (non-zero) in the // case of success.

{VideoFeed.retrieve(NewFrame); // Decodes and returns the grabbed video frame.

BGModel.operator()(NewFrame, FGMask, 0); //Update Background model and return foreground mask.BGModel.getBackgroundImage(BG ); //Returns the current Background image.

cv::imshow(“Input Video", NewFrame); //Display video frames.cv::imshow(“ForeGround Mask", FGMask);cv::imshow(“BackGround", BG);

}cv::waitKey(3);

}cvDestroyWindow(“Input Video“); //Delete Window and free the memory used by it.cvDestroyWindow(“ForeGround Mask");cvDestroyWindow(“BackGround");……


Non Parametric Model for Background(Ahmed Elgammal ,David Harwood, Larry Davis, ECCV ’00)

• It is a non – parametric model ie., distribution is derived solely from the observed samples.

• Say we have samples of data for a pixel {Xi}, i= 1 to nthe probability density of an intensity ‘x’ is given by

• ‘K’ is the kernel function, typically a Gaussian.

Reference Paper - Ahmed Elgammal ,David Harwood, Larry Davis.“Non Parametric Model for Background Subtraction”


Pixel Intensity over time


Histogram


KDE - Example

• data=[randn(100,1);randn(100,1)*2+35 ;randn(100,1)+55];

• kde(data,2^14,min(data)-5,max(data)+5);

• hold on;plot(data,0,'+');


Choice of kernel bandwidth


Kernel Bandwidth

• Bandwidth decides the smoothness.

• Small bandwidth leads to spurious data artifacts.

• Large bandwidth may result in over smoothing

• Bandwidth has to be chosen optimally.


Finding the bandwidth

• Find the median absolute deviation for every 2 consecutive samples for each color channel.

– Median m of |Xi+1 –Xi| for every 2 consecutive samples (Xi+1, Xi) .

– BW =


Kernel

• K = N(0,Σ), where, Σ represents kernel BW

• KDE:

– Assuming:

– Then, KDE=

KDE using Gaussian Kernel Generalization of GMM where each sample is considered N(0, Σ).


Result


Suppression of false Detection:Pixel displacement distribution

• If then we say that particular value is a foreground value. Otherwise it is background.

• To make sure that the detected pixel is a foreground pixel, we define a Pixel displacement distribution given by

• It is a measure of the maximum probability of the pixel being the background pixel of a nearby pixel.


Component displacement distribution

Further constraints :

• Foreground and background intensities may be similar.

• So we check if the entire detected object has displaced from nearby pixels.

• For this we define Component displacement distribution given by


Component displacement distribution

• Detected Pixel ‘x’ is BG iff

• Neigborhood 5 dia circular neighborhood


Results


KDE - Results


Model Update

• Selective Update:

• Blind Update


ViBE = Visual Background Extractor

ViBe: A Universal Background Subtraction Algorithm for Video Sequences,

Olivier Barnich and Marc Van Droogenbroeck (ICASSP ’09 & IEEE Trans. IP ‘11)


Model

• Classify a new pixel value with respect to its immediate neighborhood

• Model each background pixel with a set of samples instead of with an explicit pixel model.

– No estimation of pdfs


BG Pixel Classification

• Each background pixel xis modeled by a collection of Nbackground sample values

• New pixel value v(x) is BG iff:

> #min


ViBE: Parameters

• Radius of Sphere R

– Set at 20 for Monochrome Images

• #min:

– Set as 2

• These parameters are fixed for all pixels/frames

• Adjust the sensitivity by changing the ratio: #min /N


Model Initialization

• From a single frame populate the pixel models with values found in the spatial neighborhood of each pixel.

– Values randomly taken in their neighborhood

– Neighborhood chosen – 8 connected


Background model initialization

• The Model is initialized as follows:

M0(x) = { v0 (y) | y ∈ NG (x) }

Where,

Vo(y) = Intensity at pixel ‘y’ at t=0

NG(x) = Neighborhood of pixel ‘x’

M0(x) = Background model for pixel ‘x’ at t=0


Background model initialization

Problem with this kind of initialization :

Foreground objects being present in the first frame itself – presence of ‘ghosts’ in the background model.


Is a pixel Background or Foreground?

Now we have the background model for a pixel ‘x’ as

M(x) = { v1 , v2 , v3 … vN }

Say v(x) is the observed intensity value of pixel ‘x’.

Steps:

• Define a sphere of radius ‘R’ centered at the intensity value v(x).

• Check how many points from M(x) are inside this sphere.

• If more than nthresh values from M(x) are inside the sphere, v(x) is a background value, otherwise not.


Is a pixel Background or Foreground?


Updating the background model over time

• Background model of a pixel is updated randomly !



Conservative Update :• Update the model only if a pixel value is declared as

a background. If it is foreground, do not update at all.

• Consider a parked car that begins to move suddenly. According to conservative update policy, the region uncovered by the car will not be updated. This leads to permanent presence of the car’s “ghost”.

• But in ViBE, this problem is overcome because neighboring pixels can update the background model of a given pixel.



Memoryless Update:

• Which value is discarded from the model M(x) is determined randomly.

• Every value in M(x) has equal probability of being discarded after a fixed time.

• The past has no effect in the update process.


Model Update



Random time Subsampling:• Update the background model randomly.• Say a pixel has been declared as background some ‘k’

times, we update the background model of that pixel only once, out of these ‘k’ times, that too randomly.– K =16 (used) a background pixel value has 1 chance in 16 of

being selected to update its pixel model.

• Advantage: The old model is not discarded immediately.• Disadvantage: Failure whenever there are fast moving

objects.



Updating the neighboring pixels’ background values:

If the current pixel value v(x) has been declared a background, we randomly choose a pixel from the neighborhood of x and update its background value.

Assumption: The pixels are spatially correlated.


Results


ViBE software download

http://www2.ulg.ac.be/telecom/research/vibe/

(Pre-compiled)





Eigen Backgrounds

• Take N sample images and compute mean and covariance matrix (µb,Cb)

• Diagonalize Cb

• Reduce Dimension by keeping only M Eigen vectors corresponding to M largest Eigen values.

• Project each input image Ii onto the space expanded by the eigen background images

• Detect Moving objects:

“A Bayesian Computer Vision System for Modeling Human Interactions” Nuria M. Oliver, Barbara Rosario, and Alex P. Pentland, IEEE Trans. PAMI, 2000

Eigen Vector Matrix

Where,

Background Subtraction - II - Supercomputer …venky/SE263/slides/BGM2.pdfBackground Subtraction -...

Documents

Transcript of Background Subtraction - II - Supercomputer …venky/SE263/slides/BGM2.pdfBackground Subtraction -...