CS-F441: Selected Topics from Computer Science (Deep Learning for NLP & CV) · 2019-11-06 ·...

CS-F441: SELECTED TOPICS FROM COMPUTER

SCIENCE (DEEP LEARNING FOR NLP & CV)

Lecture-KT-10: SIFT, HOG

Dr. Kamlesh Tiwari,Assistant Professor,

Department of Computer Science and Information Systems,BITS Pilani, Rajasthan-333031 INDIA

Nov 06, 2019 (Campus @ BITS-Pilani July-Dec 2019)

Recap: Harris OperatorUse

f =λ1λ2

λ1 + λ2=

determinant(H)

trace(H)

Do the following:1 Compute cornerness score of each point2 Find points whose surrounding window gave large corner response

(f > threshold)3 Take the points of local maxima, i.e., perform non-maximum

suppression

Rotation but Scale Invariance

STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 2 / 16

Recap: Scale invariant interest points

Spatial selection: the magnitude of the Laplacian response will achievea maximum at the center of the blob, provided the scale of theLaplacian is “matched” to the scale of the blob


Recap: Scale invariant interest pointsInterest points are local extrema in both position and scale.


Recap: Laplacian and Difference of Gaussians


Building Scale Space

∂G∂σ

= σ42 G

G(x , y , kσ)−G(x , y , σ)kσ − σ

= σ42 G

G(x , y , kσ)−G(x , y , σ) = (k − 1)σ2 42 G

Difference of Gaussian at two scale is similar to Laplacian


Scale Space Octave


SIFT Steps1 Determine interest points in scale space by looking extrema2 For x = (x , y , σ)T we can approximate DoG by

D(x) = D + ∂DT

∂x x + 12xT ∂2D

∂2x x it has extrema at (̂x) = −∂2D−1

∂x2∂D−1

∂xFilter outliers where |D(x)| > Th Effect: 832 -to- 729

3 Find Hessian and its determinant, trace. For r = λ1λ2

Discard points having r > 10 potential edges. Effect: 729 -to- 5364 Get orientation of interest point. In small neighborhood; histogram

of 36 bins of 10◦ each. Use magnitude for weighting


SIFT Descriptor

1 Gradient is more robust than magnitude2 Compute relative orientation and magnitude in the 16× 16

neighbourhood (8 bins)

3 Take 4× 4 blocks and create weighted histogram (scale andorientation) 1

18 × (16/4)× (16/4) = 128 dimensional vectorSTCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-10 (Nov 06, 2019) 9 / 16

SIFT Parameters


SIFT Matching 2

1 Key point with minimum Euclidean distance are similar2 Ratio between best and second best match should be large (.8 is

not good)

253406: Lowe, David G, “Distinctive image features from scale-invariant keypoints”, International journal of computer vision,

60(2), pp 91–110, Springer (2004)


SIFT Parameters


HOG: Histogram of Oriented Gradients

Histograms of oriented gradients for human detection 3

3[cite 28654] Histograms of oriented gradients for human detection, Dalal, Navneet and Triggs, Bill, CVPR 2005


HOG: Histogram of Oriented Gradients

Take 64× 128 image, divide it into 16× 16 blocks of 50% overlapTotal blocks 7× 15 = 105. Block is 2× 2 cell of 8× 8 sizeQuantize orientation in 9 direction (amplitude is vote)Feature size 105× (2× 2)× 9 = 3780


HOG

Example: 80◦ has distance 15 and 5 from 70 and 90. Hence ratiois 5/20 and 15/20

More: deformable part model (as there could be occlusion and action)


Thank You!

Thank you very much for your attention4!

Queries ?

4https://www.cs.cornell.edu/courses/cs6670/2011sp/lectures/lec02 filter.pdf


CS-F441: Selected Topics from Computer Science (Deep Learning for NLP & CV) · 2019-11-06 ·...

Documents

Transcript of CS-F441: Selected Topics from Computer Science (Deep Learning for NLP & CV) · 2019-11-06 ·...