Spectral clustering with cortical kernelsgmvision.lsis.org/slides/barbieri.pdf · Spectral...
Embed Size (px)
Transcript of Spectral clustering with cortical kernelsgmvision.lsis.org/slides/barbieri.pdf · Spectral...

Spectral clustering with cortical kernelsDavide Barbieri1, Giovanna Citti2, Giacomo Cocci2, Marta Favali3, Alessandro Sarti3
1Universidad Autónoma de Madrid 2Università di Bologna 3CAMS, EHESS/CNRS Paris
Spatial and spatiotemporal receptive fieldsThe spatial and a spatiotemporal behavior of V1 receptive fields is well modeledby Gabors: if ζ = (ξ, η, τ ) are spatiotemporal coordinates on the visual space
gσ,ωz ,θ,v(ζ) = e−2πiω((ξ−x) cos θ+(η−y) sin θ−v(τ−t))e
12〈(ζ−z),σ(ζ−z)〉R3
where z = (x , y , t) is the receptive field center and σ is a covariance matrix.
Figure: Gabor fit of spatial RF in macaque [Ringach, 2002] and cat [Jones and Palmer, 1987]
Figure: Gabor fit of spatiotemporal RF in cat [DeAngelis et al., 1995 and Cocci et al., 2012]
Spatial visual stimuli are represented in the 3dimensional feature space
MS = R2 × S 1 = {(x , y , θ)}where θ is the local orientation of the stimulus, while spatiotemporal visualstimuli are represented in the 5dimensional feature space
MT = R3 × S 1 × R = {(x , y , t, θ, v)}where v is the velocity orthogonal to the spatial wavefront.
The tangent bundleSmooth level sets of greyscale images are lifted to integral surfaces of thecontact structure defined by the 1form
cos θdx + sin θdy
while smooth level sets of greyscale moving images are lifted to integral surfacesof the contact structure defined by the 1form
cos θdx + sin θdy − vdt.The associated contact structures are generated by the following vector fields:
ker(cos θdx + sin θdy) = span{X1,X2}ker(cos θdx + sin θdy − vdt) = span{X1,X2,X4,X5} , where
X1 = − sin θ∂x + cos θ∂y , X2 = ∂θ , X4 = ∂v , X5 = ∂t + v (cos θ∂x + sin θ∂y).The nonintegrability of the structure is expressed by the nonzero commutators
[X1,X2] = [X4,X5] = X3 = cos θ∂x + sin θ∂y (Reeb)
[X2,X3] = X1 , [X2,X5] = vX1.
Diffusion on fiber variables with drift on base variables can be implemented interms of the FokkerPlanck hypoelliptic operators[MS]
L3 = X1 − κ2X 22 = − sin θ∂x + cos θ∂y − κ2∂2θ[MV
]LV = X1 − κ2X 22 − α2X 24 = − sin θ∂x + cos θ∂y −
(κ2∂2θ + α
2∂2v)[
MT]
LT = X5 − κ2X 22 − α2X 24 = ∂t + v(cos θ∂x + sin θ∂y)−(κ2∂2θ + α
2∂2v)
which can be used to define a distance for which two points are closer thehigher is their probability of being connected by a smooth contour or by amotion trajectory that is compatible with their apparent velocity ~vθ = v êθ.
Spectral ClusteringConsider a data set {fi}ni=1 in a metric space as the vertices of a weightedgraph, where the edge weights {aij}ni ,j=1 define an affinity matrix A. Thenormalized affinity matrix P is given by
P = D−1A
where D is the diagonal degree matrix, having elements
di =n∑
j=1
aij.
When A is symmetric, the eigenvalues {λj}nj=1 of P satify 0 ≤ λj ≤ 1, and itseigenvectors can be chosen with real components. The more P is close to beingblock diagonal, the more its spectrum is close to being bipartite, witheigenvalues close to 1 or to 0. In such a case, eigenvectors corresponding tolarge eigenvalues are close to indicator functions of areas of the graph that arestrongly internally connected but weakly connected to the rest of the graph.
An affinity that captures the geometry of the stimulus can then be used tocluster it into its strongly connected components in terms of eigenvectors of P .
Geometric affinitiesWe have performed our spectral clusterings with kernels obtained from thefundamental solutions
L3Γ3 = δx ,y ,θ on MSLVΓV = δx ,y ,θ,v on MVLTΓT = δx ,y ,t,θ,v on MT .
The considered affinity for spatial stimuli with local orientations is
AS =1
2(Γ3 + Γ
†3)
where Γ†3 is the fundamental solution of the adjoint operator L†3
Figure: Clustering with Euclidean affinity (first two rows) and with AS (third row)
An affinity for spatiotemporal stimuli with orientation and apparent motion is
AST = ΓT +1
2(Γ̃V + Γ̃
†V)
where Γ̃V is the extension of ΓV to MT. This preserves causality and iscompatible with psychophysiological evidence that two different mechanismscooperate in the motion integration performed by the visual cortex.
Identification of inductors on Kanizsa figuresSpectral clustering with subRiemannian kernels may provide figuregroundsegmentation, and the first eigenvector may be used for modal completion.
Figure: First eigenvector of PS (red) for aligned and not aligned inductors [Favali et al. 2014]
When the inductors are cocircularly aligned, with limit curvature that dependson the scale of the kernels, their mutual affinity is higher than with the rest ofthe closed contour. Otherwise, the affinity of the closed contour is dominant.The appearence of a full 6 segmentstriangle, instead of three 2 segmentslinesseparately, is due to the polarity of the contours, which are lifted to [0, 2π].
Clustering of moving locally oriented stimuliSpatiotemporal kernels are able to cluster moving oriented stimuli against anoisy background, and to follow them during their evolution.
Figure: Clustering obtained with PST [Cocci et al. 2014]
ReferencesG. Cocci, D. Barbieri, G. Citti, A. Sarti, Cortical spatiotemporal dimensionalityreduction for visual grouping. Preprint arxiv.org/abs/1407.0733 (2014).M. Favali, G. Citti, A. Sarti, Figureground segmentation with subRiemanniankernel PCA. Preprint (2014).D. Barbieri, G. Citti, G. Cocci, A. Sarti, A corticalinspired geometry for contourperception and motion integration. J. Math. Imag. Vis. 49 (2014).G. Cocci, D. Barbieri, A. Sarti, Spatiotemporal receptive fields of cells in V1 areoptimally shaped for stimulus velocity estimation. JOSA A 29 (2012).
Supported by the E.C. FP7th Marie Curie Program, PEOPLE2013IEF Project 626055 “HAViX” (www.uam.es/davide.barbieri) and PEOPLE2013ITN Project 607643 “MAnET” (manet.dm.unibo.it)