Predicting responses of V1 neurons to natural movies · 2008-11-01 · This work was supported by...
1
Contact information: Amir Khosrowshahi, Redwood Center for Theoretical Neuroscience, University of California, 132 Barker, MC #3190, Berkeley CA 94720-3190 – Email: [email protected]; Web: http://redwood.berkeley.edu/ Results Summary Validation ρ for STA from natural scenes vs m-sequences 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 msequence STA natural STA natural vs msequence STA correlations 10% 20% 50% Validation ρ for STC from natural scenes vs m-sequences 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 msequence STC natural STC natural vs msequence STC correlations 10% 20% 50% Validation ρ for natural scene STA vs STC 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 natural STA natural STC natural STA vs natural STC correlations 10% 20% 50% Conclusions • Computing informative spatiotemporal STA and STC kernels using full-field natural movie stimuli is feasible. • Second order kernels computed from natural movies displayed additional structure when m-sequences did not. • Natural movie kernels resulted in improved prediction over msequence kernels in many of the validation tests. • Overall, however, the predictive capabilities of this model are poor in most cases. References 1. Paninski, L., Convergence properties of three spike-triggered analysis techniques. Network: Comput. Neural Syst. 14, 437–464 (2003). 2. Touryan, J. and Felsen, G. and Dan, Y. Spatial structure of complex cell receptive fields measured with natural images. Neuron 45, 781–791 (2005). 3. Rust, N.C., Schwartz, O., Movshon, J.A. and Simoncelli, E. Spatiotemporal elements of macaque V1 receptive fields. Neuron 46, 945–956 (2005). 4. David, S.V. and Gallant, J.L., Predicting neuronal responses during natural vision. Network 16, 239–260 (2005). 5. Yen, S-C., Baker, J. and Gray, C.M. Heterogeneity in the responses of adjacent neurons to natural stimuli in cat striate cortex. J Neurophys 97, 1326–1341 (2007). Acknowledgments This work was supported by NIH grant EY07043, NGA grant MCA 015894-UCB, NSF grant IIS-06-25223, and a Kopriva Fellowship to J.B. Example Cells Simple Cell STA validation for kernels from msequence and two natural movies. Note the temporal differences in RF structure. Predicted firing rates are in red, actual in blue. Simple/Complex Cell A cell with STA and STC structure using m-sequences and two long movies. Note additional natural movie STC structure aids validtion. Complex Cell Natural movie STC structure captures pronounced peaks in firing rate. Model Spatiotemporal frames and correcting for correlations An ensemble of spike-triggered vectors was constructed from the 8 movie frames displayed at 35ms intervals before each spike. RF spatial bounds were determined from m-sequence-derived 1st-order kernel structure or by hand mapping when no structure was evident. Extracted movie patches were downsampled to the same spatial resolution as m-sequences. To correct for intrinsic 2nd-order correlations in natural stimuli, the spike-triggered ensemble was projected onto a subspace consisting of the most significant eigenvectors of a whitening transform. Axes were rescaled to equate variance along each dimension (see Touryan et al. [2]). Calculation of whitening trans- form: E[xx T ]= UDV T (SVD) W = D - 1 2 U T v w = Wv with x the whole movie ensem- ble, W the whitening transform, v w the projected vector. Eigen- vectors are columns of U and are displayed to the right for a sam- ple cell and movie. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 a223 tetrode 3 cell 4 Kernel and non-linearity estimation 1st and 2nd order spatiotemporal kernels K 0 (STA) and K i (STC) are respectively the mean and the eigen- vectors of the covariance matrix of the spike-triggered ensemble. Separate kernels were derived for each m-sequence and long movie session. An output non-linearity was fit to firing rates calculated from the short movie repeats with parameters varied to reduce the mean-squared error between actual and predicted fir- ing rates. Predictions are best-case correlations given a fixed kernel and the functional form of the static non-linearity. The form of the STC component was motivated by Rust et al. [3]. r (t)= r 0 + α(|K 0 I t + θ | + ) p + i β i (K i I t ) 2 + _ _ + _ + _ . . . Image I (x, y, t) r (t) Receptive Field Nonlinearity Response K 0 (x, y, t) K 1 (x, y, t) K n (x, y, t) Firing rates as a function of movies projected onto kernels Inset histograms are frequency of spiking given a projection of movie onto kernel. Corresponding inset plots are of firing rate per frame as a function of the projection. 0 0 1 2 3 4 5 6 STA 10 10 20 0.5 1.0 1.5 2.0 2.5 3.0 0 0 1 2 3 4 5 6 STC 1 10 10 20 0.5 1.0 1.5 2.0 2.5 3.0 0 0 1 2 3 4 5 6 STC 2 10 10 20 0.5 1.0 1.5 2.0 2.5 0 0 1 2 3 4 5 6 STC 3 10 10 20 0.5 1.0 1.5 2.0 0 0 1 2 3 4 5 6 STC 4 10 10 20 0.5 1.0 1.5 2.0 2.5 3.0 a223 tetrode 3 cell 4, 25 cutoff 0 0 1 2 3 4 5 6 7 8 9 STA 2 2 4 6 1 2 3 4 5 6 0 0 1 2 3 4 5 6 7 8 9 STC 1 2 2 4 6 1 2 3 4 5 6 7 0 0 1 2 3 4 5 6 7 8 9 STC 2 2 2 4 6 1 2 3 4 5 6 7 0 0 1 2 3 4 5 6 7 8 9 STC 3 2 2 4 6 1 2 3 4 5 6 7 0 0 1 2 3 4 5 6 7 8 9 STC 4 2 2 4 6 1 2 3 4 5 6 7 a223 tetrode 2 cell 1, 25 cutoff Introduction Response properties of visual cortical neurons have been typically characterized by the conditional probability distribution that a neuron spikes given the presentation of a particular stimulus. We studied to what extent one functional class for this distribution, the linear-nonlinear Poisson (LNP) cascade receptive field (RF) model, can account for the variance of responses to natural movies when fit using different training stimuli 1-4 . Methods Single cell recordings were made from striate cortex of anesthetized cats viewing the following full-field stimuli 5 : • 2-D binary m-sequences (44min) • Long duration natural movies with few repeats (44min) • Short 30s natural movie segments with 100 repeats Kernels were computed from m-sequence and long movie sessions. Non-linearities were fit to either long or short movie sessions. Extracellular recordings using silicon tetrode probes Full-field natural movies Heterogeneity in response to 30s natural movie 5 200 400 600 800 20 40 60 80 100 frame trial a219 tetrode 3 cell 4 200 400 600 800 20 40 60 80 100 frame trial a220 tetrode 3 cell 4 200 400 600 800 20 40 60 80 100 frame trial a219 tetrode 2 cell 1 200 400 600 800 20 40 60 80 100 frame trial a220 tetrode 2 cell 1 200 400 600 800 20 40 60 80 100 frame trial a219 tetrode 3 cell 2 200 400 600 800 20 40 60 80 100 frame trial a220 tetrode 3 cell 2 REDWOOD CENTER for Theoretical Neuroscience Amir Khosrowshahi 1 , Jonathan Baker 2 , Shih-Cheng Yen 3 , Charles M. Gray 2 , and Bruno A. Olshausen 1 1 Redwood Center for Theoretical Neuroscience, University of California, Berkeley, 2 Center for Computational Biology, Montana State University, Bozeman, 3 Department of Electrical and Computer Engineering, National University of Singapore. Predicting responses of V1 neurons to natural movies
Transcript of Predicting responses of V1 neurons to natural movies · 2008-11-01 · This work was supported by...
Contact information: Amir Khosrowshahi, Redwood Center for Theoretical Neuroscience, University of California, 132Barker, MC #3190, Berkeley CA 94720-3190 – Email:[email protected]; Web:http://redwood.berkeley.edu/
Results SummaryValidation ρ for STA from natural scenes vs m-sequences
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
msequence STA
natu
ral S
TA
natural vs msequence STA correlations
10%20%50%
Validation ρ for STC from natural scenes vs m-sequences
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
msequence STC
natu
ral S
TC
natural vs msequence STC correlations
10%20%50%
Validation ρ for natural scene STA vs STC
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
natural STA
natu
ral S
TC
natural STA vs natural STC correlations
10%20%50%
Conclusions• Computing informative spatiotemporal STA and STC kernels using full-field natural movie stimuli is feasible.
• Second order kernels computed from natural movies displayed additional structure when m-sequences did not.
• Natural movie kernels resulted in improved prediction overmsequence kernels in many of the validation tests.
• Overall, however, the predictive capabilities of this model are poor in most cases.
References
1. Paninski, L., Convergence properties of three spike-triggered analysis techniques. Network: Comput. Neural Syst.14, 437–464 (2003).
2. Touryan, J. and Felsen, G. and Dan, Y. Spatial structure of complex cell receptive fields measured with natural images.Neuron 45, 781–791(2005).
3. Rust, N.C., Schwartz, O., Movshon, J.A. and Simoncelli, E. Spatiotemporal elements of macaque V1 receptive fields.Neuron 46, 945–956 (2005).
4. David, S.V. and Gallant, J.L., Predicting neuronal responses during natural vision.Network 16, 239–260 (2005).5. Yen, S-C., Baker, J. and Gray, C.M. Heterogeneity in the responses of adjacent neurons to natural stimuli in cat striate cortex. J Neurophys 97,
1326–1341 (2007).
Acknowledgments
This work was supported by NIH grant EY07043, NGA grant MCA 015894-UCB, NSF grant IIS-06-25223, and a Kopriva Fellowshipto J.B.
Example CellsSimple CellSTA validation for kernels from msequence and two natural movies. Note the temporal differences in RF structure.Predictedfiringrates are in red,actualin blue.
Simple/Complex CellA cell with STA and STC structure using m-sequences and two long movies. Note additional natural movie STC structure aidsvalidtion.
Complex CellNatural movie STC structure captures pronounced peaks in firing rate.
ModelSpatiotemporal frames and correcting for correlationsAn ensemble of spike-triggered vectors was constructed from the 8 movie frames displayed at 35ms intervalsbefore each spike. RF spatial bounds were determined from m-sequence-derived 1st-order kernel structureor by hand mapping when no structure was evident. Extracted movie patches were downsampled to thesame spatial resolution as m-sequences. To correct for intrinsic 2nd-order correlations in natural stimuli, thespike-triggered ensemble was projected onto a subspace consisting of the most significant eigenvectors of awhitening transform. Axes were rescaled to equate variancealong each dimension (see Touryanet al. [2]).
Calculation of whitening trans-form:
E[xxT] = UDV T (SVD)
W = D−1
2UT
vw = Wv
with x the whole movie ensem-ble, W the whitening transform,vw the projected vector. Eigen-vectors are columns ofU and aredisplayed to the right for a sam-ple cell and movie.
1 2
3 4
5 6
7 8
9 10
11 12
13 14
15 16
17 18
19 20
a223 tetrode 3 cell 4
Kernel and non-linearity estimation1st and 2nd order spatiotemporal kernelsK0 (STA) andKi (STC) are respectively the mean and the eigen-vectors of the covariance matrix of the spike-triggered ensemble. Separate kernels were derived for eachm-sequence and long movie session. An output non-linearitywas fit to firing rates calculated from the shortmovie repeats with parameters varied to reduce the mean-squared error between actual and predicted fir-ing rates. Predictions arebest-case correlations given a fixed kernel and the functional form of the staticnon-linearity. The form of the STC component was motivated by Rustet al. [3].
r(t) = r0 + α(|K0It + θ|+)p +∑
i
βi(KiIt)2
+_
_
+_
+_
. . .
Image
I(x, y, t) r(t)
Receptive Field Nonlinearity Response
K0(x, y, t)
K1(x, y, t)
Kn(x, y, t)
Firing rates as a function of movies projected onto kernelsInset histograms are frequency of spiking given a projection of movie onto kernel. Corresponding inset plotsare of firing rate per frame as a function of the projection.
0
0123456
STA
-10 10 20
0.5
1.0
1.5
2.0
2.5
3.0
0
0123456
STC 1
-10 10 20
0.5
1.0
1.5
2.0
2.5
3.0
0
0123456
STC 2
-10 10 20
0.5
1.0
1.5
2.0
2.5
0
0123456
STC 3
-10 10 20
0.5
1.0
1.5
2.0
0
0123456
STC 4
-10 10 20
0.5
1.0
1.5
2.0
2.5
3.0
a223 tetrode 3 cell 4, 25% cutoff
0
0123456789
STA
-2 2 4 6
1
2
3
4
5
6
0
0123456789
STC 1
-2 2 4 6
1
2
3
4
5
6
7
0
0123456789
STC 2
-2 2 4 6
1
2
3
4
5
6
7
0
0123456789
STC 3
-2 2 4 6
1
2
3
4
5
6
7
0
0123456789
STC 4
-2 2 4 6
1
2
3
4
5
6
7
a223 tetrode 2 cell 1, 25% cutoff
IntroductionResponse properties of visual cortical neurons have been typically characterized by the conditional probabilitydistribution that a neuron spikes given the presentation ofa particular stimulus. We studied to what extent onefunctional class for this distribution, the linear-nonlinear Poisson (LNP) cascade receptive field (RF) model,can account for the variance of responses to natural movies when fit using different training stimuli1−4.
MethodsSingle cell recordings were made from striate cortex of anesthetized cats viewing the followingfull-fieldstimuli5:
• 2-D binary m-sequences (44min)
• Long duration natural movies with few repeats (44min)
• Short 30s natural movie segments with 100 repeats
Kernels were computed from m-sequence and long movie sessions. Non-linearities were fit to either long orshort movie sessions.
Extracellular recordings using silicon tetrode probes
Full-field natural movies
Heterogeneity in response to 30s natural movie5
200 400 600 800
20
40
60
80
100
frame
tria
l
a219 tetrode 3 cell 4
200 400 600 800
20
40
60
80
100
frame
tria
l
a220 tetrode 3 cell 4
200 400 600 800
20
40
60
80
100
frame
tria
l
a219 tetrode 2 cell 1
200 400 600 800
20
40
60
80
100
frame
tria
l
a220 tetrode 2 cell 1
200 400 600 800
20
40
60
80
100
frame
tria
l
a219 tetrode 3 cell 2
200 400 600 800
20
40
60
80
100
frame
tria
l
a220 tetrode 3 cell 2
R E D W O O D C E N T E R
f o r T h e o r e t i c a l N e u r o s c i e n c eAmir Khosrowshahi1 , Jonathan Baker2, Shih-Cheng Yen3, Charles M. Gray2, and Bruno A. Olshausen1
1Redwood Center for Theoretical Neuroscience, University of California, Berkeley, 2Center for Computational Biology, Montana State University, Bozeman, 3Department of Electrical and Computer Engineering, National University of Singapore.
Predicting responses of V1 neurons to natural movies
