ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.
-
date post
21-Dec-2015 -
Category
Documents
-
view
222 -
download
1
Transcript of ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.
![Page 1: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/1.jpg)
ICA
Alphan Altinok
![Page 2: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/2.jpg)
Outline
PCA ICA
Foundation Ambiguities Algorithms Examples Papers
![Page 3: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/3.jpg)
PCA & ICA
PCA Projects d-dimensional data onto a lower dimensional subspace
in a way that is optimal in Σ|x0 – x|2.
ICA Seek directions in feature space such that resulting signals show
independence.
![Page 4: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/4.jpg)
PCA
Compute d-dimensional μ (mean). Compute d x d covariance matrix. Compute eigenvectors and eigenvalues. Choose k largest eigenvalues.
k is the inherent dimensionality of the subspace governing the signal and (d – k) dimensions generally contain noise.
Form a d x k matrix A with k columns of eigenvalues. The representation of data by principal components
consists of projecting data into k-dimensional subspace by x = At (x – μ).
![Page 5: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/5.jpg)
PCA A simple 3-layer neural network can form such a
representation when trained.
![Page 6: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/6.jpg)
ICA
While PCA seeks directions that represents data best in a Σ|x0 – x|2 sense, ICA seeks such directions that are most independent from each other.
Used primarily for separating unknown source signals from their observed linear mixtures.
Typically used in Blind Source Separation problems. ICA is also used in feature extraction.
![Page 7: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/7.jpg)
ICA – Foundation q source signals s1(k), s2(k), …, sq(k)
with 0 means k is the discrete time index or pixels in images scalar valued mutually independent for each value of k
h measured mixture signals x1(k), x2(k), …, xh(k)
Statistical independence for source signals p[s1(k), s2(k), …, sq(k)] = П p[si(k)]
![Page 8: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/8.jpg)
ICA – Foundation The measured signals will be given by
xj(k) = Σsi(k)aij + nj(k)
For j = 1, 2, …, h, the elements aij are unknown.
Define vectors x(k) and s(k), and matrix A Observed: x(k) = [x1(k), x2(k), …, xh(k)]
Source: s(k) = [s1(k), s2(k), …, sq(k)]
Mixing matrix: A = [a1, a2, …, aq]
The equation above can be stated in vector-matrix form x(k) = As(k) + n(k) = Σsi(k)ai + n(k)
![Page 9: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/9.jpg)
Ambiguities with ICA
The ICA expansion x(k) = As(k) + n(k) = Σsi(k)ai + n(k)
Amplitudes of separated signals cannot be determined.
There is a sign ambiguity associated with separated signals.
The order of separated signals cannot be determined.
![Page 10: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/10.jpg)
ICA – Using NNs
Prewhitening – transform input vectors x(k) by v(k) = V x(k) Whitening matrix V can be obtained by NN or PCA
Separation (NN or contrast approximation) Estimation of ICA basis vectors (NN or batch approach)
![Page 11: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/11.jpg)
ICA – Fast Fixed Point Algorithm
FFPA converges rapidly to the most accurate solution allowed by the data structure.
![Page 12: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/12.jpg)
ICA – Example
![Page 13: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/13.jpg)
ICA – Example
![Page 14: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/14.jpg)
ICA – Example
![Page 15: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/15.jpg)
ICA – Example
![Page 16: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/16.jpg)
ICA – Example
![Page 17: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/17.jpg)
ICA – Example BSS of recorded speech and music signals.
speech / music speech / speechspeech / speech in difficult environment
mic1 mic1 mic1
mic2 mic2 mic2
separated1 separated1 separated1
separated2 separated2 separated2
http://www.cnl.salk.edu/~tewon/ica_cnl.html
![Page 18: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/18.jpg)
ICA – Example
Source images
separation demo
http://www.open.brain.riken.go.jp/demos/researchBSRed.html
![Page 19: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/19.jpg)
ICA – Papers Hinton – A New View of ICA
Interprets ICA as a probability density model. Overcomplete, undercomplete, and multi-layer non-linear ICA
becomes simpler.
Cardoso – Blind Signal Separation, Statistical Principles Modelling identifiability. Contrast functions. Estimating functions. Adaptive algorithms. Performance issues.
![Page 20: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/20.jpg)
ICA – Papers Hyvarinen – ICA Applied to Feature Extraction from
Color and Stereo Images Seeks to extend ICA by contrasting it to the processing done in
neural receptive fields.
Hyvarinen – Survey on ICA
Lawrence – Face Recognition, A Convolutional Neural Network Approach Combines local image sampling, a SOM, and a convolutional
NN that provides partial invariance to translation, rotation, scaling, and deformations.
![Page 21: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/21.jpg)
ICA – Papers Sejnowski – Independent Component Representations for
Face Recognition
Sejnowski – A Comparison of Local vs Global Image Decompositions for Visual Speechreading
Bartlett – Viewpoint Invariant Face Recognition Using ICA and Attractor Networks
Bartlett – Image Representations for Facial Expression Coding
![Page 22: ICA Alphan Altinok. Outline PCA ICA Foundation Ambiguities Algorithms Examples Papers.](https://reader031.fdocument.org/reader031/viewer/2022032006/56649d6d5503460f94a4d43d/html5/thumbnails/22.jpg)
ICA – Links
http://sig.enst.fr/~cardoso/
http://www.cnl.salk.edu/~tewon/ica_cnl.html
http://nucleus.hut.fi/~aapo/
http://www.salk.edu/faculty/sejnowski.html