Introduction to Machine Learning - Kangwonleeck/AI2/06-3.pdf · 2018. 10. 23. · Introduction to...
18
Transcript of Introduction to Machine Learning - Kangwonleeck/AI2/06-3.pdf · 2018. 10. 23. · Introduction to...
Multidimensional Scaling
srsr
srsr
srsr
srsr
E
,
,
||
|
2
2
2
2
xx
xxxgxg
xx
xxzz
X
2
Given pairwise distances between N points,
dij, i,j =1,...,N
place on a low-dim map s.t. distances are preserved.
z = g (x | θ ) Find θ that min Sammon stress(Sammon Mapping gradient descent)
Map of Europe by MDS
3
Map from CIA – The World Factbook: http://www.cia.gov/
PCA, Sammon mapping (Welfare and poverty of one country)
4
Isomap A manifold of dimension n is a
space that near each point resembles n-dimensional Euclidean space.
Geodesic distance is the distance along the manifoldthat the data lies in, as opposed to the Euclidean distance in the input space
5
Isomap Instances r and s are connected in the graph if
||xr-xs||<e or if xs is one of the k neighbors of xr
The edge length is ||xr-xs||
For two nodes r and s not connected, the distance is equal to the shortest path between them
Once the NxN distance matrix is thus formed, use MDS to find a lower-dimensional mapping
6
7-150 -100 -50 0 50 100 150
-150
-100
-50
0
50
100
150Optdigits after Isomap (with neighborhood graph).
0
0
7
4
6
2
55
08
71
9 5
3
0
4
7
84
7
85
9
1
2
0
6
1
8
7
0
7
6
9
1
93
94
9
2
1
99
6
43
2
8
2
7
1
4
6
2
0
4
6
37 1
0
2
2
5
2
4
81
73
0
3 377
9
13
3
4
3
4
2
889 8
4
71
6
9
4
0
1 3
6
2
Matlab source from http://web.mit.edu/cocosci/isomap/isomap.html
Swissroll data (Isomap)
8
9
10
Locally Linear Embedding LLE has several advantages over Isomap
Faster optimization and better results with many problems
1. Given xr find its neighbors xs(r)
2. Find Wrs that minimize
3. Find the new coordinates zr that minimize (eigen value problem)
11
2
r s
srrs
rXE )()|( xWxW
2
r s
srrs
r zzE )()|( WWz
12
LLE on Optdigits
13
-3.5 -3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.51
1
1
1
1
1
00
7
4
6
2
5
5
0
8
7
1
9
5
3
0
47
84
7
8
5
91
2
0
6
18
7
0
76
91
9
3
9
4
9
2
1
99
6
432
82
7
14
6
2
0
46
3
7
1
0
22 52
48
1
7
3
0
33
77
91
334
342
88
98 4
7
1
6 94
0
1
36
2
Matlab source from http://www.cs.toronto.edu/~roweis/lle/code.html
14
15
16
T-distributed Stochastic Neighbor Embedding (t-SNE)
17
<고차원> <저차원>
yi 에대해미분 gradient descent
Limitations of MDS, Isomap, LLE MDS, Isomap, and LLE do not learn a general mapping
function that will allow mapping a new test point
The new point should be added to the dataset and the whole algorithm needs to be run once more
So, we can not use them as features for classification
18
