of 18

• date post

27-Feb-2021
• Category

## Documents

• view

0

0

Embed Size (px)

### Transcript of Introduction to Machine Learning - Kangwon leeck/AI2/06-3.pdf¢  2018. 10. 23.¢ ...

• Multidimensional Scaling

   

     

 

 

sr sr

srsr

sr sr

srsr

E

,

,

||

|

2

2

2

2

xx

xxxgxg

xx

xxzz



 X

2

 Given pairwise distances between N points,

dij, i,j =1,...,N

place on a low-dim map s.t. distances are preserved.

 z = g (x | θ ) Find θ that min Sammon stress (Sammon Mapping  gradient descent)

• Map of Europe by MDS

3

Map from CIA – The World Factbook: http://www.cia.gov/

• PCA, Sammon mapping (Welfare and poverty of one country)

4

• Isomap  A manifold of dimension n is a

space that near each point resembles n-dimensional Euclidean space.

 Geodesic distance is the distance along the manifold that the data lies in, as opposed to the Euclidean distance in the input space

5

• Isomap  Instances r and s are connected in the graph if

||xr-xs||

• 7 -150 -100 -50 0 50 100 150

-150

-100

-50

0

50

100

150 Optdigits after Isomap (with neighborhood graph).

0

0

7

4

6

2

5 5

0 8

7 1

9 5

3

0

4

7

84

7

8 5

9

1

2

0

6

1

8

7

0

7

6

9

1

9 3

9 4

9

2

1

9 9

6

4 3

2

8

2

7

1

4

6

2

0

4

6

3 7 1

0

2

2

5

2

4

8 1

7 3

0

3 3 77

9

1 3

3

4

3

4

2

88 9 8

4

7 1

6

9

4

0

1 3

6

2

Matlab source from http://web.mit.edu/cocosci/isomap/isomap.html

• Swissroll data (Isomap)

8

• 9

• 10

• Locally Linear Embedding  LLE has several advantages over Isomap

 Faster optimization and better results with many problems

1. Given xr find its neighbors xs(r) 2. Find Wrs that minimize

3. Find the new coordinates zr that minimize (eigen value problem)

11

2

  r s

s rrs

rXE )()|( xWxW

2

  r s

s rrs

r zzE )()|( WWz

• 12

• LLE on Optdigits

13

-3.5 -3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 1

1

1

1

1

1

00

7

4

6

2

5

5

0

8

7

1

9

5

3

0

47

84

7

8

5

9 1

2

0

6

1 8

7

0

76

9 1

9

3

9

4

9

2

1

99

6

43 2

8 2

7

14

6

2

0

4 6

3

7

1

0

22 52

48

1

7

3

0

3 3

77

9 1

3 34

34 2

8 8

9 8 4

7

1

6 9 4

0

1

3 6

2

Matlab source from http://www.cs.toronto.edu/~roweis/lle/code.html

• 14

• 15

• 16

• T-distributed Stochastic Neighbor Embedding (t-SNE)

17