Introduction to Machine Learning - Kangwon leeck/AI2/06-3.pdf¢  2018. 10. 23.¢ ...

download Introduction to Machine Learning - Kangwon leeck/AI2/06-3.pdf¢  2018. 10. 23.¢  Introduction to Machine

of 18

  • date post

    27-Feb-2021
  • Category

    Documents

  • view

    0
  • download

    0

Embed Size (px)

Transcript of Introduction to Machine Learning - Kangwon leeck/AI2/06-3.pdf¢  2018. 10. 23.¢ ...

  • Multidimensional Scaling

       

         

     

     

    sr sr

    srsr

    sr sr

    srsr

    E

    ,

    ,

    ||

    |

    2

    2

    2

    2

    xx

    xxxgxg

    xx

    xxzz

    

     X

    2

     Given pairwise distances between N points,

    dij, i,j =1,...,N

    place on a low-dim map s.t. distances are preserved.

     z = g (x | θ ) Find θ that min Sammon stress (Sammon Mapping  gradient descent)

  • Map of Europe by MDS

    3

    Map from CIA – The World Factbook: http://www.cia.gov/

  • PCA, Sammon mapping (Welfare and poverty of one country)

    4

  • Isomap  A manifold of dimension n is a

    space that near each point resembles n-dimensional Euclidean space.

     Geodesic distance is the distance along the manifold that the data lies in, as opposed to the Euclidean distance in the input space

    5

  • Isomap  Instances r and s are connected in the graph if

    ||xr-xs||

  • 7 -150 -100 -50 0 50 100 150

    -150

    -100

    -50

    0

    50

    100

    150 Optdigits after Isomap (with neighborhood graph).

    0

    0

    7

    4

    6

    2

    5 5

    0 8

    7 1

    9 5

    3

    0

    4

    7

    84

    7

    8 5

    9

    1

    2

    0

    6

    1

    8

    7

    0

    7

    6

    9

    1

    9 3

    9 4

    9

    2

    1

    9 9

    6

    4 3

    2

    8

    2

    7

    1

    4

    6

    2

    0

    4

    6

    3 7 1

    0

    2

    2

    5

    2

    4

    8 1

    7 3

    0

    3 3 77

    9

    1 3

    3

    4

    3

    4

    2

    88 9 8

    4

    7 1

    6

    9

    4

    0

    1 3

    6

    2

    Matlab source from http://web.mit.edu/cocosci/isomap/isomap.html

  • Swissroll data (Isomap)

    8

  • 9

  • 10

  • Locally Linear Embedding  LLE has several advantages over Isomap

     Faster optimization and better results with many problems

    1. Given xr find its neighbors xs(r) 2. Find Wrs that minimize

    3. Find the new coordinates zr that minimize (eigen value problem)

    11

    2

      r s

    s rrs

    rXE )()|( xWxW

    2

      r s

    s rrs

    r zzE )()|( WWz

  • 12

  • LLE on Optdigits

    13

    -3.5 -3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 1

    1

    1

    1

    1

    1

    00

    7

    4

    6

    2

    5

    5

    0

    8

    7

    1

    9

    5

    3

    0

    47

    84

    7

    8

    5

    9 1

    2

    0

    6

    1 8

    7

    0

    76

    9 1

    9

    3

    9

    4

    9

    2

    1

    99

    6

    43 2

    8 2

    7

    14

    6

    2

    0

    4 6

    3

    7

    1

    0

    22 52

    48

    1

    7

    3

    0

    3 3

    77

    9 1

    3 34

    34 2

    8 8

    9 8 4

    7

    1

    6 9 4

    0

    1

    3 6

    2

    Matlab source from http://www.cs.toronto.edu/~roweis/lle/code.html

  • 14

  • 15

  • 16

  • T-distributed Stochastic Neighbor Embedding (t-SNE)

    17

    yi 에대해미분 gradient descent

  • Limitations of MDS, Isomap, LLE  MDS, Isomap, and LLE do not learn a general mapping

    function that will allow mapping a new test point

     The new point should be added to the dataset and the whole algorithm needs to be run once more

     So, we can not use them as features for classification

    18