CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis.
19
CHAPTER 14 CHAPTER 14 Clustering and Clustering and Unsupervised Classification Unsupervised Classification CLASSIFICATION CLASSIFICATION A. Dermanis
-
Upload
eddy-hackley -
Category
Documents
-
view
244 -
download
2
Transcript of CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis.
CHAPTER 14CHAPTER 14
Clustering andClustering andUnsupervised ClassificationUnsupervised Classification
CLASSIFICATIONCLASSIFICATION
A. Dermanis
m = x1
N i xi
CT = ST
1
N
i xi
ST = (x – mi)(x – mi)T
xi
Si = (x – mi)(x – mi)T
Ci = Si
1ni
mi = xxi
1ni
Clustering = dividing of N pixels into K classes ω1, ω2, …, ωK Clustering = dividing of N pixels into K classes ω1, ω2, …, ωK
global mean
ClusteringClustering
total scatter matrix:
mean of class ωi :scatter matrix of class ωi :
total covariance matrix:
covariance matrix of class ωi :
A. Dermanis
Sex = ni (mi – m)(mi – m)T i
i xi
Sin = Si = (x – mi)(x – mi)T i
Clustering criteria Clustering criteria
overall compactness of the clusters internal scatter matrix
degree of distinction between the clusters external scatter matrix
Optimal algorithm: Sin = min and Sex = max (simultaneously)
Problem: How many clusters ? (K = ?)
Extreme choice: K = N (each pixel a different class) k = {xk}
Extreme choice: K = 1 (all pixels in a single class) Sin = ST, Sex = 0
ST = Sin + Sex = constantST = Sin + Sex = constant
mk = xk, Sk = 0, Sin = Sk = 0 = min, Sex = ST =maxk
A. Dermanis
F
E
D
C
B
AG
GLO
ME
RA
TIV
E
DIV
ISIV
E
1 2 3 4 5 6A
Hierarchical Clustering
Agglomerative clustering:
Unifying at each step the two closest clusters
Divisive clustering :
Dividing at each step the most disperse cluster into two new clusters
Needed:
Unification criteria.Division criteria and procedures.
A. Dermanis
A B D
F
E
D
C
B
1 2
3
4
5
6
AG
GLO
ME
RA
TIV
E
DIV
ISIV
E
1 2 3 4 5 6A
Hierarchical Clustering
C E F
A. Dermanis
Distance between two clusters (alternatives): Distance between two clusters (alternatives):
i kki nn
d x x
xx ||||1
)()( |||| xxxxxx T
mean distance:
||||min,
min xxxx
ki
d
minimum distance:
||||max,
max xxxx
ki
d
maximum distance:
Used in agglomerative and divisive clustering Used in agglomerative and divisive clustering
A. Dermanis
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
The K-means or migrating means algorithm The K-means or migrating means algorithm
A. Dermanis
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
The K-means or migrating means algorithm The K-means or migrating means algorithm
Step 0:
Selection of K = 3 pixels as initial positions of means
A. Dermanis
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
Step 1:
Assignment each pixels to the clusterof its closest mean
Calculation of the new meansfor each cluster
The K-means or migrating means algorithm The K-means or migrating means algorithm
A. Dermanis
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
Step 2:
Assignment each pixels to the clusterof its closest mean
Calculation of the new meansfor each cluster
The K-means or migrating means algorithm The K-means or migrating means algorithm
A. Dermanis
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
Step 3:
Assignment each pixels to the clusterof its closest mean
Calculation of the new meansfor each cluster
The K-means or migrating means algorithm The K-means or migrating means algorithm
A. Dermanis
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
Step 4:
Assignment each pixels to the clusterof its closest mean
All pixels remain in the same cluster.Means remain the same.
Termination of the algorithm !
The K-means or migrating means algorithm The K-means or migrating means algorithm
A. Dermanis
The Isodata AlgorithmThe Isodata Algorithm
A variant of the K means algorithm.In each step one of 3 additional procedures can be used:
A variant of the K means algorithm.In each step one of 3 additional procedures can be used:
1. Cluster ELIMINATIONELIMINATION
2. Cluster UNIFICATIONUNIFICATION
3. Cluster DIVISIONDIVISION
Eliminate clusterswith very few pixels
Unify pairs of clustersVery close to each other
Divide large clusterswhich are elongatedInto two clusters
A. Dermanis
The Isodata AlgorithmThe Isodata Algorithm
1. Cluster ELIMINATIONELIMINATION
Eliminate clusterswith very few pixels
A. Dermanis
The Isodata AlgorithmThe Isodata Algorithm
2. Cluster UNIFICATIONUNIFICATION
Unify pairs of clustersVery close to each other
A. Dermanis
The Isodata AlgorithmThe Isodata Algorithm
3. Cluster DIVISIONDIVISION
Divide large clusterswhich are elongatedInto two clusters
A. Dermanis
The unification process The unification process
The division process The division process
The Isodata AlgorithmThe Isodata Algorithm
m2
m1
m2+kσ2
m2–kσ2
A. Dermanis
K-means: 5 classes
K-means: 7 classes K-means: 9 classes
K-means: 3 classes
Examples ofclassifiction usingthe K-mean algorithm
Examples ofclassifiction usingthe K-mean algorithm
A. Dermanis
ISODATA : 3 classes ISODATA : 5 classes
ISODATA : 7 classes ISODATA : 9 classes
Examples ofclassifiction usingthe ISODATA algorithm
Examples ofclassifiction usingthe ISODATA algorithm
A. Dermanis
