x ∈ Rn

F

F

2480

978-1-4244-1821-3/08/$25.00 c©2008 IEEE

U s {u1, u2, . . . , us}xi ∈ R

l i = 1, . . . , nl(< s)

n(= s − l + 1)

xi = [ui, . . . , ui+l−1]T i

X ∈ Rn×l

X = [x1, . . . ,xn]T

n NX Xk ∈ R

n×l kXk∑N

k=1Xk = 0n×l χk(i) i

Xk

Xk = [χk(1), . . . ,χk(l)].

Xk

Ccomp =1

N

N∑

k=1

XkXTk .

λv = Ccompv.

λv = Ccompv =1

N

N∑

i=1

Xi(XTi v) =

1

N

N∑

i=1

Xiαi,

αi = XTi v ∈ R

nv

Xi

χi, i = 1, . . . , NlXk, k = 1, . . . , N∑Nl

i=1χi = 0n×1

Cv′ = λ′

v′,

C =1

Nl

Nl∑

i=1

χiχTi .

v′ =

1

Nlλ′

Nl∑

i=1

α′

iχi,

α′

i ∈ R

∑Nk=1

Xk = 0n×l∑Nli=1

χi = 0n×1

N Nl

Φ : Rl →

F i χk(i) Xk ∈R

n×l Φ(χk(i))∑N

k=1Ψ(Xk) = 0

Ψ(Xk) = [Φ(χk(1)), . . . ,Φ(χk(l))]

CΨcomp =

1

N

N∑

j=1

Ψ(Xj)Ψ(Xj)T .

2008 International Joint Conference on Neural Networks (IJCNN 2008) 2481

λw = CΨcompw

w

Ψ(Xi)

w =

N∑

i=1

Ψ(Xi)β(i),

β(i) ∈ Rl×1

λΨ(Xk)Tw = Ψ(Xk)T (CΨ

compw), k = 1, . . . , N.

λ

N∑

i=1

Ψ(Xk)T Ψ(Xi)β(i)

=1

N

N∑

j=1

Ψ(Xk)T Ψ(Xj)Ψ(Xj)T

N∑

i=1

Ψ(Xi)β(i),

k = 1, . . . , N i j Nl×NlKcomp

(Kcomp)ij = Ψ(Xi)T Ψ(Xj), i, j = 1, . . . , N.

(Kcomp)ij l× l

(Ψ(Xi)T Ψ(Xj))pq = Φ(χi(p)) · Φ(χj(q)),

= κ(χi(p), χj(q)), p, q = 1, . . . , l.

κ Kcomp

Kcomp

κ(χi(p), χj(q)) = exp(−‖χi(p) − χj(q)‖2

2σ2)

κ(χi(p), χj(q)) = (χi(p) · χj(q) + 1)d

σ dKcomp

NλKcompβ = K2compβ

β = [β(1)T , . . . ,β(N)T ]T ∈ RNl×1

Nλβ = Kcompβ.

λ1 ≥ · · · ≥ λn′(n′ < Nl)Kcomp β1, · · · , βn′

wk k =

1, . . . , n′

1 = (wk · wk)

=

N∑

i,j=1

(βk(i))T Ψ(Xi)T Ψ(Xj)βk(j)

=

N∑

i,j=1

(βk(i))T (Kcomp)ijβk(j)

= (βk)T · Kcompβk

= λk(βk · βk).

X Ψ(X)wk

Ψ(X)Twk =

N∑

i=1

Ψ(X)T Ψ(Xi)βk(i)

∑Nk=1

Ψ(Xk) = 0Ψ(Xi)

Ψ̃(Xi) = Ψ(Xi) − 1

N

N∑

k=1

Ψ(Xk).

K̃comp Kcomp

(K̃comp)ij

= (Ψ(Xi) − 1

N

N∑

p=1

Ψ(Xp))T (Ψ(Xj) − 1

N

N∑

q=1

Ψ(Xq))

= Ψ(Xi)T Ψ(Xj) − 1

N

N∑

p=1

Ψ(Xp)T Ψ(Xj)

− 1

N

N∑

q=1

Ψ(Xi)T Ψ(Xq) +

1

N2

N∑

p,q=1

Ψ(Xp)T Ψ(Xq)

= (Kcomp)ij − 1

N

N∑

p=1

Il(Kcomp)pj

− 1

N

N∑

q=1

(Kcomp)iqIl +1

N2

N∑

p,q=1

Il(Kcomp)pqIl,

Il l × l N × N(BI) (BI)ij = (1/N)Il

K̃comp = Kcomp − BIKcomp − KcompBI + BIKcompBI.

Ψ(X)

Ψ̃(X) = Ψ(X) − 1

N

N∑

k=1

Ψ(Xk).

T1, . . . ,TL

Ψ̃(Ti)T Ψ̃(Xj) K

testcomp

(Ktestcomp)ij = Ψ(Ti)

T Ψ(Xj).

2482 2008 International Joint Conference on Neural Networks (IJCNN 2008)

K̃testcomp

(K̃testcomp)ij

= (Ψ(Ti) − 1

N

N∑

p=1

Ψ(Xp))T (Ψ(Xj) − 1

N

N∑

q=1

Ψ(Xq))

L × N B′

I (B′

I)ij = (1/N)Il

K̃testcomp = K

testcomp − B

′

IKcomp − K

testcompBI + B

′

IKcompBI.

NYi ∈ R

n′×l

i = 1, . . . , Nc1, c2, . . . , cNC

Ni

ci CW

CB

CW =1

N

NC∑

i=1

∑

k∈Si

(Yk − Mi)(Yk − Mi)T ,

CB =1

N

NC∑

i=1

Ni(Mi − M)(Mi − M)T ,

Mi =1

Ni

∑

k∈Si

Yk,

M =1

N

N∑

k=1

Yk.

Si

ci CB

(CB) ≤ min(n′, (NC − 1)l).

NC − 1m

W ∈ Rn′

×m

W = arg maxW

|WTCBW|

|WT CW W| .

Zk ∈ Rm×l

Zk = WTYk, k = 1, 2, . . . , N.

2 14 690

2 9 683

2 13 297

2 34 351

2 6 432

2 8 768

2 60 208

3 13 178

‖Zi − Zj‖F = (tr[(Zi − Zj)T (Zi − Zj)])

1/2

= (

m∑

p=1

l∑

q=1

(Zi − Zj)2pq)

1/2,

tr[·]

2008 International Joint Conference on Neural Networks (IJCNN 2008) 2483

82.72 83.29 84.17

±2.14 ±1.95 ±1.44 ±1.93

95.46 95.76 96.50

±1.46 ±1.37 ±0.76 ±0.78

74.14 73.54 77.98

±4.74 ±4.95 ±2.28 ±2.40

68.18 88.37 83.13

±8.32 ±3.76 ±4.71 ±1.38

78.75 86.13 77.94

±3.24 ±3.73 ±3.53 ±6.59

70.72 71.64 71.27

±2.72 ±2.32 ±2.94 ±2.07

68.05 67.91 67.83

±5.18 ±5.38 ±5.26 ±4.62

79.09 88.69 89.16

±11.07 ±6.45 ±4.98 ±3.95

77.14 81.92 81.19

σ {1, 5, 10, 50}σ

NV − 1min(15, NV − 1) NV

k k = 3

84.19 84.07

±1.46 ±1.55 ±1.33 ±1.62

96.01 96.42 96.80

±0.87 ±1.35 ±0.56 ±0.54

78.32 78.21 78.55

±3.13 ±3.48 ±2.38 ±2.44

80.14 90.73 86.23

±4.20 ±2.03 ±2.22 ±2.21

78.54 91.04 83.53

±3.26 ±2.47 ±2.19 ±1.33

71.55 72.31 72.34

±1.98 ±1.51 ±1.43 ±1.42

58.58 76.84 72.95

±6.49 ±3.50 ±4.40 ±3.59

93.80 94.01 95.14

±2.39 ±1.98 ±2.32 ±2.07

80.14 85.70 83.56

2484 2008 International Joint Conference on Neural Networks (IJCNN 2008)

∼

2008 International Joint Conference on Neural Networks (IJCNN 2008) 2485