An introduction to Neural Networks - McGill CIM › ~jer › courses › ai › readings ›...
135
An introduction to Neural Networks Patrick van der Smagt Ben Krose ..
Transcript of An introduction to Neural Networks - McGill CIM › ~jer › courses › ai › readings ›...
An
introduction
to
Neural Networks
Patrick van der SmagtBen Krose..
sigmoidsgn semi-linearii i
original discriminant functionafter weight update
C
A
B
1 2
1
2
1
nφ
φ
ΨΩ
w
w
w w
Σ
Σ
01
2
3
+
−−1
+1
−1 +1
−1 +1
+1
errorsummergains
switchespatterninput
level
output
quantizer
switchreference
?
(−1,−1)
(−1,1)
AND
(1,1)
(1,−1)
?
XOROR
−2
1
1
−0.5
(1,1,1)
(-1,-1,-1)
1
1−0.5
a. b.
c
b a
−10
1
−10
1−1
0
1
−10
1
−10
1−1
0
1
−10
1
−10
1−1
0
1
−10
1
−10
1−1
0
1
2 4 6 8
+1
-0.5
-2-4
2 4 6-4
+1
-1
i
a very slow approximation
gradient
i+1uu
0 0.5 10
0.2
0.4
0.6
0.8
1
x
y
B
0 0.5 10
0.2
0.4
0.6
0.8
1
x
y
A
number of learning samples
errorrate
test set
learning set
0 0.5 10
0.2
0.4
0.6
0.8
1
x
y
A
0 0.5 10
0.2
0.4
0.6
0.8
1
x
y
B
errorrate
number of hidden units
learning set
test set
weight vectorpattern vector
a. b.
−0.5 0 0.5 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
: input pattern : weight vector
x2
x1
forwardfeed-vector
quantisation
-2
-1
0
1
2 -2
-1
0
1
2
00.250.5
0.75
1
-2
-1
0
1
2 -2
-1
0
1
2
0255
5
1
h(i,k)
Iteration 0 Iteration 200 Iteration 600 Iteration 1900
lateral distance
excitation
LTMLTMSTM activity pattern
STM activity pattern
category representation field
feature representation field F1
F2
input
neurons
neurons
N
M++
+− +
++
−
+
+
F1
F2
G2
G1 reset
input
backward LTM from:
patterninput
4321outputoutputoutputoutput
activenotnot
active
notactive active
not
notactive
notactive
activenot
reinforcementinternal
reinforcement
state vector
detectorreinforcement
ASE systemdecoder
ACE
F
x
θ
1
4
3
2
tool frame
base frame
θ
θ
NetworkNeural
NetworkNeural Plant
’
1ε
x x’θ
θ
ε
θ PlantNetworkNeuralx x’θ
manipulator
modelinverse dynamics
d
f
i(t)θ
-+
++
K
(t)θ(t) (t)T T
T (t)
θ θ
(t)
(t)
(t)
(t)
(t)
(t)
x
x
13,1
1,1
i3
i2
i1 (t)
(t)
(t)T
T
T
TT
T
32
1(t)
(t)(t)
Σ
Σ
Σ
θ
θ
θ
(t)(t)(t)d3
d2
d1
f
f
g
g
1
13
1
13
x1,2
1,3x
x13,3
13,2x
1θ
302010−π
0
π
t/s
EDCBA
units
units45 output
29 hidden
sharp left sharp right
30x32 video input retina
8x32 range finderinput retina
straight ahead
66
neuronneuron
1 2neuronneuronneuron
0
neuron3 4 5
76neuron neuron
(unused)
orderingof the neurons:
U U ll-1
ll-1
l
l
l( (
(
(
(
)
) ) )
)
)(l-1
l-1 )(n+vv,n
nv
n
n’
n
ua
cb
v
u
u
""21100 UUUUU
b.a.
d.c.
b.a.
Network
Bus interface
or Symbolics)(DEC VAX
Bus interface
or Symbolics)(DEC VAX
Bus interface
or Symbolics)(DEC VAX
Bus interface
or Symbolics)
Front end 3
Front end 2
Front end 1
(DEC VAXFront end 0
Nexus
32
10
sequencersequencer
sequencersequencer
Connection Machine I/O System
16,384 ProcessorsConnection Machine
16,384 ProcessorsConnection Machine
16,384 ProcessorsConnection MachineConnection Machine
16,384 Processors
displayGraphicsData
VaultDataVault
DataVault
C+A*B
cellsystolic
c+a*b
b
b
a
a
c
a11
c11
11b
a 32 22a12a
a 21a 31
21b b 22 23b
b 1312b
21 12ccc cc 22 1331
14233241c c c c
42 33 24c c c3443 cc
Warp Interface & Host
address
X
Yn2
cell1
cell cell
outputs
inputs
N
M
receptor for
row 5laser for
row 1laser for
maskweight
column 6
Intensity
outV
outV
3.6
3.4
3.2
3.0
2.8
2.6
2.4
2.28765432
(b)(a)
+
+−
−
ganglionoutput
photo-receptor
δ j
j
j
a
LP
processorlearning
neural state registers
learning functions
learning register
memory
synaptic
readable
and
loadable
parallel
controland
functionlinearnon-
controllearning
accumulator
alu
addersof
tree
mul
mul
mul
mul
Fct. Fct. Fct. Fct.
