Unconventional computing with stochastic magnetic tunnel ... · PDF fileThe stochastic...

Click here to load reader

  • date post

    19-Feb-2019
  • Category

    Documents

  • view

    220
  • download

    1

Embed Size (px)

Transcript of Unconventional computing with stochastic magnetic tunnel ... · PDF fileThe stochastic...

Unconventional computing with stochastic magnetic

tunnel junctionsAlice Mizrahi, Tifenn Hirtzlin, Matthew Daniels, Nicolas Locatelli, Akio Fukushima, Hitoshi Kubota, Shinji Yuasa,

Mark Stiles, Julie Grollier, Damien Querlioz

Lowering the energy consumption reduces the stability

2

Reference magnet

Tunnel barrier

Free magnet

thermal noise

E P AP

Low Resistance High Resistance

Low E

Small size

Low write energy

Unstable

Different energy barriers for different applications

3

Energy barrier

60 kBT

15 kBT

years

ms

s

Memory for storage (MRAM)

Synaptic weight for continuously learning neural net

Stochastic regime6 kBT

Stochastic spiking neuron

Stochastic oscillator

Stochastic bitstream generator

A stochastic oscillator powered by thermal noise

4

AP

P

1

0

No external energy source

Two-state signal easy to convert to digital signal

Stochasticity well understood and controlled

5

Spin torque controls the stochasticity

Stochastic analog to digital conversion

dc current

Vo

ltag

e

-50 0 500

500

1000

Ra

te (

Hz)

Current (A)

APP

kBTI > 0

Stabilizes the antiparallel state

Stabilizes the parallel state

kBTI < 0

P AP 0.100 0.125 0.150

400

500

600

Resis

tan

ce (

)

Time (s)

0.100 0.125 0.150

400

500

600

700

Re

sis

tan

ce

(

)

Time (s)

0.100 0.125 0.150

400

500

600

Re

sis

tan

ce

(

)

Time (s)

6

1. Population coding with stochastic spiking neurons

2. Noise-induced synchronization of a stochastic oscillator

Encoding information in the spike rate of a stochastic neuron

7

Time

Vo

ltag

e

Rate coding Robust to errors Fast approximate results Rate varies with value of stimulus

Kumano et al., J. Neurophysiology 2010

Tuning curve

Direction

Neuron

8

The stochastic magnetic tunnel junction emulates a stochastic spiking neuron

Switches spikesStimulus = current

-50 0 500

500

1000

Rate

(H

z)

Current (A)

Time

Vo

ltag

e

0.100 0.125 0.150

400

500

600

700

Resis

tan

ce (

)

Time (s)

9

rates

Neurons

Firi

ng

rate

Firi

ng

rate

Preferred direction

Preferred direction

Each neuron processes a range of inputs (directions)

The direction can be inferred from the rates of the neurons

Robust coding of information by a population of neurons

10

=

=1

9

(eff)

eff = bias

Shift the tuning curves

-400 -200 0 200 400

0.0

0.2

0.4

0.6

0.8

1.0

No

rma

lize

d R

ate

Current (A)

-300 0 300

0.0

0.5

1.0

-300 0 300 -300 0 300

H(I

) (a

.u.)

Current (A) Current (A) Current (A)

12 3 4 5 6

7 8 9

Mizrahi et al., Nature Communications, 2018

Constructing non-linear transformation with a population of artificial neurons

11

OUT =

=1

IN

Weights

Input

Output

1IN IN

INInput rates

1OUT OUT

OUTOutput rates

Input value I

Transformation F

Output value O = F(I)

Stochastic junctions as neurons (NIN)

Stochastic junctions as neurons (NOUT)

Stable junctions as binary encoded synaptic weights

(NIN x NOUT)

Transfer of coded information from sensory to motor networks

Salinas et Abbott, J. Neuroscience, 1995

An artificial neural net with magnetic tunnel junctions

12

Learning rule:Catch do nothing

Miss modify the weights

Simulations with experimentally verified modelEneuron = 6 kBT

The system is capable of learning a transformation

1IN IN

INInput rates

1OUT OUT

OUTOutput rates

Direction

0 00

13

CMOS overhead: Switch detection & count Calculation (rate and weights) Learning rule

Neurons

Synaptic Weights

Area and energy consumption of our system (Work of T. Hirtzlin at C2N)

Operation7.8 nJ

12,000 m2

CMOS-only neurons:Area > 20,000 m2

Energy of operation > 20 nJ

Key asset = stochastic analog to digital conversion

The system consumes less energy and area than CMOS-only implementations

2 3 4 5 6 7 8 9 10 11 12

0.001

0.01

0.1

1

Error (%)

No

rma

lize

d p

ow

er

co

nsu

mp

tio

n

46 neurons

Increase energy barrier Ew

14

Using unreliable synapses lowers the power consumption

2

Write power

Write current

(Sato et al., APL 2014)

2 3 4 5 6 7 8 9 10 11 12

0.001

0.01

0.1

1

10 neurons

20 neurons

100 neurons

Error (%)

No

rma

lize

d p

ow

er

co

nsu

mp

tio

n

46 neurons

15

Using unreliable synapses lowers the power consumption

2

Write power

Write current

(Sato et al., APL 2014)

0.001

0.01

0.1

1

10

100

2 3 4 5 6 7 8 9 101112

10

12

14

16

18

20

100

46

20

10 neurons

OptimumNo

rma

lize

d p

ow

er

co

nsu

mp

tio

nN

um

be

r

of

ne

uro

ns

En

erg

y b

arr

ier

(k

BT

)

Error (%)

16

Using unreliable synapses lowers the power consumption

2

Write power

Write current

(Sato et al., APL 2014)

0.001

0.01

0.1

1

10

100

2 3 4 5 6 7 8 9 101112

10

12

14

16

18

20

100

46

20

10 neurons

OptimumNo

rma

lize

d p

ow

er

co

nsu

mp

tio

nN

um

be

r

of

ne

uro

ns

En

erg

y b

arr

ier

(k

BT

)

Error (%)

17

Using unreliable synapses lowers the power consumption

Example:For 3% precision 54 neurons in each population (2916 weights) Ew = 12 kBT

2

Write power

Write current

(Sato et al., APL 2014)

A low energy continuously learning neural net with magnetic tunnel junctions

18

Mizrahi et al., Nature Communications, 2018 Mizrahi et al., J. Applied Physics, 2018

Fully stochastic magnetic tunnel junction as neuron

Slightly stochastic magnetic tunnel junction as synapse

Continuous learning

+ +

Artificial neural networkLow energy

Capable of learningResilient to variability & unreliability

Adaptable to changes

=

0.100 0.125 0.150

400

500

600

Re

sis

tan

ce

(

)

Time (s)

Key application: smart sensors

19

1. Population coding with stochastic spiking neurons

2. Noise-induced synchronization of a stochastic oscillator

Noise can induce low-power synchronization

20

Deterministic Vac > Vc

Vc

-Vc

Vac

R

R

Vc

-Vc

VacVc

-Vc

Vc

-Vc

Vc

-Vc

Vac

R

Subthreshold Vac < Vc

Optimal noise range

0 5 10 15 20 25 30 35 40 45 500

10

20

30

40

50

60

70

80

0 50 100 1500 50 100 150 0 50 100 150 200

1

-Vac

RP

Fre

que

ncy (

Hz)

Electrical noise (mV)

1

Fac

2

3

2 3RAP

+Vac

Re

sis

tance

sta

te

Time (ms)Time (ms)

Time (ms)

D

rive

vo

ltage

Noise controls frequency and phase locking

21

Vac = 63 mV while Vc = 235 mV @0K

E = 22.5 kBTNatural frequency 0.1 Hz

Thermal noise (room temperature)

+Electrical noise (white

Gaussian)

Mizrahi et al., Scientific Reports, 2016

Sync

22

Fac = 50 Hz Vac = 82 mV

Natural frequency 0.1Hz

0 20 40 60 80 100

0

10

20

30

40

50

101

102

103

104

0

10

20

30

40