1

Noise-Induced Increases in the Duration of Transient Oscillations in Ring Neural Networks

and Correlations in the Periods of Ring Oscillators

Yo Horikawa Kagawa University

Japan

2

1. Problem Ring network of neurons with unidirectional coupling with noise

dxn(t)/dt = -xn(t) + f(xn-1(t)) + σxnn(t) (x0 = xN, 1 ≤ n ≤ N)

f(x) = tanh(gx) E{nn(t)} = 0, E{nn(t)nn’(t’)} = δn,n’·δ(t - t’) (A1)

xn: state of neuron n N: number of neurons f(x) = tanh(gx): output of neurong: coupling gain (|g| > gth(N) ≥ 1) nn(t): spatiotemporal white noise σx: strength of noise

45

N12

6

78

3 f (x ) = tanh(gx ) (g = -10)

-1

0

1

-2 -1 0 1 2x

3

1. Effects of noise on the duration of transient oscillations

Number of neurons is even (N = 2M). → Network is bistable. Duration T of the transient oscillations increases exponentially with the number N of neurons. T ∝ exp(N) → Spatiotemporal noise of intermediate strength (σx) increases the mean duration (m(T)) of the transient oscillations.

0

10000

20000

30000

40000

50000

60000

0.00 0.01 0.02 0.03 0.04 0.05σ x

m(T

) simulationFPT

4

2. Effects of noise on the periods of stable oscillations

Number of neurons is odd (N = 2M + 1). → Ring oscillator (Ring of inverters (NOT gates)) The oscillations are stable.

→ Spatiotemporal noise causes positive correlations in a series of the half periods (a series of the widths of the pulses) of the oscillations.

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.0 0.1 0.2 0.3 0.4 0.5ω/ 2π

S

simulationEXP

Power spectrum S(ω) of a series {Tj} of the half periods of the oscillations

-0.8

-0.4

0.0

0.4

0.8

0.0 2.0 4.0 6.0 8.0t

x n

T1 T2 T3 T4 ･･･

5

2. Effects of noise on the duration of transient oscillations Number of neurons is even (N = 2M). → Network is bistable.

Transient oscillations: Traveling waves of two inconsistencies in the signs of the states of neurons

Changes in duration of transient oscillations due to noise

45

2m

12

6

78

3

x = (+ - ･･･ + -) OR

x = (- + ･･･ - +)

+ +- --

6

We use excitatory coupling.

Neurons are separated in two blocks of the same signs.

(- - - - - - - - - + + + + + + + + + +)(+ - - - - - - - - - + + + + + + + + +)

(+ + - - - - - - - - - + + + + + + + +) (+ + + - - - - - - - - - + + + + + + +)

(+ + + + + + + + + + + + - - + + +) (+ + + + + + + + + + + + + - + + +) (+ + + + + + + + + + + + + + + + +)

Demonstration Noise changes the duration of the transient oscillations.

dx1/dt = -x1 + f(xN)

dxn/dt = -xn + f(xn - 1) (2 ≤ n ≤ N) (A2)

xn: state of neuron n f(x) = tanh(gx): output of neurong: coupling gain (g > 1)

t

7

Difference between the velocities of two blocks: v0 – v1 < 0

→ Changes in the length of a smaller block: l(t) with dl(t)/dt = v0 – v1

→ Duration T of the oscillation with the condition l(T) = 0

(+ + + + + + + + + + + + + - - - - - - - - - - - - - + + + + + + + + + + + + + +)

v1 v0

l(t)

8

Sigmoidal function: f(x) = tanh(gx) → Sign function: sgn(x)

dxn/dt = -xn - 1 + σxnn(t) (xn -1 < 0)

= -xn + 1 + σxnn(t) (xn -1 > 0) (A3)

x(t0) ≈ - 1

sgn (x )

-1

0

1

-2 -1 0 1 2x

dl(t)/dt = v0 - v1 = 1/Δt0 - 1/Δt1

l(t)

v1v0

Δt1 ≈ τlog2 + log{1 - exp[-(t1 - t0)/τ]} + σ’n1(t)

Δt0 ≈ τlog2 + log{1 - exp[-(t0 – t-1)/τ]} + σ’n2(t)

τdxn/dt = -xn + 1

τdxn/dt = -xn - 1Δt0

Δt1xn(t)

xn -1(t)

t

t

t1 t0 dl(t)/dt ≈ 1/(log2)2·[exp(-log2·(L - l))

- exp(-log2·l)] +σln(t)

σl2 = 2σ2(tp)/(log2)4 = 3/[4(log2)4]·σx

2

E{n(t)} = 0, E{n(t)n(t’)} = δ(t - t’) (A4)

9

Changes in the length of a smaller block:

dl(t)/dt ≈ 1/(log2)2·[exp(-log2·(L - l)) - exp(-log2·l)] +σln(t)

σl2 = 2σ2(tp)/(log2)4 = 3/[4(log2)4]·σx

2 ≈ 3.25 σx2

E{n(t)} = 0, E{n(t)n(t’)} = δ(t - t’) (A4)

The duration T of the transient oscillations is dealt with the first passage time (FPT) problem of Eq. (A4) from l(0) = l0 to l(T) = 0 or L.

0

10

20

30

40

0 2000 4000 6000 8000 10000t

l σ x = 0 rand1 rand2 rand3

T3

T1 T0 T2

The mean m(T(l0)) of the FPT beginning from the initial block length l0 are given in the integral form.

10

]dξπ(ξ))σ(dη)η(π[2)|0);(( 0

0

2/

η

1200

l L

lllTm

y

l

ay )dησ

)η(2exp()(π 2

a(l) = 1/(log2)2·[exp(-log2·(L - l)) - exp(-log2·l)] (A5)

Fig. A1. Mean duration m(T(l0)) of the transient oscillations vs. SD σx of the noise. N = 40 and l0 = 15. FPT for simulation with tanh(gx) (g = 10) is multiplied by the ratio (1.24) of the durations without noise.

The mean duration m(T) of the transient oscillations occurring from a fixed initial block length l0 increases in the presence of noise with intermediate strength.

0

10000

20000

30000

40000

50000

60000

0.00 0.01 0.02 0.03 0.04 0.05σ x

m(T

) simulation (sigmoid)FPT (modified)simulation (sgn)FPT

11

When an initial length l0 of a smaller block changes, the noise strength σx at a peak of the mean duration m(T) changes.

The mean duration m(T) of the transient oscillations occurring under random initial conditions decreases monotonically with the strength σx of noise.

0

10000

20000

30000

40000

50000

60000

0.00 0.01 0.02 0.03 0.04 0.05σ x

m(T

) simulation l0=15FPT l0=15FPT l0=12FPT l0=13FPT l0=14

Fig. A2. Mean duration m(T(l0)) of the transient oscillations vs. SD σx of the noise. N = 40 and l0 = 12, 13, 14, 15.

3. Effects of noise on the periods of stable oscillations Number of neurons is odd (N = 2M + 1). → Ring oscillator: Ring of inverters (NOT gates) Stable oscillations of rectangular waves

12

45

2m+1

12

6

78

3

+ - +

- - +- + +- + -+ + -+ - -+ - +

t

+ +-

Demonstration (N = 3)

13

Variations in a series of the half periods Tj of the oscillation due to noise

･･･ Tj -1 Tj Tj +1 ･･･

0.0

0.5

1.0

1.5

2.0

0 20 40 60 80 100j

T j

N = 3

14

f(x) = tanh(gx) → Sign function: sgn(x)

dxn/dt = -xn - 1 + σxnn(t) (xn -1 < 0)

= -xn + 1 + σxnn(t) (xn -1 > 0)

Condition for the stable oscillation in the absence of noise:

xm = -(xm + 1)exp(-Tm) + 1 (xm > 0)

Propagation time of the inconsistency per neuron: Δtm = log(xm + 1)

Half period: Tm = NΔtm

Maximum value of the state of neurons: xm = tanh(Tm)

xn (t)

xn -1(t)xm

Tm

Δtm

N = 3

15

The propagation time Δtj of the jth passing of the inconsistency in the presence of noise:

Δtj ≈ log{1 + xm + exp[-(N - 1)m(Δt)][(N - 1)/N]T’j} + σtnj(t) ≈ Δtm - exp(-Tm)[(N - 1)/N]Tj’ + σtnj(t)

Tj = tj - tj -1, Tj’ = Tj’ = Tj - Tm

σt2 ≈ σx

2/2·{[1 - exp(-2Δtm)] + [1 – exp(-2(N - 1)Δtm)]exp(-2Δtm)} (A6)

tj -1

xn(tj -1)

tj

xn(tj) = (xn(tj -1) - 1)exp(-Tm) + 1 + σt’’nj(t)

Δtj

dxn(t)/dt = -xn(t) - 1 + σxnn(t) (xn -1(t) < 0)

xn(tj + Δtj) = 0

Tj

16

Changes in the half period Tj at the location l in the network is approximated as

dTj(l)/dl = d(tj - tj-1)/dl ≈ Δtj - Δtj-1

≈ β(Tj - Tj-1) + σt(nj - nj-1)

β = exp(-Tm)·(N - 1)/N, Tj(0) = Tj-1(N)

Tj(0) = Tj -1(N) (nj: white noise along tj) (A7)

Z transform ZT of Tj is given by

dZT(l)/dl = β(1 - z-1)ZT(l) + (1 - z-1)Zn(l)

ZT(0) = z-1ZT(N) (A8)

'd)'())')(1(βexp()1()0())1(βexp()( 1

0

11 llZllzzZlzlZ n

N

TT

17

The power spectrum S(ω) of Tj is obtained as

S(ω) = E{|ZT(N)|2 z = exp(iω)}

= σt2/β·{exp[2βN(1 - cos(ω))] – 1}

/{1 + exp[2βN(1 - cos(ω))] - 2exp[βN(1 - cos(ω))]cos[ω - βNsin(ω)]} (A9)

The power spectrum is expressed with exponentials of cos(ω) and sin(ω).

When |βN| < 1, a sequence of the half periods is approximated by the first-order autoregressive (AR) process.

Tj(N) = tj(N) - tj -1(N) = ≈ βN/2·(Tj -1(N) + Tj(N)) + σTnj (approximation of the integral with the trapezoidal rule)

= φ1Tj -1(N) + σTnj

φ1 = βN/(2 - βN) > 0, σT2 = σt

2N/(1 - βN/2)2

Power spectrum: S(ω) = σT2/(1 - 2φ1cos(ω) + φ1

2)

Autocovariance function: E{(Tj(l) - Tm)(Tj-k(l) - Tm)} = σT2/(1 - φ1

2)·φ1k (A10)

2iω-iω

0

2 |))e1(βexp(e1|/d)))ωcos(1)((β2exp())ωcos(1(2 NllNN

T

lnlT jtj

Ld))(β(

0

18

･ N = 3

xm = (-1 + 51/2)/2 ≈ 0.618

Δtm = log((1 + 51/2)/2) ≈ 0.481

Tm = log(2 + 51/2) ≈ 1.44

dTj(l)/dl = β(Tj - Tj -1) + σt(nj - nj -1)

β = 2/3·(-2 + 51/2) ≈ 0.157 σt

2 = 2(-2 + 51/2)σx2 ≈ 0.472σx

2

S(ω) = σt2/β·{exp[2βN(1 - cos(ω))] – 1}

/{1 + exp[2βN(1 - cos(ω))] - 2exp[βN(1 - cos(ω))]cos[ω - βNsin(ω)]}

Tj(l) = φ1Tj -1(N) + σTnj

φ1 = (-1 + 51/2)/4 ≈ 0.309

σT2 = 3/4·(1 + 51/2)σx

2 ≈ 2.43σx2

S(ω) = σT2/(1 - 2φ1cos(ω) + φ1

2)

Noise causes positive correlations in a series of the half periods.

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.0 0.1 0.2 0.3 0.4 0.5ω / 2π

S

simulation (g = -10.0)simulation (sign)EXP (Eq. (A9))AR1 (Eq. (A10))

Fig. A3. Power spectra S(ω) of a series Tj of the half periods in the ring oscillator of three neurons (N = 3) in the presence of noise of σx = 0.1.

19

Circuit experiment N = 3

Noise source: untuned FM radio, Inverter: TC4049, Amplifiers: RC4558 R = 10kΩ, C = 1μF, (time constant: CR = 10ms)

Oscilloscope

Small shift in a zero-level causes large negative correlations in a series of the half periods. → A series of the periods: Tpj = T2j + T2j+1

㊉

Inverter

x10

Inverting Amplifier

C

Summing Amplifier

R

FM Radio

㊉

Inverter

x10

Inverting Amplifier

C

Summing Amplifier

R㊉

Inverter

x10

Inverting Amplifier

C

Summing Amplifier

R

･･･ T2j -2 T2j T2j+1 T2j -2 ･･･

･･･ Tpj -1 Tpj ･･･

20

Downsampling by a factor 2 of a series of the sums of the successive 2 half periods → a series of the periods: Tpj = T2j + T2j+1

The power spectrum S(ω; Tpj) of a series of the periods:

S(ω; Tpj) = (1 + cos(ω/2))S(ω/2; Tj) + (1 + cos(π - ω/2))S(π - ω/2; Tj) (A11)

S(ω; Tj) = σt2/β·{exp[2βN(1 - cos(ω))] - 1}

/{1 + exp[2βN(1 - cos(ω))] - 2exp[βN(1 - cos(ω))]cos[ω - βNsin(ω)]} (A9)

S(ω; Tj) = σT2/(1 - 2φ1cos(ω) + φ1

2) (A10)

Fig. A5. Power spectra S(ω) of a series Tpj of the periods in a circuit experiment.

Fig. A4. Sample of a series Tpj of the periods in the ring oscillator of three neurons (N = 3) in a circuit experiment.

252627282930313233

0 20 40 60 80 100j

Tpj (

ms)

0.0

0.5

1.0

1.5

2.0

0.0 0.1 0.2 0.3 0.4 0.5ω / 2π

S(ω

; Tp j

) experiment EXP (Eq. (A9)) AR1 (Eq. (A10))

21

4. Conclusion Effects of spatiotemporal noise on the oscillations in the ring networks of neurons were studied.

1. Duration of the transient oscillations in the networks of even numbers of neurons (N = 2M)

→ Mean duration of the transient oscillations increases owing to noise of intermediate strength. Noise-sustained oscillations

2. Variations and correlations in a series of the half periods of the stable oscillations in the networks of odd numbers of neurons (N = 2M + 1) (Ring oscillator)

→ Noise causes positive correlations in a series of the half periods. The power spectra are of exponential type. Similar to the interspike intervals of a spike propagating in a nerve fiber