Analog recurrent neural netwAnalog recurrent neural network simulation, ork simulation, ΘΘ(log(log22n) unorn) unordered search with an opticalldered search with an optically-inspired model of computay-inspired model of computa

tiontion

IndexIndex

Continuous Space Machine StructureContinuous Space Machine Structure Analog Recurrent Neural Network Analog Recurrent Neural Network

Simulation and Complexity ResultSimulation and Complexity Result Unordered Search AlgorithmUnordered Search Algorithm)(log2 n

The Continuous Space MachineThe Continuous Space Machine(CSM)(CSM)

Definition:Definition:

),,,,( OPILDM NNDnmD ),,( : grid dimensions

)),(),,(),,(( bbaassL : address of sta, a, and b

)},(,),,{( 11 kk iiiiI : addresses of the k input images

)},,,,,,*,,,({

)},,(,),,,{( 111

hltbrldstvh

ppppP rrr

: the r programming symbols and their addresses

)},(,),,{( 11 ll ooooO : addresses of l output images

Instructions of CSMInstructions of CSM

hh and and v : v : hh gives the 1-D Fourier transformation in gives the 1-D Fourier transformation inthe x-direction, and the x-direction, and vv gives the 1-D Fourie gives the 1-D Fourie

rrtransformation in the y-direction.transformation in the y-direction.

10),,()),((10),,()),((

),(),(),(),(

)),(()),(()),(()),((

22

xFxFvyFyFh

eyxfxFeyxfyF

xFvyxfvyFhyxfh

ii

II

Instructions of CSM (II)Instructions of CSM (II)

* :* :** gives the complex conjucate of its gives the complex conjucate of itsargument image.argument image. where fwhere f** is the complex conju is the complex conju

cate of f.cate of f.),()),(( * yxfyxf

II

Instructions of CSM (III)Instructions of CSM (III)

∙∙and +: and +: ∙∙gives the pointwise complex product of igives the pointwise complex product of i

tststwo argument images, + gives the pointwitwo argument images, + gives the pointwi

sesecomplex sum of its two argument images. complex sum of its two argument images.

),(),()),(),,((

),(),()),(),,((

yxgyxfyxgyxf

yxgyxfyxgyxf

III

Instructions of CSM (IV)Instructions of CSM (IV)

ρρ::ρρ performs amplitude thresholding on its performs amplitude thresholding on itsfirst image argument using its other twofirst image argument using its other tworeal-valued image arguments as lower anreal-valued image arguments as lower an

ddupper amplitude thresholds, respectively. upper amplitude thresholds, respectively.

IIII

),(),(),,(

),(),(),(,),(

),(),(),,(

)),(),,(),,((

yxzyxfifyxz

yxzyxfyxzifyxf

yxzyxfifyxz

yxzyxzyxf

uu

ul

ll

ul

Instructions of CSM (V)Instructions of CSM (V)

ldld and and ststld ld parameters pparameters p11 to p to p44 to image at well-knownto image at well-knownaddress a.address a.st st copies the image at well-known address acopies the image at well-known address ato a ‘rectangle’ of images specified by the to a ‘rectangle’ of images specified by the ststparameters pparameters p11 to p to p4.4.

Instructions of CSM (VI)Instructions of CSM (VI)

brbr and and hlthltbrbr gives the unconditional jump to the gives the unconditional jump to theaddress that the parameter indicates.address that the parameter indicates.hlt hlt gives the program termination.gives the program termination.

Instructions of CSM (Review)Instructions of CSM (Review)

The relation betweenThe relation betweenimages and dataimages and data

Complex-valued imageComplex-valued image A complex-valued image is a functionA complex-valued image is a function

, where [0, 1] is the real unit interval., where [0, 1] is the real unit interval.

Zero ImageZero ImageAn image that has value 0 An image that has value 0

everywhere represents 0.everywhere represents 0.

Cf ]1,0[]1,0[:

The relation betweenThe relation betweenimages and data (II)images and data (II)

Binary symbol imageBinary symbol imageThe symbol The symbol ψψ is represented by is represented bythe binary symbol image the binary symbol image ffψψ

Real number imageReal number imageThe real number r The real number r R is represented by the real number R is represented by the real numberimage fimage frr

.,0

.1,5.0,1),(

otherwise

yxifyxf

.,0

.5.0,),(

otherwise

yxifryxf r

Two kinds of Binary wordsTwo kinds of Binary words

Stack imagesStack imagesldld and and stst instead of push and pop. instead of push and pop. List imagesList imagesLoad all images at once.Load all images at once.

Matrix image for ARNN Matrix image for ARNN simulationsimulation

RRC matrix imageC matrix imageThe RThe RC matrix A with real-valued components aC matrix A with real-valued components aijij, is, isrepresented by the Rrepresented by the RC matrix image C matrix image ffAA

Riif

Riifl

Cjif

Cjifkwhere

otherwise

ly

kxifayxf likjij

A ,0

,1,

,0

,1,

,02

211,

2

211,),(

Complexity measureComplexity measure TimeTimeThe number of instructions executed in the program.The number of instructions executed in the program.

SpaceSpaceThe total space needed to execute the program.The total space needed to execute the program.

ResolutionResolutionThe maximum resolution of the grid images in theThe maximum resolution of the grid images in the

Computation sequencesComputation sequences

RangeRangeThe maximum amplitude precision needed.The maximum amplitude precision needed.

ARNNARNN

ARNNs are finite size feedback firstARNNs are finite size feedback firstorder neural networks wirh realorder neural networks wirh realweights.weights.The state of each neuron The state of each neuron xxii at time at timet + 1t + 1 is given by an update is given by an update equation of the form:equation of the form:We can take We can take pp neurons of x neurons of xii for output.for output.

nictubtxatx i

m

jjij

n

jjiji ,,1),)()(()1(

11

ARNN (II)ARNN (II)

The CSM model can simulate the The CSM model can simulate the ARNNARNN

The pseudo code is as belowThe pseudo code is as below

ARNN (III)ARNN (III)

ComplexityComplexityIf ARNN being simulated is defined for time t = 1, 2, 3, If ARNN being simulated is defined for time t = 1, 2, 3,

……

has M input, N neurons, and k is the number of stackedhas M input, N neurons, and k is the number of stacked

image elements used to encode the active input to theimage elements used to encode the active input to the

simulator, the four complexity aresimulator, the four complexity are

Time = O((N + M + 1)t + 1), Space = O(1),Time = O((N + M + 1)t + 1), Space = O(1),

Resolution = Max(2Resolution = Max(2k+M-1k+M-1, 2, 22N-22N-2, 2, 2N+M-2N+M-2, 2, 2t+N-1t+N-1),),

Range = Infinity. (Real value needs infinite bits.)Range = Infinity. (Real value needs infinite bits.)

ARNN ConclusionARNN Conclusion

Because ARNN can be simulated by Because ARNN can be simulated by CSM,CSM,

the computation power of CSM is at the computation power of CSM is at leastleast

as strong as TM.as strong as TM.

Unordered SearchUnordered Search(Needle in the haystack (Needle in the haystack

problem)problem)L = {w: w L = {w: w 0*10*}, 0*10*}, ωω L be written as L be written asωω = = ωω00ωω11…… ωωn-1.n-1.

Input: Input: ωω Output: Output: Binary representation of Binary representation of ii, where , where ωωii=1.=1.

Solve NIH in other modelSolve NIH in other model

In the classic model, this may be solved iIn the classic model, this may be solved inn

O(n) time naO(n) time naïïvely, and it seems that thevely, and it seems that thenaive method might have the best naive method might have the best performance in this model.performance in this model. In the quantum computer, this may beIn the quantum computer, this may besolved in solved in ΩΩ( ) with Grover’s work.( ) with Grover’s work.

n

NIH in the CSM modelNIH in the CSM model

Thinking…Thinking…Use a binary list image to represent Use a binary list image to represent ωω, and a binary stack, and a binary stackimage to represent n with logimage to represent n with log22n bits. Because the n bits. Because the ωω hashasonly one non-zero point, we can use some convenientonly one non-zero point, we can use some convenientinstructions in CSM to solve this problem in shorter timeinstructions in CSM to solve this problem in shorter time

……

Pseudo Code ofPseudo Code ofθθ(log(log22n) unordered searchn) unordered search