P. Read Montague

Baylor College of Medicine, Houston, TX

Why Computers Don't Care

The Origins of Meaning in Efficient Computation

Part 2. Energy, computation, and meaning

Part 3. Pursuing goals with valuation machinery

Part 1. Structural themes in the nervous system

Why can ideas veto biological instincts for survival?

Theme I: Put data near the processing capacity -maps and integrative regions

Vision

Touch

Hearing

IntegrationIntegration

Theme II: Miniaturization, self-repair, re-configuration

1 mm

1 mm

1 mm

~1-2 billion connections

~1 mile axons (output from neuron)

~2 miles dendrites (branches of neuron)

100 μm20 nm

-7

-6

-5

-4

-3

-2

-1

0

Log time (sec)

Log size (m)

single cell recording

patch clamp

Many physical probes of living neural tissue

Where does fMRI fit?

Optical

dyes

light microscope

brain

synapse

neuron

networks

map

radioactivity

invasive (needles!)

PET scan

fMRIMEG & EEG

no radioactivity

non-invasive

Important aspects of cognition

across these time scales

0 1 2 3 4 5 6 7-1-2-3

functional Magnetic Resonance Imaging (fMRI)

Four antecedents to modern fMRI

Physics and Chemistry

magnetic properties of

atoms & molecules

Physiology & Anatomy

Control of cerebral blood flow

Neurobiology & Models

Psychology models of learningComputing

Cost, Speed, Algorithms

Modern fMRI

MRI

Exploit density differences of water in different tissues

Produce anatomical snapshots

MRI

Use blood flow sensitive measurement

Produce movie of microscopic blood flow

changes

fMRI

Despite the slowness, noisiness, and imprecision of neural computations…

The neural myth

…the brain - usually through the ‘miracle’ of parallelism -performs inexplicable feats of perception and action.

Alan Turing

121

0 01 0$

*0

Turing Machine

Hardware (physical interactions)

Patterns of information processing

Computation Computational Theory of Mind (CTOM)

Turing – mind is not equivalent to our brain, but is equivalent to information processing supported by our brain.

‘Mind-like’ stuff from ‘Stuff-like’ stuff

Blurs distinction between device and algorithm

Meaning is missing

Computational Theory of Mind (CTOM)

Symbol list 1 Symbol list 2

When does meaning get added back?

What is the value of available choices?

Two questions face every mobile creature

What does each choice cost?

Recovering meaning

Organisms must assign meaning to the processes that comprise them.

They run on batteries and the future is uncertain.

Recharge or die

run tumble

A principle for value-based computation

Organisms distribute their finite resources to the computations that sustain them.

Their willingness to do so should scale with the expected long-term returns from the computation.

How is one computation assigned more or less meaning than another?

Where’s the evidence for efficientcomputation in the brain?

32 nodes8 dedicated chess processors per node (total 256)

1.4 tons (lots of AC)200,000,000 positions per second

DEEP BLUE

Kasparov stayed warm to the touch

DDeep Blue was wildly inefficient

Kasparov’s brain is freakishly efficient

~100 watts brain uses ~20 watts

What about evidence fromdetailed neural properties?

Drain batteries slowly

Have goals

Save Space

Save bandwidth

neurons compute slowly

Set and pursue goals

Small parts, sparse connectivity

Stay off the lines, noisy codes

Principles of efficient computation

MYTH is now debunked

What happened?

Why are modern computers so wasteful?

Why is meaning missing from modern model of computation?

Bletchley Park Mansion

Enigma

World War II happened

Computer programs don’t care (enough)

Flexible goals should guide their behavior (multi-level)

Value-based Computation

client client

Hyperscan server and database

hyperscan fMRI (h-fMRI)

Houston (BCM)Houston (BCM)California (Caltech)

New Jersey (Princeton)Georgia (Emory)

Germany (Uni-Ulm)Hong Kong (HKUST)

Houston (BCM)

information: www.hnl.bcm.tmc.edu/hyperscan

software download: www.hnl.bcm.tmc.edu/nemo

Sustain goal

Select goal

Pursue goal

Goal-directed choice

Actual experience Counterfactual experience‘what could have been’

guidancesignals

Sharks don’t go on hunger strikes

But humans do

Ideas gain behavioral potency of primary rewards like food and water

How?

Goal pursuit requires guidance signals dopamine

Computationalmodels

fMRIexperiments

Valuation

Dopamine neurons

Midbrain dopamine neurons

burst

pause

R timenaive

R timeAfter learning

timeAfter learning(catch trial)

Pause, burst, and ‘no change’ responses representreward prediction errors

Red Light becomes a proxy for the value of the juice

Midbrain dopamine neurons

Constant emission of reward prediction errors

ERROR SIGNAL = current reward + γ next prediction - current prediction

Neural evidence suggests that system can be re-deployed for abstractly defined rewards ideas

“I want to solve Fermat’s last theorem”

Induce reward prediction error in a passive conditioning task measure brain response

6sJ train

6s 4sJ test

y = 3

McClure et al., Neuron, 2003

Juice not expected but delivered

Juice expected but not delivered

0.4

0.8

0.0

-0.4

% s

igna

l cha

nge

2 6 10 12 sec

+-

Hemodynamic response in ventral putamen

6s 4sJ

“BETTER THAN EXPECTED”

“WORSE THAN EXPECTED”

What about something more abstract like trust?

TRUST

I love you

Let’s do lunch

BA

A BAB

Trust

Most basic form is about model-building

Requires risk – no trust needed without risk

Cooperation

How can we study trust in a social exchange?

Use simple economic game between two players

Synchronously image both interacting brains

X 3

Investor Trustee

$20

A dynamic version of the Trust game (10 rounds)

Simplifying and quantifying Trust

Trust is the amount of money a sender sends to a receiver without external enforcement.

Agent 2Agent 1I

R

What is Fair?

i

i

i

Split the profit

Split ‘loaf’Split the total

10 + i each

i

Investor Trustee

20-i

collective ownership? common goods?

client client

Hyperscan server and database

Hyperscanning: Scan all brains during the interaction

Houston (BCM)Houston (BCM)California (Caltech)

New Jersey (Princeton)Georgia (Emory)

Germany (Uni-Ulm)Hong Kong (HKUST)

Houston (BCM)

information: www.hnl.bcm.tmc.edu/hyperscan

software download: www.hnl.bcm.tmc.edu/nemo

investperiod

8 s 8 s10 s 10 s 10 s8 s8 s4 s4 s

cue to invest

repaymentrevealed toboth brains

Gave

13

Kept

29

investmentrevealed toboth brains

Kept Gave

6 14

repayperiod

cue to repay

delayperiod

delayperiod

delayperiod

delayperiod

investment phase repayment phase

Totals

2919

totalsrevealed toboth brains

inter-rounddelay period

investperiod

8 s 8 s10 s 10 s 10 s8 s8 s4 s4 s

cue to invest

repaymentrevealed toboth brains

Gave

13

Kept

29

Gave

13

Kept

29

investmentrevealed toboth brains

Kept Gave

6 14

Kept Gave

6 14

repayperiod

cue to repay

delayperiod

delayperiod

delayperiod

delayperiod

investment phase repayment phase

Totals

2919

Totals

2919

totalsrevealed toboth brains

inter-rounddelay period

Structure of a round

decisionsubmittedby investor

decisionsubmittedby trustee

investor trustee

decisionrevealed toboth brains

decisionrevealed toboth brains

investor trustee

time

What we generally see on this hyperscanned trust exchange

What is the behavioral signal that most strongly influences changes in trust (money sent) ?

Agent 2Agent 1I

R

Reciprocity = TIT-FOR-TAT

Activity related specifically to benevolent reciprocity

Caudate nucleus – action choice, reward processing, …

Trustee brain

*

time (sec)-8 0 10

-0.2

0

0.2

Trustee ‘intention to increase trust’shifts with reputation building

Reputation develops

Trustee will increasetrust on next move

Trustee will decreasetrust on next move

submit reveal *

increases or decreasesin future trust by trustee

-0.2

0

0.2

time (sec)

Signal is nowanticipating the

outcome

Temporal shift resembles value transfer in reward learning experiments

burst

pause

R timenaive

R timeAfter learning

timeAfter learning(catch trial)

rounds

.1

.2

.3

.4

3 4 5 6 7 8 9 10

fractionof

highlyaccurateguesses

Investorevents

Trusteeevents

How do we know a reputation is forming?

8 s 10 s4 s. . .

investperiod

cue to invest

investmentrevealed toboth brains

delayperiod

guessperiod

cue to guess

Investment phase

synchronized onsubmission time of both

partners

Summary

Finite resources force distribution decisions on life formsBiological computations always had to assign meaning

Efficient computational systems must have goals

Goal pursuing machinery identified and modeled

Ideas gain behavioral potency of food and sex

Re-deployment of reward harvesting machinery