The Human Visual The Human Visual SystemSystemThe Human Visual The Human Visual SystemSystem

Vonikakis Vasilios, Antonios Gasteratos Vonikakis Vasilios, Antonios Gasteratos

Democritus University of Thrace Democritus University of Thrace 2006 2006

Democritus University of Thrace Democritus University of Thrace 2006 2006

The Human Visual SystemThe Human Visual System

Biological background

RetinaRetina Visual CortexV1, V2…

Visual CortexV1, V2…

Optic nerveOptic nerve

light(ganglion cells)

Ανθρώπινο Οπτικό Σύστημα

The eyeThe eye

3 3 kinds of coneskinds of cones ( (long, medium, shortlong, medium, short) – ) – color vision color vision (only in bright light – photopic vision)(only in bright light – photopic vision)

RodsRods – – achromatic visionachromatic vision ( (in dim light – scotopic visionin dim light – scotopic vision))

The photoreceptorsThe photoreceptors

Only Only oneone layer of photoreceptors layer of photoreceptors

Varying distribution of Varying distribution of photoreceptors photoreceptors ((Only L and M Only L and M cones in the fovea, only rods in the cones in the fovea, only rods in the peripheryperiphery))

Different ratios of photoreceptors Different ratios of photoreceptors between individuals (generally between individuals (generally L>M>S)L>M>S)

Hexagonal distribution of Hexagonal distribution of photoreceptorsphotoreceptors

No refresh rate – parallel No refresh rate – parallel transmission of visual information transmission of visual information to the brainto the brain

Differences from a ccdDifferences from a ccd

What retina seesWhat retina sees

DayDay NightNight

OutputOutput

photoreceptorsphotoreceptors

Ganglion cellGanglion cell

Basic retinal circuitBasic retinal circuit

Ganglion cells are the Ganglion cells are the onlyonly output of from the retinaoutput of from the retina

Digital output with an FM Digital output with an FM modulation (spikes)modulation (spikes)

The number of The number of photoreceptors that a photoreceptors that a ganglion cell “sees” and the ganglion cell “sees” and the kind of the connectionkind of the connection

Ganglion cells have Ganglion cells have antagonistic center-surround antagonistic center-surround receptive fieldreceptive field

++ ----

Receptive fieldReceptive field

++ ----

Center-surround antagonismCenter-surround antagonism

++ ----

++ ----

++ ----

++ ----

nothingnothing

lightlightNo lightNo light lightlightNo lightNo light

inhibitioninhibition

excitationexcitation

Center-surround responsesCenter-surround responses

Ganglion cells are edge detectors – they Ganglion cells are edge detectors – they respond respond onlyonly to changes and to changes and notnot to to uniform areasuniform areas

By stimulating only the cells that detect By stimulating only the cells that detect differences, the HVS minimizes the differences, the HVS minimizes the number of active neuronsnumber of active neurons

Example: Instead of transmitting a Example: Instead of transmitting a sequence of long numbers e.g.sequence of long numbers e.g. 2003453, 2003453, 2003453, 2003455, 2003451 2003453, 2003455, 2003451 it transmits it transmits only their differencesonly their differences: 0, 0, +2, -2: 0, 0, +2, -2

Center-surround : factsCenter-surround : facts

White paper in dim light reflects less light (is darker) White paper in dim light reflects less light (is darker) than the black letters in bright lightthan the black letters in bright light

The absolute value of reflected light is The absolute value of reflected light is notnot important important

By responding only to differences, ganglion cells By responding only to differences, ganglion cells prevent the white paper from being perceived as prevent the white paper from being perceived as blackblack

Center-surround : advantageCenter-surround : advantage

Dim lightDim light Bright lightBright light

Kinds of Ganglion cellsKinds of Ganglion cells

Biological background

Photoreceptor mosaic

Bcenter - (R+G)surround

Rcenter - Gsurround

Gcenter - Rsurround

(R+G+B)center - (R+G+B)surround

Red-Green oponency

Blue-Yellow oponency

Achromatic opponency

Midget ganglion cellsMidget ganglion cells

Biological background

Midget ganglion cellsMidget ganglion cellsMidget ganglion multiplex 2 signalsMidget ganglion multiplex 2 signals

1.1. Red-Green chromatic opponencyRed-Green chromatic opponency

2.2. Achromatic high acuity (1 cone = 1 center of the Achromatic high acuity (1 cone = 1 center of the receptive field)receptive field)

Parasol ganglion cellsParasol ganglion cellsParasol ganglion cells are:Parasol ganglion cells are:

1.1. AchromaticAchromatic

2.2. Have 3 times greater receptive filedHave 3 times greater receptive filed

3.3. Respond better to movementRespond better to movement

22

Bistratified ganglion cellsBistratified ganglion cells

Bistratified ganglion cells:Bistratified ganglion cells:

1.1. Carry the Blue – Yellow opponencyCarry the Blue – Yellow opponency

2.2. Have 3 times greater receptive filedHave 3 times greater receptive filed

22

Retinal outputRetinal output

At least 8 independent and parallel mosaics of At least 8 independent and parallel mosaics of ganglion cells outputs scan the photoreceptors and ganglion cells outputs scan the photoreceptors and transmit different information to the visual cortextransmit different information to the visual cortex

The primary visual cortex V1The primary visual cortex V1

The visual cortex analyses the retinal output in 3 different and independent maps:

1. color 2.motion-depth3.orientation of edges

The primary visual cortex V1The primary visual cortex V1

The visual cortex analyses the retinal output in 3 different and independent maps:

1. color 2.motion-depth3.orientation of edges

Demultiplexing RG in cortexDemultiplexing RG in cortex

In every position of the In every position of the visual field 4 visual field 4 different midget different midget cells (from the 4 cells (from the 4 mosaics) are mosaics) are connected in connected in couplescouples

Chromatic opponency is Chromatic opponency is canceled (same colors to canceled (same colors to center and surround). Now center and surround). Now only sensitive only to only sensitive only to luminance incrementsluminance increments

Chromatic opponency is canceled Chromatic opponency is canceled (same colors to center and (same colors to center and surround). Now only sensitive surround). Now only sensitive only to only to luminance decrementsluminance decrements

Center-surround Center-surround antagonism is antagonism is canceled (they canceled (they are the same)are the same)

Center-surround Center-surround antagonism is antagonism is canceledcanceled

Cell types Cell types

For every position of the visual field there are 8 different cells that detect chromatic and achromatic signals in 2 different scales

Cell outputs Cell outputs original Red-Green opponency Blue-Yellow opponency

Achromatic (dark-light)

Double opponent cells Double opponent cells Are formed by combinations of simple center-surround cells

Are excited only by chromatic differences of a very specific color (color edges)

ResponsesResponses Double opponent cells respond only to very specific changes

between certain hues (color edges)

original

Elongated receptive fields (formed Elongated receptive fields (formed by combinations of center-by combinations of center-surround receptive fields)surround receptive fields)

~~12 12 different orientationsdifferent orientations ( (everyevery 1515°°))

Detect edges of particular Detect edges of particular orientations orientations onlyonly in a very specific in a very specific positionposition

Simple Orientation cellsSimple Orientation cells

Formed by combinations of Formed by combinations of simple orientation cellssimple orientation cells

Detect edges of particular Detect edges of particular orientation orientation anywhereanywhere in their in their receptive fieldreceptive field

Complex Orientation cellsComplex Orientation cells

Orientation cellsOrientation cells At every position of the visual field there are all

possible orientations of an edge

Every edge excites a particular orientation cell in a particular position of the visual cortex

HypercolumnsHypercolumns

For every position of the visual field, all cells are grouped into hyper columns

Every hypercolumn is a complete and independent feature detector for a very small part of the visual field

Every hypercolumn contains color cells, orientation cells, disparity cells, motion cells

HypercolumnsHypercolumns Competition exists between cells of the same

hypercolumn and between hypercolumns

Association field

Biological background

Orientation cells prefer to be connected with others that favor the smooth continuity of contours

Connection of orientation cellsConnection of orientation cells

Smooth combinations emerge from the group of orientation cells

This is the first step for contour perception

Salient contoursSalient contours

More complex cells code certain combinations of More complex cells code certain combinations of salient orientation cellssalient orientation cells

Contour integrationContour integration

All the features (contours, colors, texture, depth) are All the features (contours, colors, texture, depth) are being bind in one perceptionbeing bind in one perception

Binding is described by the Binding is described by the GestaltGestalt rules e.g. rules e.g. common common fate rulefate rule, , proximity ruleproximity rule, , similarity rulesimilarity rule etc.etc.

Feature bindingFeature binding

There is a tendency to spatially diffuse strong signals There is a tendency to spatially diffuse strong signals over the weak onesover the weak ones

This way, regions that do not have a strong feature This way, regions that do not have a strong feature ‘get’ one from a nearby region that has a strong one‘get’ one from a nearby region that has a strong one

Edges act like barriers that stop the diffusions of the Edges act like barriers that stop the diffusions of the signalssignals

There is filling-in forThere is filling-in for::• TextureTexture• ColorColor

• DisparityDisparity

Filling-in the featuresFilling-in the features

Filling-in illusionsFilling-in illusions

shapeshape colorcolor texturetexture motionmotion

Object spaceObject space

bindingbindingbindingbinding

Binding to one perceptBinding to one percept

What & Where streamWhat & Where stream

Cell(s) for every objectCell(s) for every object

Finally there is one cell (or one population of Finally there is one cell (or one population of cells) that respond cells) that respond onlyonly to a very specific to a very specific object object

Every perception of an object (either vision Every perception of an object (either vision triggered or mind triggered) activates these triggered or mind triggered) activates these cellscells

This ‘databank’ of cells is located at the This ‘databank’ of cells is located at the inferior temporal cortexinferior temporal cortex

Inferior temporal cortexInferior temporal cortexInferior temporal cortex has columnar organizationInferior temporal cortex has columnar organization

Many aspects of an object are stored in neighboring Many aspects of an object are stored in neighboring columnscolumns

Similar objects are stored in neighboring rowsSimilar objects are stored in neighboring rows

Inferior temporal cortexInferior temporal cortexEvery object is stored in the object space in many Every object is stored in the object space in many rotated versionsrotated versions

……but we are trained only to the versions we usually but we are trained only to the versions we usually see…see…

Attention modelsAttention models

Attention modelsAttention models

Thank you! Thank you!