Visual perception basics Image aquisition...

41
P. Strumillo Image aquisition system Visual perception basics

Transcript of Visual perception basics Image aquisition...

Page 1: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

P. Strumiłło

Image aquisition system

Visual perception basics

Page 2: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Light perception by humans

Humans perceive approx. 90% of information about the

environment by means of visual system.

Efficiency of the human visual system is characterised by a

number of features:

• the ability to resolve image details (θ=1’=1°/60=pi/10800);

• the ability to discriminate between brightness levels

(contrast sensitivity);

• colour perception;

• brightness adaptation;

Page 3: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

retina125×106 receptors

Human

visual system

pupil

Optic nerve

cornea iris

Optic chaism

Visual cortex

lens

Lateral

geniculate nucleus

Page 4: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Structure of the

human eye

lens

retina125××××106

receptors

Nerve PWN

Fovea

Blindspot

Visual axis

iris

Page 5: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Distribution of rods and cones in the retina

PWN

Cones Cones

RodsRods

Blind spot

Angle

No of cones/rods per mm2 50x50 um50x50 um

Webvision

Page 6: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Eye convergence angleEye convergence angle Disparity in binocular visionDisparity in binocular vision

~10 m

Binocular vision

Perception of depth (distance)

17 mm

Page 7: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Denis Meredith

Role of colours Role of colours

in depth perceptionin depth perceptionA A B B

Page 8: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Electromagnetic spectrum

10181024 1016 1014 1012 1010 108 106 104 1021022 1020

frequency [Hz]

gamma rays radio waves

X rays microwaves

Infrared wavesultravioletVisible Visible

spectrumspectrum

400 500 600 700 [nm]

Page 9: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

infrared

Spectral sensitivity characteristic of the human eye

ulraviolet

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 µm00

0.2

0.4

0.6

0.8

1rods

cones

Page 10: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Visual perception

Subjective brightness sensation assumes a logarithmic characteristic.

Human eye can percieve brightness in the range of 1010.

Log [mL]-6 -4 -2 0 2 4

Eye adaptation range

Local adaptation range

Glare limitGlare limit

Page 11: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Contrast sensitivity (Weber fraction)

The ratio is termed the Weber fraction.

It reflects contrast sensitivity characteristc of the human eye.

2%

I

I∆

I

3⋅10-1 3⋅101 3⋅103

I

I+∆ I

[cd/m2]

I

I∆

Page 12: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

The eye achieves maximum sensitivity for:

I+∆I≈ I0

I

I∆

I

3 3⋅101 3⋅103

0I

[cd/m2]

I+∆ I

I

3⋅102

Io=3 Io=30 Io=300 Io=3000

Contrast sensitivity (Weber fraction)

Page 13: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Image coded using 16 gray levels

MIT4 bits/pixel

Page 14: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Mach bands

Subjective brightness

Brightness intensity

Page 15: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Visual illusions

Page 16: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Image aquisition system

Visual perception basics

Page 17: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Visual path of the image processing system

Visual path – a set optical and electronic elements converting

radiant energy into an electrical signal and imaging it using

display devices.

Image formationOpto-electrical

conversion Visualisation

3D

Imagingsensor

2D

Page 18: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Pros and cons:

• small hole → little light goes in

• large hole → image blurring

Pinhole camera (camera obscura)

Pin hole

Page 19: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Cameras today

CCD sensorCCD sensor

Pros and cons:

• sharp, high-contrast image

• geometric distortions

Pointgrey

Page 20: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Image formation model

x

y

Image plane of the imaging sensor

3D

Image formation model

(x,y) f(x,y)

(α,β)

2D

Page 21: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Image formation model

Image – a 2-D light intensity function

f(x,y)>=0 reflecting light energy

distribution

),(),(),( yxryxiyxf =

∞<< ),(0 yxi

1),(0 << yxr

- illumination (x,y)

reflectance coefficient at (x,y)

y

x

(x,y) f(x,y)

illumination: sunny day ~ 5000 cd/m2,

cloudy day ~ 1000 cd/m2, full moon ~ 0.001 cd/m2,

Reflectance coeff.: black velvet - 0.01, white wall - 0.8, snow - 0.93.

Page 22: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

For a linear process of energy acummulation in the image

sensor plane:

( ) ( ) ( ) βαβαβα dd,,y,xh,fy,xf ∫ ∫∞

∞−

=

h(.) – is the impulse response of the system; in optical

systems it is termed the point spread function of the system

Image formation model

Page 23: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

If the point spread function is shift invariant, then the

image formation model is given by a convolution integral:

( ) ( ) ( ) βαβαβα ddy,xh,fy,xf −−= ∫ ∫∞

∞−

+−=2

22

2

yxexp)y,x(h

σ

Image formation model

Page 24: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Sampling of 1-D signals

x

f(x)

ω

F(ω)

-Ω Ω

x

s(x)

∆xω

S(ω)

-1/∆x 1/∆x

s(x)f(x)

ω

S(ω)*F(ω)

-Ω Ω

. . .. . .

Page 25: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Assume the source image (analog image) features a

limited Fourier bandwidth

ωmaxx

ωmaxy

−ωmaxy

−ωmaxx

ωy

ωx

( )yx ,F ωω

Sampling of 2-D signals

Page 26: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

( ) ( )∑ ∑−

=

=∆−∆−=

1

0

1

0,,

M

i

N

kykyxixyxS δ

Image sampling function:

and a sampled image:

( ) ( ) ( )

( ) ( )∑ ∑−

=

=∆−∆−∆∆=

==1

0

1

0,,

,,,M

i

N

k

s

ykyxixykxif

yxSyxfyxf

δ

∆x∆y

Sampling of 2-D signals

Page 27: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

( ) ( )∑ ∑−

=

=∆−∆−

∆∆=

1

0

1

0,

1,

M

i

N

kyyxxyxs kiF

yxF ωωωωωω

Fourier spectrum of the sampled image:

where:

yx yx ∆=∆

∆=∆ 1

,1 ωω

∆ωx

∆ωy

Sampling of 2-D signals

Page 28: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

∆ωx

∆ωy

ωmaxy

ωmaxx image bandwidth

xx

∆=∆<Ω

2

1

2maxω

Sampling of 2-D signals

Page 29: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Aliasing distortion - example

100 dpi(dots per inch)

500 dpiScanned images:

Page 30: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Image acquisition

Image acquisition is the process of converting light energy

radiating from image scene points into an electrical signal

(suitable for storing or transmission).

Image acquisition devices:

• CCD camera

• Video camera

• Scanner

• Digitizer

Page 31: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

There are two basic schems of converting optical images

into electrical signals:

• without accumulation of photo-charges (eg. optical

scanner),

• with accumulation of photo-charges (np. vidicon,

CCD array)

Image acquisition

Page 32: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Imaging sensor (no photo-charges)

+

s(x,y)

photodiode

focusing screen

R

scanning direction

Page 33: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

CCD array (accumulation of photo-charges)

Image formation is based on the internal photo-electric

phenomenon

UF<0Φ

n

Capacitor cell

electrical potential well

~10µm

insulator

Page 34: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

The Bayer matrix

Raw CCD Format

Calculate RGB image by interpolating colour components from the Bayer matrix

N

M

N

M

Page 35: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Pixim – Digital Pixel System (DPS)

A/D converter for each pixel (no charge couplings)

Single A/D converter

Page 36: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

CMOS image sensors

Pros:

• cheap technology (used for fabricating memory and CPU modules),

• low power consumption (100 times!)

• random access to pixel regions (block image processing)

• no „charge leaking” typical for CCD technology

• on-chip analog-to-digital conversion and signal processing

OmniVision

Cons:

• more susceptibel to noise

than CCD

• lower light sensitivity due to

many transistors used

for single pixel

Page 37: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Monochrome TV standards

European CCIR standard: (625 (575) lines, line display

time 64us, 50 half-images per sec., 1Vpp, 75Ω, signal

American RS170 standard: (525 (484) lines, line display

time 63,5 us, 60 half-images per sec.,1.4 Vpp, 75Ω signal

American RS-343 standard”: (875 lines, 60 half-images,

dedicated to CCTV, scientific applications,…)

Page 38: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

TV CCIR standard

Composite video signal

peak white level

blanking level

sync level

odd lines

even lines625/575 lines

video signal

horizontal sync pulse

4

64 µs

0 V (DC)

0.7 V

-0.3 V

3

52 µs

Page 39: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Resolution:RS-170A: 580 horizontal TVL, 350 vertical TVL; CCIR: 560 horizontal TVL, 450 vertical TVL

Synchronization:Crystal/H&V/Asynchronous, standard

Shutter: 1/60 to 1/10,000 AGC: 20 dB Integration: 2 - 16 Fields Sensitivity:

Full video, No AGC: 0.65 lux; 80% video, AGC on: 0.04 lux; 30% video, AGC on: 0.008 lux

S/N Ratio (Gamma 1, gain 0 dB): 55 dB

Specification Highlights

Imager: 1/2" interline transfer CCD

Picture Elements: RS-170A: 768 (H) x 494 (V);CCIR: 752 (H) x 582 (V)

Pixel Cell Size: RS-170A: 8.4 µm (H) x 9.8 µm (V); CCIR: 8.6 µm (H) x 8.3 µm (V)

COHU® CCD camera

Page 40: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

CCD image sensors characteristics

small size,

robust to mechanical vibrations (70 G),

no geometrical distortions,

low supply voltage (12 V, 1.4W),

SNR ~70 dB,

linear (gamma coefficient),

no intra-frame photo-charge accumulation,

high resolution,

reliable

cheap

Page 41: Visual perception basics Image aquisition systemmstrzel.eletel.p.lodz.pl/mstrzel/pattern_rec/acquisition.pdf · Visual perception basics. Light perception by humans Humans perceive

Image frame grabber

Matrox Cronos Plus

Video capture board for PCI captures from NTSC, PAL, RS-170 and CCIR video sources, connect up to 4 CVBS or 1 Y/C trigger input, 7 TTL auxiliary I/Os, 32-bit/33MHz PCI-bus master Matrox ®

Software is sold separately, includes e.g., Matrox ®Imaging Library for Microsoft® Windows®