Image aquisition system Visual perception basics

41
P. Strumillo Image aquisition system Visual perception basics

Transcript of Image aquisition system Visual perception basics

Page 1: Image aquisition system Visual perception basics

P. Strumiłło

Image aquisition system

Visual perception basics

Page 2: Image aquisition system Visual perception basics

Light perception by humans

Humans perceive approx. 90% of information about the

environment by means of visual system.

Efficiency of the human visual system is characterised by a

number of features:

• the ability to resolve image details (θ=1’=1°/60=pi/10800);

• the ability to discriminate between brightness levels

(contrast sensitivity);

• colour perception;

• brightness adaptation;

Page 3: Image aquisition system Visual perception basics

retina125×106 receptors

Human

visual system

pupil

Optic nerve

cornea iris

Optic chaism

Visual cortex

lens

Lateral

geniculate nucleus

Page 4: Image aquisition system Visual perception basics

Structure of the

human eye

lens

retina125××××106

receptors

Nerve PWN

Fovea

Blindspot

Visual axis

iris

Page 5: Image aquisition system Visual perception basics

Distribution of rods and cones in the retina

PWN

Cones Cones

RodsRods

Blind spot

Angle

No of cones/rods per mm2 50x50 um50x50 um

Webvision

Page 6: Image aquisition system Visual perception basics

Eye convergence angleEye convergence angle Disparity in binocular visionDisparity in binocular vision

~10 m

Binocular vision

Perception of depth (distance)

17 mm

Page 7: Image aquisition system Visual perception basics

Denis Meredith

Role of colours Role of colours

in depth perceptionin depth perceptionA A B B

Page 8: Image aquisition system Visual perception basics

Electromagnetic spectrum

10181024 1016 1014 1012 1010 108 106 104 1021022 1020

frequency [Hz]

gamma rays radio waves

X rays microwaves

Infrared wavesultravioletVisible Visible

spectrumspectrum

400 500 600 700 [nm]

Page 9: Image aquisition system Visual perception basics

infrared

Spectral sensitivity characteristic of the human eye

ulraviolet

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 µm00

0.2

0.4

0.6

0.8

1rods

cones

Page 10: Image aquisition system Visual perception basics

Visual perception

Subjective brightness sensation assumes a logarithmic characteristic.

Human eye can percieve brightness in the range of 1010.

Log [mL]-6 -4 -2 0 2 4

Eye adaptation range

Local adaptation range

Glare limitGlare limit

Page 11: Image aquisition system Visual perception basics

Contrast sensitivity (Weber fraction)

The ratio is termed the Weber fraction.

It reflects contrast sensitivity characteristc of the human eye.

2%

I

I∆

I

3⋅10-1 3⋅101 3⋅103

I

I+∆ I

[cd/m2]

I

I∆

Page 12: Image aquisition system Visual perception basics

The eye achieves maximum sensitivity for:

I+∆I≈ I0

I

I∆

I

3 3⋅101 3⋅103

0I

[cd/m2]

I+∆ I

I

3⋅102

Io=3 Io=30 Io=300 Io=3000

Contrast sensitivity (Weber fraction)

Page 13: Image aquisition system Visual perception basics

Image coded using 16 gray levels

MIT4 bits/pixel

Page 14: Image aquisition system Visual perception basics

Mach bands

Subjective brightness

Brightness intensity

Page 15: Image aquisition system Visual perception basics

Visual illusions

Page 16: Image aquisition system Visual perception basics

Image aquisition system

Visual perception basics

Page 17: Image aquisition system Visual perception basics

Visual path of the image processing system

Visual path – a set optical and electronic elements converting

radiant energy into an electrical signal and imaging it using

display devices.

Image formationOpto-electrical

conversion Visualisation

3D

Imagingsensor

2D

Page 18: Image aquisition system Visual perception basics

Pros and cons:

• small hole → little light goes in

• large hole → image blurring

Pinhole camera (camera obscura)

Pin hole

Page 19: Image aquisition system Visual perception basics

Cameras today

CCD sensorCCD sensor

Pros and cons:

• sharp, high-contrast image

• geometric distortions

Pointgrey

Page 20: Image aquisition system Visual perception basics

Image formation model

x

y

Image plane of the imaging sensor

3D

Image formation model

(x,y) f(x,y)

(α,β)

2D

Page 21: Image aquisition system Visual perception basics

Image formation model

Image – a 2-D light intensity function

f(x,y)>=0 reflecting light energy

distribution

),(),(),( yxryxiyxf =

∞<< ),(0 yxi

1),(0 << yxr

- illumination (x,y)

reflectance coefficient at (x,y)

y

x

(x,y) f(x,y)

illumination: sunny day ~ 5000 cd/m2,

cloudy day ~ 1000 cd/m2, full moon ~ 0.001 cd/m2,

Reflectance coeff.: black velvet - 0.01, white wall - 0.8, snow - 0.93.

Page 22: Image aquisition system Visual perception basics

For a linear process of energy acummulation in the image

sensor plane:

( ) ( ) ( ) βαβαβα dd,,y,xh,fy,xf ∫ ∫∞

∞−

=

h(.) – is the impulse response of the system; in optical

systems it is termed the point spread function of the system

Image formation model

Page 23: Image aquisition system Visual perception basics

If the point spread function is shift invariant, then the

image formation model is given by a convolution integral:

( ) ( ) ( ) βαβαβα ddy,xh,fy,xf −−= ∫ ∫∞

∞−

+−=2

22

2

yxexp)y,x(h

σ

Image formation model

Page 24: Image aquisition system Visual perception basics

Sampling of 1-D signals

x

f(x)

ω

F(ω)

-Ω Ω

x

s(x)

∆xω

S(ω)

-1/∆x 1/∆x

s(x)f(x)

ω

S(ω)*F(ω)

-Ω Ω

. . .. . .

Page 25: Image aquisition system Visual perception basics

Assume the source image (analog image) features a

limited Fourier bandwidth

ωmaxx

ωmaxy

−ωmaxy

−ωmaxx

ωy

ωx

( )yx ,F ωω

Sampling of 2-D signals

Page 26: Image aquisition system Visual perception basics

( ) ( )∑ ∑−

=

=∆−∆−=

1

0

1

0,,

M

i

N

kykyxixyxS δ

Image sampling function:

and a sampled image:

( ) ( ) ( )

( ) ( )∑ ∑−

=

=∆−∆−∆∆=

==1

0

1

0,,

,,,M

i

N

k

s

ykyxixykxif

yxSyxfyxf

δ

∆x∆y

Sampling of 2-D signals

Page 27: Image aquisition system Visual perception basics

( ) ( )∑ ∑−

=

=∆−∆−

∆∆=

1

0

1

0,

1,

M

i

N

kyyxxyxs kiF

yxF ωωωωωω

Fourier spectrum of the sampled image:

where:

yx yx ∆=∆

∆=∆ 1

,1 ωω

∆ωx

∆ωy

Sampling of 2-D signals

Page 28: Image aquisition system Visual perception basics

∆ωx

∆ωy

ωmaxy

ωmaxx image bandwidth

xx

∆=∆<Ω

2

1

2maxω

Sampling of 2-D signals

Page 29: Image aquisition system Visual perception basics

Aliasing distortion - example

100 dpi(dots per inch)

500 dpiScanned images:

Page 30: Image aquisition system Visual perception basics

Image acquisition

Image acquisition is the process of converting light energy

radiating from image scene points into an electrical signal

(suitable for storing or transmission).

Image acquisition devices:

• CCD camera

• Video camera

• Scanner

• Digitizer

Page 31: Image aquisition system Visual perception basics

There are two basic schems of converting optical images

into electrical signals:

• without accumulation of photo-charges (eg. optical

scanner),

• with accumulation of photo-charges (np. vidicon,

CCD array)

Image acquisition

Page 32: Image aquisition system Visual perception basics

Imaging sensor (no photo-charges)

+

s(x,y)

photodiode

focusing screen

R

scanning direction

Page 33: Image aquisition system Visual perception basics

CCD array (accumulation of photo-charges)

Image formation is based on the internal photo-electric

phenomenon

UF<0Φ

n

Capacitor cell

electrical potential well

~10µm

insulator

Page 34: Image aquisition system Visual perception basics

The Bayer matrix

Raw CCD Format

Calculate RGB image by interpolating colour components from the Bayer matrix

N

M

N

M

Page 35: Image aquisition system Visual perception basics

Pixim – Digital Pixel System (DPS)

A/D converter for each pixel (no charge couplings)

Single A/D converter

Page 36: Image aquisition system Visual perception basics

CMOS image sensors

Pros:

• cheap technology (used for fabricating memory and CPU modules),

• low power consumption (100 times!)

• random access to pixel regions (block image processing)

• no „charge leaking” typical for CCD technology

• on-chip analog-to-digital conversion and signal processing

OmniVision

Cons:

• more susceptibel to noise

than CCD

• lower light sensitivity due to

many transistors used

for single pixel

Page 37: Image aquisition system Visual perception basics

Monochrome TV standards

European CCIR standard: (625 (575) lines, line display

time 64us, 50 half-images per sec., 1Vpp, 75Ω, signal

American RS170 standard: (525 (484) lines, line display

time 63,5 us, 60 half-images per sec.,1.4 Vpp, 75Ω signal

American RS-343 standard”: (875 lines, 60 half-images,

dedicated to CCTV, scientific applications,…)

Page 38: Image aquisition system Visual perception basics

TV CCIR standard

Composite video signal

peak white level

blanking level

sync level

odd lines

even lines625/575 lines

video signal

horizontal sync pulse

4

64 µs

0 V (DC)

0.7 V

-0.3 V

3

52 µs

Page 39: Image aquisition system Visual perception basics

Resolution:RS-170A: 580 horizontal TVL, 350 vertical TVL; CCIR: 560 horizontal TVL, 450 vertical TVL

Synchronization:Crystal/H&V/Asynchronous, standard

Shutter: 1/60 to 1/10,000 AGC: 20 dB Integration: 2 - 16 Fields Sensitivity:

Full video, No AGC: 0.65 lux; 80% video, AGC on: 0.04 lux; 30% video, AGC on: 0.008 lux

S/N Ratio (Gamma 1, gain 0 dB): 55 dB

Specification Highlights

Imager: 1/2" interline transfer CCD

Picture Elements: RS-170A: 768 (H) x 494 (V);CCIR: 752 (H) x 582 (V)

Pixel Cell Size: RS-170A: 8.4 µm (H) x 9.8 µm (V); CCIR: 8.6 µm (H) x 8.3 µm (V)

COHU® CCD camera

Page 40: Image aquisition system Visual perception basics

CCD image sensors characteristics

small size,

robust to mechanical vibrations (70 G),

no geometrical distortions,

low supply voltage (12 V, 1.4W),

SNR ~70 dB,

linear (gamma coefficient),

no intra-frame photo-charge accumulation,

high resolution,

reliable

cheap

Page 41: Image aquisition system Visual perception basics

Image frame grabber

Matrox Cronos Plus

Video capture board for PCI captures from NTSC, PAL, RS-170 and CCIR video sources, connect up to 4 CVBS or 1 Y/C trigger input, 7 TTL auxiliary I/Os, 32-bit/33MHz PCI-bus master Matrox ®

Software is sold separately, includes e.g., Matrox ®Imaging Library for Microsoft® Windows®