A primer on Information Theory & Fundamentals of Digital Communications.

61
A primer on Information Theory & Fundamentals of Digital Communications

Transcript of A primer on Information Theory & Fundamentals of Digital Communications.

Page 1: A primer on Information Theory & Fundamentals of Digital Communications.

A primer on Information Theory &Fundamentals of Digital Communications

Page 2: A primer on Information Theory & Fundamentals of Digital Communications.

2

Network/Link Design Factors Transmission media

Signals are transmitted over transmission media

Examples: telephone cables, fiber optics, twisted pairs, coaxial cables

Channel capacity and Bandwidth (εύρος ζώνης) Higher channel capacity (measured in Hz),

gives higher bandwidth (range of frequencies), gives higher data rate

Transmission impairments Attenuation (εξασθένηση) Interference (παρεμβολή)

Number of receivers In guided media More receivers (multi-point) introduce more

attenuation

Page 3: A primer on Information Theory & Fundamentals of Digital Communications.

3

Channel Capacity and data rate Bandwidth

In cycles per second, or Hertz (e.g. 10 Mhz bandwidth)

Available bandwidth for signal transmission is constrained by transmitter and transmission medium makeup and capacity

Data rate Related to the bandwidth capabilities of the

medium in bits per second (the higher the bandwidth

the higher the rate) Rate at which data can be communicated Baud rate (symbols/sec) ≠ bit rate (bits/sec)

• Number of symbol changes made to the transmission medium per second

• One symbol can carry one or more bits of information

Page 4: A primer on Information Theory & Fundamentals of Digital Communications.

4

Data Rate and Bandwidth Any transmission system has a limited

band of frequencies This limits the data rate that can be

carried E.g., telephone cables due to

construction can carry signals within frequencies 300Hz – 3400Hz

This has severe effect on what signals and what capacity in effect the channel has.

To understand this lets consider any complex signal and see if we can analyse its frequency content, and what effect the channel may have on the signal (distortion?) and in effect limit the rate of the transmitted signal (measured in bits/sec)

Page 5: A primer on Information Theory & Fundamentals of Digital Communications.

5

Frequency content of signals http://

www.allaboutcircuits.com/vol_2/chpt_7/2.html

any repeating, non-sinusoidal waveform can be equated to a combination of DC voltage, sine waves, and/or cosine waves (sine waves with a 90 degree phase shift) at various amplitudes and frequencies.

This is true no matter how strange or convoluted the waveform in question may be. So long as it repeats itself regularly over time, it is reducible to this series of sinusoidal waves.

Page 6: A primer on Information Theory & Fundamentals of Digital Communications.

6

Fourier series

Mathematically, any repeating signal can be represented by a series of sinusoids in appropriate weights, i.e. a Fourier Series.

http://en.wikipedia.org/wiki/Fourier_series

Page 7: A primer on Information Theory & Fundamentals of Digital Communications.

The Mathematic Formulation

A periodic function is any function that satisfies

( ) ( )f t f t T

where T is a constant and is called the period of the function.

Note: for a sinusoidal waveform the frequency is the reciprocal of the period (f=1/T)

Page 8: A primer on Information Theory & Fundamentals of Digital Communications.

Synthesis

T

ntb

T

nta

atf

nn

nn

2sin

2cos

2)(

11

0

DC Part Even Part Odd Part

T is a period of all the above signals

)sin()cos(2

)( 01

01

0 tnbtnaa

tfn

nn

n

Let 0=2/T.

Page 9: A primer on Information Theory & Fundamentals of Digital Communications.

112

200

dta

,2,1 0sin1

cos2

200

nntn

ntdtan

,6,4,20

,5,3,1/2)1cos(

1 cos

1sin

2

100

n

nnn

nnt

nntdtbn

2 3 4 5--2-3-4-5-6

f(t)1

Example (Square Wave)

-0.5

0

0.5

1

1.5

ttttf 5sin

5

13sin

3

1sin

2

2

1)(

Page 10: A primer on Information Theory & Fundamentals of Digital Communications.

Fourier series example Thus, square waves (and indeed and

waves) are mathematically equivalent to the sum of a sine wave at that same frequency, plus an infinite series of odd-multiple frequency sine waves at diminishing amplitude

10

Page 11: A primer on Information Theory & Fundamentals of Digital Communications.

11

Another example

With 4 sinusoids we represent quite well a triangular waveform

Page 12: A primer on Information Theory & Fundamentals of Digital Communications.

12

The ability to represent a waveform as a series of sinusoids can be seen in the opposite way as well:

What happens to a waveform if sent through a bandlimited (practical) channel

E.g. some of the higher frequencies are removed, so signal is distorted…

E.g what happens if a square waveform of period T is sent through a channel with bandwidth (2/T)?

Page 13: A primer on Information Theory & Fundamentals of Digital Communications.

13

The Electromagnetic Spectrum

The electromagnetic spectrum and its uses for communication.

Note: frequency is related to wavelength (f=1/l)

Useful spectrum is limited, therefore its allocation/usage is managed

Page 14: A primer on Information Theory & Fundamentals of Digital Communications.

14

Electromagnetic Spectrum

Note: frequency is related to wavelength (f=1/l) which is related to antenna length for radio, capacity, power and distance trade-offs exist, etc…

Page 15: A primer on Information Theory & Fundamentals of Digital Communications.

15

Generally speaking there is a push into higher frequencies due to: efficiency in propagation, immunity to some forms of noise and

impairments as well as the size of the antenna required.

The antenna size is typically related to the wavelength of the signal and in practice is usually ¼ wavelength.

Page 16: A primer on Information Theory & Fundamentals of Digital Communications.

Data and Signal: Analog or Digital Data

Digital data – discrete value of data for storage or communication in computer networks

Analog data – continuous value of data such as sound or image

Signal Digital signal – discrete-time signals

containing digital information Analog signal – continuous-time signals

containing analog information

16

Page 17: A primer on Information Theory & Fundamentals of Digital Communications.

Periodic and Aperiodic Signals (1/4) Spectra of periodic analog signals: discrete

f1=100 kHz

400k Frequency

Amplitude

Time

100k

Amplitude

f2=400 kHz periodic analog signal

17

Page 18: A primer on Information Theory & Fundamentals of Digital Communications.

Periodic and Aperiodic Signals (2/4) Spectra of aperiodic analog signals:

continous

aperiodic analog signal

f1

Amplitude

Amplitude

f2

Time

Frequency

18

Page 19: A primer on Information Theory & Fundamentals of Digital Communications.

Periodic and Aperiodic Signals (3/4) Spectra of periodic digital signals: discrete

(frequency pulse train, infinite)

frequency = f kHzAmplitude periodic digital signal

Amplitude frequency pulse train

Time

Frequencyf 2f 3f 4f 5f

...

...

19

Page 20: A primer on Information Theory & Fundamentals of Digital Communications.

Periodic and Aperiodic Signals (4/4) Spectra of aperiodic digital signals:

continuous (infinite)

aperiodic digital signalAmplitude

Amplitude

0

Time

Frequency

...

20

Page 21: A primer on Information Theory & Fundamentals of Digital Communications.

21

Sine Wave

Peak Amplitude (A) maximum strength of signal volts

Frequency (f) Rate of change of signal Hertz (Hz) or cycles per second Period = time for one repetition (T) T = 1/f

Phase () Relative position in time

Page 22: A primer on Information Theory & Fundamentals of Digital Communications.

22

Varying Sine Waves

Page 23: A primer on Information Theory & Fundamentals of Digital Communications.

23

Signal Properties

Page 24: A primer on Information Theory & Fundamentals of Digital Communications.

24

Figure 1.8 Modes of transmission: (a) baseband transmission

Baseband Transmission

Page 25: A primer on Information Theory & Fundamentals of Digital Communications.

25

Modulation (Διαμόρφωση)

Η διαμόρφωση σήματος είναι μία διαδικασία κατά την οποία, ένα σήμα χαμηλών συχνοτήτων (baseband signal), μεταφέρεται από ένα σήμα με υψηλότερες συχνότητες που λέγεται φέρον σήμα (carrier signal)

Μετατροπή του σήματος σε άλλη συχνότητα

Χρησιμοποιείται για να επιτρέψει τη μεταφορά ενός σήματος σε συγκεκριμένη ζώνη συχνοτήτων π.χ. χρησιμοποιείται στο ΑΜ και FM ραδιόφωνο

Page 26: A primer on Information Theory & Fundamentals of Digital Communications.

26

Πλεονεκτήματα Διαμόρφωσης Δυνατότητα εύκολης μετάδοσης του

σήματος Δυνατότητα χρήσης πολυπλεξίας

(ταυτόχρονη μετάδοση πολλαπλών σημάτων)

Δυνατότητα υπέρβασης των περιορισμών των μέσων μετάδοσης

Δυνατότητα εκπομπής σε πολλές συχνότητες ταυτόχρονα

Δυνατότητα περιορισμού θορύβου και παρεμβολών

Page 27: A primer on Information Theory & Fundamentals of Digital Communications.

27

Modulated Transmission

Page 28: A primer on Information Theory & Fundamentals of Digital Communications.

28

Continuous & Discrete SignalsAnalog & Digital Signals

Page 29: A primer on Information Theory & Fundamentals of Digital Communications.

29

Analog Signals Carrying Analog and Digital Data

Page 30: A primer on Information Theory & Fundamentals of Digital Communications.

30

Digital Signals Carrying Analog and Digital Data

Page 31: A primer on Information Theory & Fundamentals of Digital Communications.

31

Digital Data, Digital Signal

Page 32: A primer on Information Theory & Fundamentals of Digital Communications.

32

Encoding (Κωδικοποίηση) Signals propagate over a physical

medium modulate electromagnetic waves e.g., vary voltage

Encode binary data onto signals binary data must be encoded before

modulation e.g., 0 as low signal and 1 as high signal

• known as Non-Return to zero (NRZ)

Bits

NRZ

0 0 1 0 1 1 1 1 0 1 0 0 0 0 1 0

Page 33: A primer on Information Theory & Fundamentals of Digital Communications.

33

Encodings (cont)Bits

NRZ

Clock

Manchester

NRZI

0 0 1 0 1 1 1 1 0 1 0 0 0 0 1 0

If the encoded data contains long 'runs' of logic 1's or 0's, this does not result in any bit transitions. The lack of transitions makes impossible the detection of the boundaries of the received bits at the receiver.This is the reason why Manchester coding is used in Ethernet.

Page 34: A primer on Information Theory & Fundamentals of Digital Communications.

34

Other Encoding Schemes

Unipolar NRZ Polar NRZ Polar RZ Polar Manchester and Differential

Manchester Bipolar AMI and Pseudoternary Multilevel Coding Multilevel Transmission 3 Levels RLL

Page 35: A primer on Information Theory & Fundamentals of Digital Communications.

The Waveforms of Line Coding Schemes

1 1 1 1 1 10 0 0 0 0 0

Clock

Data stream

Polar RZ

Polar NRZ-L

Manchester

Polar NRZ-I

Differential Manchester

AMI

MLT-3

Unipolar NRZ-L

35

Page 36: A primer on Information Theory & Fundamentals of Digital Communications.

The Waveforms of Line Coding Schemes

1 1 1 1 1 10 0 0 0 0 0

Clock

Data stream

Polar RZ

Polar NRZ-L

Manchester

Polar NRZ-I

Differential Manchester

AMI

MLT-3

Unipolar NRZ-L

36

Page 37: A primer on Information Theory & Fundamentals of Digital Communications.

Bandwidths of Line Coding (2/3)• The bandwidth of Manchester.

• The bandwidth of AMI.

1N 2N Frequncy

Power Bandwidth of Manchester Line Codingsdr=2, average baud rate = N (N, bit rate)

00

1.0

0.5

N/2 3N/2

1N 2N Frequncy

Power Bandwidth of AMI Line Codingsdr=1, average baud rate = N/2 (N, bit rate)

00

1.0

0.5

N/2 3N/2

37

Page 38: A primer on Information Theory & Fundamentals of Digital Communications.

Bandwidths of Line Coding (3/3)

1N 2N Frequncy

Power Bandwidth of 2B1Q Line Codingsdr=1/2, average baud rate=N/4 (N, bit rate)

00

1.0

0.5

N/2 3N/2

• The bandwidth of 2B1Q

38

Page 39: A primer on Information Theory & Fundamentals of Digital Communications.

39

Digital Data, Analog Signal

After encoding of digital data, the resulting digital signal must be modulated before transmitted

Use modem (modulator-demodulator) Amplitude shift keying (ASK) Frequency shift keying (FSK) Phase shift keying (PSK)

Page 40: A primer on Information Theory & Fundamentals of Digital Communications.

40

Modulation Techniques

Page 41: A primer on Information Theory & Fundamentals of Digital Communications.

Constellation Diagram (1/2) A constellation diagram: constellation

points with two bits: b0b1

+1-1

+1

-1

I

Amplitue

Amplitue of I component

Amplitue of Q component

PhaseIn-phase Carrier

QQuadrature Carrier

1101

1000

41Chapter 2: Physical Layer

Page 42: A primer on Information Theory & Fundamentals of Digital Communications.

Amplitude Shift Keying (ASK) and Phase Shift Keying (PSK) The constellation diagrams of ASK and

PSK.

(a) ASK (OOK): b0 (b) 2-PSK (BPSK): b0 (c) 4-PSK (QPSK): b0b1 (d) 8-PSK: b0b1b2 (e) 16-PSK: b0b1b2

+1-1

+1

-1

Q

I

1101

1000

Q

I

110011

101000

111

100

001

010Q

I+1-1

Q

I

10

+1

Q

I0

10

42Chapter 2: Physical Layer

Page 43: A primer on Information Theory & Fundamentals of Digital Communications.

The Circular Constellation Diagrams

The constellation diagrams of ASK and PSK.

(a) Circular 4-QAM: b0b1 (b) Circular 8-QAM: b0b1b2 (c) Circular 16-QAM: b0b1b2b3

Q

I+1-1

+1

-1

Q

I+1+ 3-1 - 3

+1+ 3

-1 - 3

+1-1

+1

-1

Q

I

1101

1000

43Chapter 2: Physical Layer

Page 44: A primer on Information Theory & Fundamentals of Digital Communications.

The Rectangular Constellation Diagrams

(a) Alternative Rectangular 4-QAM: b0b1

(b) Rectangular 4-QAM: b0b1

(c) Alternative Rectangular 8-QAM: b0b1b2

(d) Rectangular 8-QAM: b0b1b2

(e) Rectangular 16-QAM: b0b1b2b3

+1 +3-3 -1

+1

-1

Q

I +1-1

+1

-1

Q

I +1 +3-3 -1

+1

+3

-1

-3

Q

I

101111110011 0111

101011100010 0110

100011000000 0100

100111010001 0101+1

+1

Q

I-1

-1

+1

+1Q

I0

44Chapter 2: Physical Layer

Page 45: A primer on Information Theory & Fundamentals of Digital Communications.

45

Quadrature PSK More efficient use if each signal

element (symbol) represents more than one bit e.g. shifts of /2 (90o) 4 different phase

angles Each element (symbol) represents

two bits• With 2 bits we can represent the 4 different

phase angles• E.g. Baud rate = 4000 symbols/sec and each

symbol has 8 states (phase angles). Bit rate=??

If a symbol has M states each symbol can carry log2M bits

Can use more phase angles and have more than one amplitude

• E.g., 9600bps modem use 12 angles, four of which have two amplitudes

Page 46: A primer on Information Theory & Fundamentals of Digital Communications.

46

Modems (2)

(a) QPSK.(b) QAM-16.(c) QAM-64.

Page 47: A primer on Information Theory & Fundamentals of Digital Communications.

47

Modems (3)

(a) V.32 for 9600 bps.(b) V32 bis for 14,400 bps.

(a) (b)

Page 48: A primer on Information Theory & Fundamentals of Digital Communications.

48

Ak

Bk

16 “levels”/ pulse4 bits / pulse4W bits per second

Ak

Bk

4 “levels”/ pulse2 bits / pulse2W bits per second

2-D signal2-D signal

Page 49: A primer on Information Theory & Fundamentals of Digital Communications.

49

Ak

Bk

4 “levels”/ pulse2 bits / pulse2W bits per second

Ak

Bk

16 “levels”/ pulse4 bits / pulse4W bits per second

Page 50: A primer on Information Theory & Fundamentals of Digital Communications.

50

Analog Data, Digital Signal

Page 51: A primer on Information Theory & Fundamentals of Digital Communications.

51

Signal Sampling and Encoding

Page 52: A primer on Information Theory & Fundamentals of Digital Communications.

52

Digital Signal Decoding

Page 53: A primer on Information Theory & Fundamentals of Digital Communications.

53

Alias generation due to undersampling

Page 54: A primer on Information Theory & Fundamentals of Digital Communications.

54

Nyquist Bandwidth

If rate of signal transmission is 2B then signal with frequencies no greater than B is sufficient to carry signal rate

Given bandwidth B, highest signal (baud) rate is 2B

Given binary signal, data rate supported by B Hz is 2B bps (if each symbol carries one bit)

Can be increased by using M signal levels

C= 2B log2M

Page 55: A primer on Information Theory & Fundamentals of Digital Communications.

55

Transmission Impairments

Signal received may differ from signal transmitted

Analog degradation of signal quality Channel impairements Digital bit errors Caused by

Attenuation and attenuation distortion Delay distortion Noise

Page 56: A primer on Information Theory & Fundamentals of Digital Communications.

56

Attenuation Signal strength falls off with distance Depends on medium Received signal strength:

must be enough to be detected must be sufficiently higher than noise to be

received without error Attenuation is an increasing function

of frequency

Page 57: A primer on Information Theory & Fundamentals of Digital Communications.

57

Noise (1)

Additional signals, exisiting or inserted, between transmitter and receiver

Thermal Due to thermal agitation of electrons Uniformly distributed White noise

Intermodulation Signals that are the sum and difference of

original frequencies sharing a medium

Page 58: A primer on Information Theory & Fundamentals of Digital Communications.

58

Noise (2)

Crosstalk A signal from one line is picked up by

another Impulse

Irregular pulses or spikes e.g. External electromagnetic interference Short duration High amplitude

Page 59: A primer on Information Theory & Fundamentals of Digital Communications.

59

signal noise signal + noise

signal noise signal + noise

HighSNR

LowSNR

SNR = Average Signal Power

Average Noise Power

SNR (dB) = 10 log10 SNR

t t t

t t t

Page 60: A primer on Information Theory & Fundamentals of Digital Communications.

60

Shannon’s Theorem

Real communication have some measure of noise. This theorem tells us the limits to a channel’s capacity (in bits per second) in the presence of noise. Shannon’s theorem uses the notion of signal-to-noise ratio (S/N), which is usually expressed in decibels (dB):

)/(log10 10 NSdB

Page 61: A primer on Information Theory & Fundamentals of Digital Communications.

61

Shannon’s Theorem – cont.

))/(1(log2 NSBC

Kbps30)10001(log3000 2 C

Shannon’s Theorem:

C: achievable channel rate (bps)B: channel bandwidth

For POTS, bandwidth is 3000 Hz (upper limit of 3300 Hz and lower limit of 300 Hz), S/N = 1000

For S/N = 100

Kbps5.19)1001(log3000 2 C