Information Theory and Coding Shannon Code

م. فؤاد حمادي

9102 - 9191

Note: If we use fixed length coding L = [ log 2] = [ 2.3219] = 3

ξ code= H ( x ) / L * 100 % =2 . 228 3* 100 % = 74 %

.: coding in Shannon – fanon is more efficient than coding in fixed length

coding .

Ex2 : A source produce 5 independent symbols ( x1, x2, x3, x4, x5 ) with its

corresponding probabilities 0.1, 0.05, 0.25, 0.5, 0.1. design a binary code

for the above source symbol using Shannon – fanon method .

Ex3 : A source produce 5 independent symbols ( A, B, C, D, Z ) with the

below probabilities: 1. For the letters (A, B, C, D ) , the probability of

each letter is twice as its successor 2. The probability of the letter Z is

equal to the summation of the probabilities of B and D.

design a binary code for the above source symbol using Shannon – fanon

method .

Shannon Code

Example

Develop the Shannon code for the following set of messages,

p(x) = [0.3 0.2 0.15 0.12 0.1 0.08 0.05]

then find

a) Code efficiency,

b) p(0) at the encoder output.

Solution