Modeling Delta Encoding of Compressed Files S.T. Klein, T.C. Serebro, D. Shapira.
-
Upload
laurel-rose -
Category
Documents
-
view
219 -
download
0
Transcript of Modeling Delta Encoding of Compressed Files S.T. Klein, T.C. Serebro, D. Shapira.
Delta Encoding
Example:
S=The Prague Stringology ClubT=The Prague Stringology Conference 06
Δ=(1, 24)onferenc(3,2)06
Compressed Differencing
Goal- Create a delta file of S and T, without decompressing the compressed files.
S T
Δ(S,T)
E(S)Delta encoding:Semi Compressed Differencing:
E(T)SE(S)Full Compressed Differencing:
LZW compressionSTR = input character WHILE there are input characters {C = input character IF STR C is in T then
STR = STR C ELSE {
output the code for STR add STR C to T STR = C
}} output the code for STR
construct the trie of E(S)i 1while i ≤ u{ P Starting at the root,
traverse the trie using P When a leaf v is reached k depth of v in trie output the position in S
corresponding to v ii+ k}
uii TTT ...1
Semi Compressed Differencing Algorithm
Full Compressed Differencing Algorithm1 construct the trie of E(S)2 flag 0 // output character k3 counter 1 // position in T4 input oldcw from E(T)5 while oldcwNULL // still processing E(T) {5.1 input cw from E(T)5.2 node Dictionary[oldcw]5.3 if (Dictionary[cw] NULL)5.3.1 k first character of string corresponding to Dictionary[cw]5.4 else5.4.1 k first character of string corresponding to node5.5 if ((node has a child k) and (cwNULL))5.5.1 output (pos+flag,len-flag) corresponding to child k of node5.5.2 flag 15.6 else5.6.1 output (pos+flag, len-flag) corresponding to node5.6.2 create a new child of node corresponding to k5.6.3 flag 05.7 pos of child k of node counter5.8 oldcw cw5.9 counter counter + len - flag }
ExampleE(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=coldcw=3
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccoldcw=3cw=3
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccoldcw=3cw=3k=c
3
Example
4(1,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccoldcw=3cw=3k=cΔ(S,T)=<3, 2>
3
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccoldcw=3cw=3k=c
Example
4(1,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccoldcw=3cw=3 flag=1k=cΔ(S,T)=<3, 2>
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccboldcw=3cw=3 flag=1k=c
Example
4(1,2,c)
Δ(S,T)=<3, 2>
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccboldcw=3cw=2 flag=1k=c
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccboldcw=3cw=2 flag=1k=b
<5, 1>
5(2,2,c)
Example
4(1,2,c)
Δ(S,T)=<3, 2>
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccboldcw=3cw=2 flag=1k=b
<5, 1>
5(2,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbboldcw=3cw=2 flag=1k=b
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbboldcw=2cw=2 flag=1k=b
6(3,2,b)
b
<b, 0>
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbboldcw=2cw=2 flag=0k=b
Example
4(1,2,c)
5(2,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbboldcw=2cw=2 flag=0k=b
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaoldcw=2cw=2 flag=0k=b
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaoldcw=2cw=1 flag=0k=a
6(3,2,b)
Δ(S,T)=<3, 2> <5, 1>
4(1,2,c)
5(2,2,c)
7(4,2,b)
<5, 2>
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaoldcw=2cw=1 flag=1k=a
b
Example
4(1,2,c)
Δ(S,T)=<3, 2> <5, 1>
5(2,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaoldcw=2cw=1 flag=1k=a
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaboldcw=2cw=1 flag=1k=a
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbaboldcw=1cw=2 flag=1k=b
6(3,2,b)
4(1,2,c)
5(2,2,c)
7(4,2,b)
<5, 2>
8(5,2,a)
<2,1>
b
Example
4(1,2,c)
Δ(S,T)=<3, 2> <5, 1>
5(2,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccoldcw=2cw=4 flag=1k=c
6(3,2,b)
b
4(1,2,c)
5(2,2,c)
7(4,2,b)
<5, 2>
8(5,2,a)
<2,1>
9(6,2,b)
<3, 1>
Example
4(1,2,c)
Δ(S,T)=<3, 2> <5, 1>
5(2,2,c)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbaoldcw=4cw=7 flag=1k=b
6(3,2,b)
7(4,2,b)
<5, 2>
8(5,2,a)
<2,1>
9(6,2,b)
<3, 1>
10(7,3,c)
b
(2, 1)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbaoldcw=4cw=7 flag=0k=b
b
Example
4(1,2,c)
Δ(S,T)=<3, 2> <5, 1>
5(2,2,c)
6(3,2,b)
b
7(4,2,b)
<5, 2>
8(5,2,a)
<2,1>
9(6,2,b)
<3, 1>
10(7,3,c)
b
(2, 1)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbabcoldcw=7cw=9 flag=0k=b
11(9,3,b)
b
(4, 2)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbabccboldcw=9cw=5 flag=0k=c
<9, 3>
12(11,3,b)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbabccboldcw=9cw=5 flag=1k=c
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbabccbbaoldcw=5cw=7 flag=1k=b
13(13,3,c)
b
(3, 1)
E(S) =1233219571 E(T) =33221247957S =abccbaaabccba T=ccbbabccbabccbbaoldcw=7cw=Null flag=0k=b
(4, 2)
Combination of Pairs
Δ(S,T)=<3, 2> <5, 1> <5, 2><2,1> <3, 1> (2, 1)(4, 2)<9, 3> (3, 1)(4, 2)
S =abccbaaabccbaS =abccbaaabccba
<3, 2> <5, 1>
S =abccbaaabccba
<3, 3>
If two consecutive ordered pairs are of the form and , we combine them into a single ordered pair
1, li 21, lli 21, lli
Combination of Pairs
If two consecutive ordered pairs are of the form and , we combine them into a single ordered pair
1, li 21, lli 21, lli
Δ(S,T)= <5, 2><2,1> <3, 1> (2, 1)(4, 2)<9, 3> (3, 1)(4, 2)
S =abccbaaabccbaS =abccbaaabccbaS =abccbaaabccba
<3, 3> <2,1><3, 1><2, 2>
Δ(S,T)= <5, 2> (4, 2) <9, 3> (4, 2)<3, 3> <2,2 > c b
Encoding the delta fileΔ(S,T)= <5, 2> (4, 2) <9, 3> (4, 2)<3, 3> <2,2 > c b
File consists of:
(pos, len) in S
(pos, len) in T
Characters
flags