The Chomsky-Schutzenb erger Theorem for Quantitative Context … · 2020. 12. 18. ·...

The Chomsky-Schützenberger Theorem for

Quantitative Context-Free Languages

Manfred Droste, University of Leipzig, GermanyHeiko Vogler, TU Dresden, Germany

DLT 2013

context-free CF

1/23

G : ρ1 : S → AS A

ρ2 : S → b

ρ3 : A→ a

2/23

G : ρ1 : S → AS A

ρ2 : S → b

ρ3 : A→ a

S → [1ρ1A]1ρ1 [

2ρ1S ]

2ρ1 [

3ρ1A]

3ρ1

S → [ρ2 ]ρ2A→ [ρ3 ]ρ3

2/23

G : ρ1 : S → AS A

ρ2 : S → b

ρ3 : A→ a

S → [1ρ1A]1ρ1 [

2ρ1S ]

2ρ1 [

3ρ1A]

3ρ1

S → [ρ2 ]ρ2A→ [ρ3 ]ρ3

3/23

G : ρ1 : S → AS A

ρ2 : S → b

ρ3 : A→ a

S → [1ρ1A]1ρ1 [

2ρ1S ]

2ρ1 [

3ρ1A]

3ρ1

S → [ρ2 ]ρ2A→ [ρ3 ]ρ3

[1ρ1 [ρ3 ]ρ3 ]1ρ1 [

2ρ1 [ρ2 ]ρ2 ]

2ρ1 [

3ρ1 [ρ3 ]ρ3 ]

3ρ1

3/23

G : ρ1 : S → AS A

ρ2 : S → b

ρ3 : A→ a

S → [1ρ1A]1ρ1 [

2ρ1S ]

2ρ1 [

3ρ1A]

3ρ1

S → [ρ2 ]ρ2A→ [ρ3 ]ρ3

[1ρ1 [ρ3 ]ρ3 ]1ρ1 [

2ρ1 [ρ2 ]ρ2 ]

2ρ1 [

3ρ1 [ρ3 ]ρ3 ]

3ρ1 ∈ D({[

1ρ1 , [

2ρ1 , [

3ρ1 , [ρ2 , [ρ3}︸︷︷︸Y

)

Dyck-language

3/23

G : ρ1 : S → AS A

ρ2 : S → b

ρ3 : A→ a

S → [1ρ1A]1ρ1 [

2ρ1S ]

2ρ1 [

3ρ1A]

3ρ1

S → [ρ2 ]ρ2A→ [ρ3 ]ρ3

[1ρ1 [ρ3 ]ρ3 ]1ρ1 [

2ρ1 [ρ2 ]ρ2 ]

2ρ1 [

3ρ1 [ρ3 ]ρ3 ]

3ρ1 ∈ D({[

1ρ1 , [

2ρ1 , [

3ρ1 , [ρ2 , [ρ3}︸︷︷︸Y

)

Dyck-language

[ρ3 ]ρ3 [2ρ1 [ρ3 ]ρ3 ]

2ρ1 [

1ρ1 [ρ3 ]ρ3 ]

1ρ1 ∈ D(Y )

3/23

G : ρ1 : S → AS A

ρ2 : S → b

ρ3 : A→ a

S → [1ρ1A]1ρ1 [

2ρ1S ]

2ρ1 [

3ρ1A]

3ρ1

S → [ρ2 ]ρ2A→ [ρ3 ]ρ3

[1ρ1 [ρ3 ]ρ3 ]1ρ1 [

2ρ1 [ρ2 ]ρ2 ]

2ρ1 [

3ρ1 [ρ3 ]ρ3 ]

3ρ1 ∈ D({[

1ρ1 , [

2ρ1 , [

3ρ1 , [ρ2 , [ρ3}︸︷︷︸Y

)

Dyck-language

[ρ3 ]ρ3 [2ρ1 [ρ3 ]ρ3 ]

2ρ1 [

1ρ1 [ρ3 ]ρ3 ]

1ρ1 ∈ D(Y )

check local properties: D(Y ) ∩ R

3/23

G : ρ1 : S → AS A

ρ2 : S → b

ρ3 : A→ a

S → [1ρ1A]1ρ1 [

2ρ1S ]

2ρ1 [

3ρ1A]

3ρ1

S → [ρ2 ]ρ2A→ [ρ3 ]ρ3

[1ρ1 [ρ3 ]ρ3 ]1ρ1 [

2ρ1 [ρ2 ]ρ2 ]

2ρ1 [

3ρ1 [ρ3 ]ρ3 ]

3ρ1 ∈ D({[

1ρ1 , [

2ρ1 , [

3ρ1 , [ρ2 , [ρ3}︸︷︷︸Y

)

Dyck-language

[ρ3 ]ρ3 [2ρ1 [ρ3 ]ρ3 ]

2ρ1 [

1ρ1 [ρ3 ]ρ3 ]

1ρ1 ∈ D(Y )

check local properties: D(Y ) ∩ R

alphabetic morphism:

h : Y ∪ Y → Σ ∪ {ε} h(y) =

b if y = [ρ2a if y = [ρ3ε otherwise

3/23

Theorem [Chomsky, Schützenberger 63]

Let L ⊆ Σ∗.

If L = L(G ) for some CF grammar G ,

then there are

I an alphabet Y ,

I a recognizable language R over Y ∪ Y , andI an alphabetic morphism h : Y ∪ Y → Σ ∪ {ε}

such that L = h′(D(Y ) ∩ R).

Moreover, for every w ∈ L(G ):

|DG (w)| = |{u ∈ D(Y ) ∩ R | h′(u) = w}|

(degree of ambiguity)

4/23

Theorem [Chomsky, Schützenberger 63]

Let L ⊆ Σ∗.

If L = L(G ) for some CF grammar G ,

then there are

I an alphabet Y ,


such that L = h′(D(Y ) ∩ R).

Moreover, for every w ∈ L(G ):

|DG (w)| = |{u ∈ D(Y ) ∩ R | h′(u) = w}|

(degree of ambiguity)

L : Σ∗ → N; w 7→ |DG (w)|

4/23

weighted languages (formal power series) L : Σ∗ → K

(K ,+, ·, 0, 1) semiringI (K ,+, 0) commutative monoid

I (K , ·, 1) monoidI · distributes over +I 0 is absorbing for ·, i.e., a · 0 = 0 · a = 0

Examples:

1. natural numbers (N,+, ·, 0, 1)2. Boolean semiring ({true, false},∨,∧, false, true)3. distributive bounded lattices (L,∨,∧, 0, 1)4. fields

5/23

weighted CF languages over semiring (K ,+, ·, 0, 1):

I CF grammar with weights G = (N,Σ,Z ,P,wt)wt : P → K

I weight of a derivation: wt(ρ1 . . . ρn) = wt(ρ1) · . . . · wt(ρn)

I weighted CF grammar:I CF grammar with weights andI K is complete or DG (w) is finite for every w ∈ Σ∗

I weighted language of

a weighted CF grammar

G :

||G || : Σ∗ → K , w 7→∑

d∈DG (w) wt(d)

I L : Σ∗ → K is a weighted CF language:∃ weighted CF grammar G over semiring K : L = ||G ||.

6/23



I weight of a derivation: wt(ρ1 . . . ρn) = wt(ρ1) · . . . · wt(ρn)


I weighted language of a weighted CF grammar G :

||G || : Σ∗ → K , w 7→∑

d∈DG (w) wt(d)


6/23

Theorem [Chomsky, Schützenberger 63] (revisited)

Let L : Σ∗ → N and (N,+, ·, 0, 1) natural number semiring.

If L = L(G ) for weighted CF grammar G with wt(p) = 1for every p ∈ P,

then there are

I an alphabet Y ,


such that L = h′(char(D(Y ) ∩ R))

7/23

Theorem [Salomaa, Soittola 78]

Let L : Σ∗ → K and (K ,+, ·, 0, 1) commutative semiring.

The following are equivalent:

1. L is weighted CF language

2. there areI an alphabet Y ,I a recognizable language R over Y ∪ Y , andI an alphabetic morphism h : Y ∪ Y → KΣ∪{ε}

such that L = h′(char(D(Y ) ∩ R))

8/23

quantitative languages L : Σ∗ → K “average”[Chatterjee, Doyen, Henzinger 10]

(K ,+, val, 0) valuation monoid [Droste, Meinecke 10]

I (K ,+, 0) commutative monoid

I val : K+ → K , val(a) = a, val(. . . 0 . . .) = 0

9/23


(K ,+, val, 0) valuation monoid [Droste, Meinecke 10]


I val : K+ → K , val(a) = a, val(. . . 0 . . .) = 0

Examples:

1. (R ∪ {−∞}, sup, avg,−∞) avg(a1 . . . an) = 1n ·∑n

i=1 ai

2. strong bimonoid (K ,+, ·, 0, 1)I (K ,+, 0) commutative monoidI (K , ·, 1) monoidI 0 absorbing for ·

e.g., semirings, bounded lattices

9/23


(K ,+, val, 0

, 1

)

unital

valuation monoid

[Droste, Vogler 13]


I val : K+ → K , val(a) = a, val(. . . 0 . . .) = 0

I val(. . . 1 . . .) = val(. . . . . .) ignore “1”s

I val(ε) = 1

10/23


(K ,+, val, 0, 1) unital valuation monoid [Droste, Vogler 13]


I val : K ∗ → K , val(a) = a, val(. . . 0 . . .) = 0I val(. . . 1 . . .) = val(. . . . . .) ignore “1”s

I val(ε) = 1

10/23





I val(ε) = 1

Example:

(R ∪ {−∞}, sup, avg,−∞) avg(a1 . . . an) = 1n ·∑n

i=1 ai

(R ∪ {−∞,+∞}, sup, avg,−∞,+∞)

avg(. . .+∞ . . .) = avg(. . . . . .)

10/23





I val(ε) = 1

10/23





I val(ε) = 1

In general, val is

I not commutative

I not associative

I not distributive over +

10/23



I weight of a derivation:wt(ρ1 . . . ρn) = wt(ρ1) · . . . · wt(ρn)



||G || : Σ∗ → K , w 7→∑

d∈DG (w) wt(d)


11/23

quantitative CF languages over unital val. monoid (K ,+, val, 0, 1):





||G || : Σ∗ → K , w 7→∑

d∈DG (w) wt(d)


12/23






||G || : Σ∗ → K , w 7→∑

d∈DG (w) wt(d)


13/23



I weight of a derivation:wt(ρ1 . . . ρn) = val(wt(ρ1) . . .wt(ρn))



||G || : Σ∗ → K , w 7→∑

d∈DG (w) wt(d)


14/23






||G || : Σ∗ → K , w 7→∑

d∈DG (w) wt(d)


15/23





I quantitative language of a weighted CF grammar G :

||G || : Σ∗ → K , w 7→∑

d∈DG (w) wt(d)


16/23





I quantitative language of a weighted CF grammar G :

||G || : Σ∗ → K , w 7→∑

d∈DG (w) wt(d)

I L : Σ∗ → K is a quantitative CF language:∃ weighted CF gr. G over unital val. monoid K : L = ||G ||.

17/23

... in a very similar way define:

weighted pushdown automaton over unital valuation monoid

18/23

... in a very similar way define:

weighted pushdown automaton over unital valuation monoid

Theorem [Droste, Vogler 13]

Let L : Σ∗ → K and (K ,+, val, 0, 1) unital val. monoid.The following are equivalent:

1. L is quantitative CF language

2. L is quantitative behaviour of a weighted PDA.

18/23




2. there areI an alphabet ∆,I an unambiguous CF grammar G over ∆, andI an alphabetic morphism h : ∆→ KΣ∪{ε}

such that L = h′(L(G ))

19/23






h′ : ∆∗ → KΣ∗

h′(δ1 . . . δn) = val(a1 . . . an) . x1 . . . xn if h(δi ) = ai .xi

19/23






h′ : ∆∗ → KΣ∗


L ⊆ ∆∗h′(L) =

∑v∈L h

′(v)

19/23






h′ : ∆∗ → KΣ∗


L ⊆ ∆∗h′(L) =

∑v∈L h

′(v) if (h′(v) | v ∈ L) locally finiteor K complete.

19/23






20/23







such that L = h′(D(Y ) ∩ R)

21/23







such that L = h′(D(Y ) ∩ R)

thanks

21/23





such that L = h′(D(Y ) ∩ R)4. there are

I an alphabet Y ,I a deterministically recognizable series r ∈ K (Y∪Y )∗ , andI an alphabetic morphism h : Y ∪ Y → Σ ∪ {ε}

such that L = h′(D(Y ) ∩ r)

(D(Y ) ∩ r) ∈ K (Y∪Y )∗ w 7→{

(r ,w) if w ∈ D(Y )0 otherwise

23/23

The Chomsky-Schutzenb erger Theorem for Quantitative Context … · 2020. 12. 18. ·...

Documents

Transcript of The Chomsky-Schutzenb erger Theorem for Quantitative Context … · 2020. 12. 18. ·...