Characterization of Alpha Toxin hla Gene Variants, Alpha Toxin ...
Posterior Matching Variants and Fixed-Point Eliminationofersha/allerton09seminar.pdf · 2010. 1....
Transcript of Posterior Matching Variants and Fixed-Point Eliminationofersha/allerton09seminar.pdf · 2010. 1....
-
Posterior Matching Variants andFixed-Point Elimination
Ofer Shayevitz, UCSD
Allerton 2009
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 1/14
-
Feedback Communication Setting
⊆ (0,1)
Message point Θ0 ∼ Unif(0, 1)
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 2/14
-
Feedback Communication Setting
⊆ (0,1)
Message point Θ0 ∼ Unif(0, 1)
Transmission scheme achieves a rate R if
PΘ0|Y n(∆n|Yn) → 1 , |∆n(Y
n))| = O(
2−nR)
Possibly under input constraints limn→∞
n−1n
∑
k=1
η(Xk) ≤ Γ a.s.
Implies standard achievability
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 2/14
-
The Posterior Matching (PM) Scheme[Shayevitz & Feder ’07,’08]
For any input/channel pair (PX , PY |X)
Xn+1 = F−1X ◦ FΘ0|Y n(Θ0|Y
n)
Simple and sequential, achieves R < I(X;Y ) under general
conditions, within input constraints encapsulated in PX
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 3/14
-
The Posterior Matching (PM) Scheme[Shayevitz & Feder ’07,’08]
For any input/channel pair (PX , PY |X)
Xn+1 = F−1X ◦ FΘ0|Y n(Θ0|Y
n)
Simple and sequential, achieves R < I(X;Y ) under general
conditions, within input constraints encapsulated in PX
Specifically, holds for (minus issues immediately discussed)
DMC
PXY with a sufficiently smooth p.d.f.
Horstein & Schalkwijk-Kailath are special cases
Stochastic control formulation [Coleman ’09]
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 3/14
-
Normalized Channel and Recursive Representation
PM scheme recursive representation [Shayevitz & Feder ’08]
Θ1 = Θ0 , Θn+1 = FΘ|Y (Θn|Yn)
FΘ|Y (·|·) is called the PM kernel
Posterior c.d.f. evolves by function composition with PM kernel
FΘ0|Y n(·|Yn) = FΘ|Y (·|Yn) ◦ FΘ0|Y n−1(·|Y
n−1)
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 4/14
-
A Necessary Condition for Achievability
The PM kernel has a fixed-point at θf if
FΘ|Y (θf |y) = θf , ∀ y ∈ Y
Claim: If the PM kernel has a fixed point, then the PM scheme isnot ergodic and cannot achieve any positive rate
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 5/14
-
A Necessary Condition for Achievability
The PM kernel has a fixed-point at θf if
FΘ|Y (θf |y) = θf , ∀ y ∈ Y
Claim: If the PM kernel has a fixed point, then the PM scheme isnot ergodic and cannot achieve any positive rate
For a DMC, maximal number of fixed-points is |X | − 2
No fixed point for a binary input alphabet
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 5/14
-
A Necessary Condition for Achievability
The PM kernel has a fixed-point at θf if
FΘ|Y (θf |y) = θf , ∀ y ∈ Y
Claim: If the PM kernel has a fixed point, then the PM scheme isnot ergodic and cannot achieve any positive rate
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 5/14
-
3 × 2 Example
PM kernel comprised of two quasi-linear functions
Suppose there is a fixed point, i.e., FΘ|Y (θf |0) = FΘ|Y (θf |1) = θf
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 6/14
-
3 × 2 Example
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 7/14
-
3 × 2 Example
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 7/14
-
3 × 2 Example
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 7/14
-
3 × 2 Example
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 7/14
-
3 × 2 Example
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 7/14
-
3 × 2 Example – What can we do?
Two invariant intervals
Idea I:
Decode two messages
Resolve (e.g. via repetition)
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 8/14
-
3 × 2 Example – What can we do?
Two invariant intervals
Idea I:
Decode two messages
Resolve (e.g. via repetition)
Problems:
Rate per invariant interval can be < I(X;Y )
Non-ergodic chain, input constraints not necessarily satisfied
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 8/14
-
3 × 2 Example – What can we do?
Two invariant intervals
Idea II:
Map message to ”best“ interval
R ≥ I(X;Y )
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 9/14
-
3 × 2 Example – What can we do?
Two invariant intervals
Idea II:
Map message to ”best“ interval
R ≥ I(X;Y )
Equivalent to PM with a different (binary) input distribution
Related to the observation that when |X | > |Y|, using |Y| input
symbols suffices to achieve the unconstrained capacity [Shannon ’57]
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 9/14
-
3 × 2 Example – What can we do?
Two invariant intervals
Idea II:
Map message to ”best“ interval
R ≥ I(X;Y )
Equivalent to PM with a different (binary) input distribution
Related to the observation that when |X | > |Y|, using |Y| input
symbols suffices to achieve the unconstrained capacity [Shannon ’57]
Problem: Input constraints not satisfied
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 9/14
-
3 × 2 Example – How Common?
Choose PXY uniformly over the simplex
Let FΘ|Y be the corresponding PM kernel
How likely is a fixed-point for FΘ|Y ?
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 10/14
-
3 × 2 Example – How Common?
Choose PXY uniformly over the simplex
Let FΘ|Y be the corresponding PM kernel
How likely is a fixed-point for FΘ|Y ?
Claim: P(
FΘ|Y has a fixed point)
= 13
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 10/14
-
3 × 2 Example – How Common?
Proof:
Define δk = PX|Y (k|1) − PX|Y (k|0), k = 0, 1, 2∑
δk = 0
Suppose |δ0| ≤ |δ2| ≤ |δ1|, hence δ1 = −(δ0 + δ2)
The c.d.f’s must intersect
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 11/14
-
3 × 2 Example – How Common?
Proof:
Define δk = PX|Y (k|1) − PX|Y (k|0), k = 0, 1, 2∑
δk = 0
Suppose |δ0| ≤ |δ2| ≤ |δ1|, hence δ1 = −(δ0 + δ2)
The c.d.f’s must intersect
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 11/14
-
3 × 2 Example – How Common?
Proof:
Define δk = PX|Y (k|1) − PX|Y (k|0), k = 0, 1, 2∑
δk = 0
Suppose |δ0| ≤ |δ2| ≤ |δ1|, hence δ1 = −(δ0 + δ2)
The c.d.f’s must intersect
E(FΘ|Y (θ|Y )) = FΘ(θ) = θ
Intersection is a fixed point
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 11/14
-
3 × 2 Example – How Common?
Proof:
Define δk = PX|Y (k|1) − PX|Y (k|0), k = 0, 1, 2∑
δk = 0
Suppose |δ0| ≤ |δ2| ≤ |δ1|, hence δ1 = −(δ0 + δ2)
The c.d.f’s must intersect
E(FΘ|Y (θ|Y )) = FΘ(θ) = θ
Intersection is a fixed point
Can always permute/relabel the input so that δk satisfies this
Fixed point obtained also for mirror case
2 out of 6 permutations ⇒ fixed point
Result follows from symmetry
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 11/14
-
A Systematic Solution
Note the glass is 23 full..
Relabeling the input s.t. δ0 ≥ δ1 ≥ δ2
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 12/14
-
A Systematic Solution
Note the glass is 23 full..
Relabeling the input s.t. δ0 ≥ δ1 ≥ δ2 ⇒ no fixed points!
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 12/14
-
A Systematic Solution
Note the glass is 23 full..
Relabeling the input s.t. δ0 ≥ δ1 ≥ δ2 ⇒ no fixed points!
For any DMC with I(X;Y ) > 0
∃ y1, y2 ∈ Y s.t. PX|Y (·|y1) 6= PX|Y (·|y2).
Find an input permutation σ sorting the δk, for which
Fσ(X)|Y (·|y1) < Fσ(X)|Y (·|y2)
Consider PM for the equivalent input/channel pair (Pσ(X), PY |σ(X))
No fixed points, achieves I(X ; Y ) within input constraints
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 12/14
-
The General Case
Beyond permutations: A bijective function µ : (0,1) 7→ (0,1) is called
uniformity preserving if
Θ ∼ Unif(0,1) ⇒ µ(Θ) ∼ Unif(0,1)
The µ-variant of the PM scheme is
Xn+1 = F−1X ◦ µ ◦ FΘ0|Y n(Θ0|Y
n)
µ-variant kernel µ ◦ Fµ−1(Θ)|Y (·|y) ◦ µ−1
µ eliminates fixed points, I(X;Y ) achieved under input constraints
For DMC, µ was a permutation of intervals that correspond to thediscrete input alphabet
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 13/14
-
Summary
With fixed points, PM fails to achieve capacity
PM variants can eliminate fixed points, achieving capacity
Many variants achieving capacity, some “closer” to having a
fixed-point than others
What is the “best” variant, minimizing error probability for a giveninput/channel pair?
Ofer Shayevitz, UCSD Posterior Matching Variants and Fixed-Point Elimination – p. 14/14
s {Feedback Communication Setting}s {Feedback Communication Setting}
s {The Posterior Matching (PM) Scheme {iny [Shayevitz & Feder'07,'08]}}s {The Posterior Matching (PM)Scheme {iny [Shayevitz & Feder '07,'08]}}
s {Normalized Channel and Recursive Representation}s {A Necessary Condition for Achievability}s {A Necessary Condition for Achievability}s {A Necessary Condition for Achievability}
s {$3imes 2$ Example}s {$3imes 2$ Example}s {$3imes 2$ Example}s {$3imes 2$ Example}s {$3imes 2$ Example}s {$3imes 2$ Example}
s {$3imes 2$ Example -- What can we do?}s {$3imes 2$ Example -- What can we do?}
s {$3imes 2$ Example -- What can we do?}s {$3imes 2$ Example -- What can we do?}s {$3imes 2$ Example -- What can we do?}
s {$3imes 2$ Example -- How Common?}s {$3imes 2$ Example -- How Common?}
s {$3imes 2$ Example -- How Common?}s {$3imes 2$ Example -- How Common?}s {$3imes 2$ Example -- How Common?}s {$3imes 2$ Example -- How Common?}
s {A Systematic Solution}s {A Systematic Solution}s {A Systematic Solution}
s {The General Case}s {Summary}