Compressive Sensing with Biorthogonal Wavelets via Structured
Transcript of Compressive Sensing with Biorthogonal Wavelets via Structured
![Page 1: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/1.jpg)
Compressive Sensing with Biorthogonal Wavelets
via Structured Sparsity
Richard G. BaraniukMarco F. Duarte
![Page 2: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/2.jpg)
• Data x K-sparse in orthonormal basis : • Measure linear projections onto incoherent basis
where data is not sparse
• Reconstruct via optimization or greedy algorithms
Compressive Imaging
project transmit
receive recovery
x
(CoSaMP, OMP, IHT, ...)
![Page 3: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/3.jpg)
• Random matrices lead to RIP with high probability if
when is an orthonormal basis
• i.i.d. Subgaussian entries: Gaussian, Rademacher, ...
Φ
• RIP of order 2K implies: for all K-sparse and
[Candès and Tao 2005]• Preserve distances between sparse/compressible signals
Restricted Isometry Property (RIP)
RIP of order 2K enables K-sparse signal recovery
![Page 4: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/4.jpg)
Example: Compressive Imaging• Example: Recovery via CoSaMP
using 2-D Daubechies-8 wavelet
N = 262144, M = 60000
SNR = 17.93dB
• 2-D wavelets and 2-D DCT are common
Original
![Page 5: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/5.jpg)
• JPEG 2000
• Lossy compression via transform coding• Cohen-Daubechies-Feauveau 9/7
Biorthogonal Wavelet (CDF 9/7)
State-of-the-Art Image Compression
![Page 6: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/6.jpg)
• Biorthogonal wavelets involve:– an analysis basis – a synthesis basis
• Transform coding:
• Analysis and synthesis bases are not orthonormal:
• Standard guarantees and algorithms are not necessarily suitable for biorthogonal wavelets
Properties of Biorthogonal Wavelets
![Page 7: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/7.jpg)
Beyond Orthonormal Bases• Dictionary: arbitrary matrix (basis/frame)
that provides sparsity• Dictionary coherence
• Theorem:If has rows and
then the matrix has RIP of order K
• Sadly, CDF 9/7 synthesis basis has large coherence:
[Rauhut, Schnass, Vandergheynst 2008]
![Page 8: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/8.jpg)
CS with Coherent Dictionaries
• For tight frame dictionaries with arbitrary coherence, can use -analysis: -norm minimization for analysis coefficients
• New -RIP preserves distances between signal vectors instead of coefficient vectors
• -RIP is tailored to dictionary and enables guarantees for recovery via -analysis (in contrast to -synthesis)
• Random matrix has -RIP if
[Candès, Eldar, Needell, Randall 2010]
![Page 9: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/9.jpg)
Compressive Imaging via Biorthogonal Wavelets
• Example: Recovery via CoSaMP using CDF 9/7 wavelet
N = 262144, M = 60000
SNR = 4.6dBOriginal
![Page 10: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/10.jpg)
• Example: Recovery via -synthesis using CDF 9/7 wavelet
N = 262144, M = 60000
SNR = 21.54dB!Original
Compressive Imaging via Biorthogonal Wavelets
![Page 11: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/11.jpg)
Benefits of Biorthogonal Wavelets
• Why does -synthesis work well?• Because biorthogonal wavelet analysis and
synthesis bases are “interchangeable”:
• Thus, we have and the following two formulations are equivalent:
( -synthesis)
( -analysis)
![Page 12: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/12.jpg)
CDF 9/7 Recovery Artifacts
“Ringing”
![Page 13: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/13.jpg)
Coherence of Biorthogonal Wavelets
• Each wavelet is coherent with spatial neighbors across different wavelet orientations and scales
• Ringing artifacts caused by ambiguity due to coherent/neighbor wavelets during sparse wavelet selection
• Can inhibit supports that include coherent pairs of neighboring wavelets
![Page 14: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/14.jpg)
Structured Sparsity Models
• Promote structure common in natural images
• Inhibit selection of additional highly-coherent wavelet pairs
![Page 15: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/15.jpg)
• Modify existing greedy algorithms that rely on thresholding (e.g. CoSaMP)
• Replace thresholding with best structured sparse approximation that finds the closest point to input x in a restricted union of subspaces that encodes structure:
Structured Sparse Recovery Algorithms
x
ΩK ⊆ ΣK
RN
[Baraniuk, Cevher, Duarte, Hegde 2010]
![Page 16: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/16.jpg)
• Measurements needed:for random matrices with i.i.d entries,
Connected Rooted Subtree Sparsity
without structure with structure
• Structure: K-sparse coefficients + nonzero coefficients lie on a rooted subtree
• Approximation algorithm: – condensing sort and select [Baraniuk]
– dynamic programming [Donoho]
– computational complexity:
![Page 17: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/17.jpg)
CS via Biorthogonal Wavelets
• Example: Recovery via tree-CoSaMP using CDF 9/7 wavelet
N = 262144, M = 60000
SNR = 23.31dBOriginal
![Page 18: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/18.jpg)
-analysisCDF 9/7
SNR = 23.31dB
Tree-CoSaMPCDF 9/7
SNR = 22.14dB
Tree-CoSaMPDaubechies-8
SNR = 21.54dB
SNR = 17.93dB
CoSaMPDaubechies-8
SNR = 4.60dB
CoSaMPCDF 9/7
OriginalN = 262144
![Page 19: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/19.jpg)
Performance Comparison - Cameraman
N = 262144
-analysis/CDF97
![Page 20: Compressive Sensing with Biorthogonal Wavelets via Structured](https://reader031.fdocument.org/reader031/viewer/2022022801/621adc74b494fc0daf230533/html5/thumbnails/20.jpg)
Conclusions• Structured sparsity enables improved greedy
algorithms for compressive imaging with 2-D biorthogonal wavelets– promote structure present in wavelet representations of
natural images– inhibit interference from neighboring wavelets that do not
match model– simple-to-implement modifications to recovery that are
computationally efficient– reduced number of random measurements required for
improved image recovery
• Current and future work:– analytical study of 2-D biorthogonal wavelets (coherence,
RIP) for compressive imaging– Extensions to redundant wavelet frames
dsp.rice.edu/csdsp.rice.edu/~richbcs.duke.edu/~mduarte