Professor Gonzalo R. Arce gave a lecture on "Compressed sensing in spectral imaging" in the Distinguished Lecturer Series - Leon The Mathematician.
More Information available at:
http://goo.gl/satkf
1. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
Compressive Spectral Imaging
Gonzalo R. Arce
Department of Electrical and Computer Engineering
University of Delaware
Email:arce@ece.udel.edu
Distinguished Lecture Series
Aristotle University of Thessaloniki
October 19th - 2010
Gonzalo R. Arce Compressive Spectral Imaging -1
2. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
Outline
Introduction to Compressive Sensing
Sparsity and ℓ1 Norm
Incoherent Sampling
Sparse Signal Recovery
Compressive Spectral Imaging
Single Shot CASSI System
Spectral Selectivity in (CASSI)
Random Convolution SSI (RCSSI)
Low-rank Anomaly Recovery in (CASSI)
Gonzalo R. Arce Compressive Spectral Imaging -2
3. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Traditional signal sampling and signal compression.
Nyquist sampling rate gives exact reconstruction.
Pessimistic for some types of signals!
Gonzalo R. Arce Compressive Spectral Imaging -3
4. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Sampling and Compression
Transform data and keep important coefficients.
Lots of work to then throw away majority of data!.
e.g. JPEG 2000 Lossy Compression: A digital camera can
take millions of pixels but the picture is encoded on a few
hundred of kilobytes.
Gonzalo R. Arce Compressive Spectral Imaging -4
5. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Problem: Recent applications require a very large number of
samples:
Higher resolution in medical imaging devices, cameras,
etc.
Spectral imaging, confocal microscopy, radar arrays, etc.
y
λ
x
Spectral Imaging
Medical Imaging
Gonzalo R. Arce Compressive Spectral Imaging -5
6. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Fundamentals of Compressive Sensing
Donoho † , Candès ‡ , Romberg and Tao, discovered
important results on the minimum number of data needed
to reconstruct a signal
Compressive Sensing (CS) unifies sensing and
compression into a single task
Minimum number of samples to reconstruct a signal
depends on its sparsity rather than its bandwidth.
†
D. Donoho. "Compressive Sensing". IEEE Trans. on Information Theory. Vol.52(2), pp.5406-5425, Dec.2006.
‡
E. Candès, J. Romberg and T. Tao. "Robust Uncertainty Principles: Exact Signal Reconstruction from Highly
Incomplete Frequency Information". IEEE Trans. on Information Theory. Vol.52(4), pp.1289-1306, Apr.2006.
Gonzalo R. Arce Compressive Spectral Imaging -6
7. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Sparsity
Signal sparsity critical to CS
Plays roughly the same role in CS that bandwidth plays in
Shannon-Nyquist theory
A signal x ∈ RN is S-sparse on the basis Ψ if x can be
represented by a linear combination of S vectors of Ψ as
x = Ψα with S ≪ N
At most S non-zero components
x Ψ
α
Gonzalo R. Arce Compressive Spectral Imaging -7
8. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
The ℓ1 Norm and Sparsity
Sparsity of x is measured by its number of non-zero
elements, the ℓ0 norm
x 0 = #{i : x(i) = 0}
The ℓ1 norm can be used to measure sparsity of x
x 1 = |x(i)|
i
The ℓ2 norm is not effective in measuring sparsity of x
x 2 =( |x(i)|2 )1/2
i
The ℓ0 and ℓ1 norms promote sparsity
Gonzalo R. Arce Compressive Spectral Imaging -8
9. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Why ℓ1 Norm Promotes Sparsity?
Given two N -dimensional signals:
x1 = (1, 0, ..., 0) → "Spike" signal
√ √ √
x2 = (1/ N , 1/ N , ..., 1/ N ) → "Comb" signal
x 2
x1 and x2 have the same ℓ2
norm:
x1 2 = 1 and x2 2 = 1.
x 1
However, x1
√ 1 = 1 and
x2 1 = N .
Gonzalo R. Arce Compressive Spectral Imaging -9
10. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Compressive Measurements
Measurements in CS are different than samples taken in
traditional A/D converters.
The signal x is acquired as a series of non-adaptive inner
products of different waveforms {φ1 , φ2 , ..., φM }
yk =< φk , x >; k = 1, ..., M ; with M ≪ N
y Φ x
Mx1
MxN
Measurements
Sampling Operator
Nx1
Sparse Signal
Gonzalo R. Arce Compressive Spectral Imaging -10
11. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Recoverability
yk =< φk , x >; k = 1, ..., M ; with M ≪ N
Recovering x from yk is an inverse problem.
Need to solve an under determined system of equations
y = Φx.
Infinitely solutions for the system since M ≪ N .
Amplitude
Amplitude
Original sparse signal Compressed measurements Reconstructed signal using least-squares.
Solution not sparse
Gonzalo R. Arce Compressive Spectral Imaging -11
12. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Recoverability: Incoherent Sampling
The number of samples required to recover x from M samples
depends on the mutual coherence between Φ and Ψ
Mutual Coherence
√
µ(Φ, Ψ) = N max{| < φk , ψ j > | : φk ∈ Rows(Φ), ψ j ∈ Columns(Ψ)};
where, ψj 2 = φk 2 =1
The coherence µ(Φ, Ψ) satisfies:
√
1 ≤ µ(Φ, Ψ) ≤ N
Gonzalo R. Arce Compressive Spectral Imaging -12
13. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Recoverability: Incoherent Sampling
The random measurement matrix Φ has to be incoherent
to the dictionary Ψ and x can be recovered from M
samples exactly when M satisfies:
M ≥ C · µ2 · S · log(N ), C ≥ 1
(a) (b)
(a) Very sparse vector.
(b) Examples of pseudorandom, incoherent test vectors φk † .
†
J. Romberg. "Imaging Via Compressive Sampling". IEEE Signal Processing Magazine. March,2008.
Gonzalo R. Arce Compressive Spectral Imaging -13
14. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Compressive Sensing Signal Reconstruction
Goal: Recover signal x from measurements y
Problem: Random projection Φ not full rank (ill-posed
inverse problem)
Solution: Exploit the sparse/compressible geometry of
acquired signal x
y Φ x
Gonzalo R. Arce Compressive Spectral Imaging -14
15. Introduction to Compressive Sensing Sparsity and ℓ1 norm
Compressive Spectral Imaging Incoherent Sampling
Low-rank Anomaly Recovery in (CASSI) Sparse Signal Recovery
Reconstruction Algorithms
Different formulations and implementations have been
proposed to find the sparsest x subject to y = Φx
Those are broadly classified in:
Regularization formulations (Replace combinatorial
problem with convex optimization)
Greedy algorithms (Iterative refinement of a sparse
solution)
Bayesian framework (Assume prior distribution of sparse
coefficients)
Gonzalo R. Arce Compressive Spectral Imaging -15
16. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Compressive Spectral Imaging
Collects spatial information from across the
electromagnetic spectrum.
Applications, include wide-area airborne surveillance,
remote sensing, and tissue spectroscopy in medicine.
Gonzalo R. Arce Compressive Spectral Imaging -16
17. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Compressive Spectral Imaging
Spectral Imaging System - Duke University†
†
A. Wagadarikar, R. John, R. Willett, D. Brady. "Single Disperser Design for Coded Aperture Snapshot Spectral Imaging."
Applied Optics, vol.47, No.10, 2008.
A. Wagadarikar and N. P. Pitsianis and X. Sun and D. J. Brady. "Video rate spectral imaging using a coded aperture
snapshot spectral imager." Opt. Express, 2009.
Gonzalo R. Arce Compressive Spectral Imaging -17
18. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot Compressive Spectral Imaging
System design
With linear dispersion:
f1 (x, y; λ) = f0 (x, y; λ)T (x, y)
f2 (x, y; λ) = δ(x′ − [x + α(λ − λc )]δ(y ′ − y)f1 (x′ , y ′ ; λ))dx′ dy ′
= δ(x′ − [x + α(λ − λc )]δ(y ′ − y)f0 (x′ , y ′ ; λ)T (x, y))dx′ dy ′
= f0 (x + α(λ − λc ), y; λ)T (x + α(λ − λc ), y)
Gonzalo R. Arce Compressive Spectral Imaging -18
19. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot Compressive Spectral Imaging
Experimental results from Duke University
Original Image
Reconstructed image cube of size:128x128x128.
Measurements Spatial content of the scene in each of 28
spectral channels between 540 and 640nm.
† A. Wagadarikar, R. John, R. Willett, D. Brady. "Single Disperser Design for Coded Aperture Snapshot Spectral Imaging."
Applied Optics, vol.47, No.10, 2008.
Gonzalo R. Arce Compressive Spectral Imaging -19
20. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot Compressive Spectral Imaging
Simulation results in RGB
Original Image Measurements
R
Reconstructed
¡ Image
Gonzalo R. Arce Compressive Spectral Imaging -20
21. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot CASSI System
Object with spectral information only in (xo , yo )
Only two spectral component are present in the object
Gonzalo R. Arce Compressive Spectral Imaging -21
22. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot CASSI System
Object with spectral information only in (xo , yo )
Gonzalo R. Arce Compressive Spectral Imaging -22
23. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot CASSI System
One pixel in the detector has information from different spectral
bands and different spatial locations
Gonzalo R. Arce Compressive Spectral Imaging -23
24. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot CASSI System
Each pixel in the detector has different amount of spectral
information. The more compressed information, the more
difficult it is to reconstruct the original data cube.
Gonzalo R. Arce Compressive Spectral Imaging -24
25. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot CASSI System
Each row in the data cube produces a compressed
measurement totally independent in the detector.
Gonzalo R. Arce Compressive Spectral Imaging -25
26. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot CASSI System
Undetermined equation system:
Unknowns = N × N × M and Equations: N × (N + M − 1)
Gonzalo R. Arce Compressive Spectral Imaging -26
27. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot CASSI System
Complete data cube 6 bands
The dispersive element shifts each spectral band in one
spatial unit
In the detector appear the compressed and modulated
spectral component of the object
At most each pixel detector has information of six spectral
components
Gonzalo R. Arce Compressive Spectral Imaging -27
28. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot CASSI System
We used the ℓ1 − ℓs reconstruction algorithm † .
†
S. J. Kim, K. Koh, M. Lustig, S. Boyd and D. Gorinevsky. "An interior-point method for large scale L1 regularized least
squares." IEEE Journal of Selected Topics in Signal Processing, vol.1, pp. 606-617, 2007.
Gonzalo R. Arce Compressive Spectral Imaging -28
29. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Coded Aperture Snapshot Spectral Image System
(CASSI)(a)
Advantages:
Enables compressive spectral imag-
ing
Simple
Low cost and complexity
Limitations:
Excessive compression
Does not permit a controllable SNR
May suffer low SNR gmn = f(m+k)nk P(m+k)n + wnm
Does not permit to extract a specific k
subset of spectral bands = (Hf )nm + wnm = (HW θ)nm + wnm
A. Wagadarikar, R. John, R. Willett, and D. Brady. "Single disperser design for coded aperture snapshot spectral imaging."
Appl. Opt., Vol.47, No.10, 2008.
Gonzalo R. Arce Compressive Spectral Imaging -29
30. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Bands Recovery
Typical example of a measurement of CASSI system. A set of bands
constant spaced between them are summed to form a measurement
Gonzalo R. Arce Compressive Spectral Imaging -30
31. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Multi-Shot CASSI System
Multi-shot compressive spectral imaging system
Advantages:
Multi-Shot CASSI allows
controllable SNR
Permits to extract a hand-
picked subset of bands
Extend Compressive Sens-
ing spectral imaging capabil-
ities
L
gmni = fk (m, n + k − 1)Pi (m, n + k − 1)
k=1
L
i
= fk (m, n + k − 1)Pr (m, n + k − 1)Pg (m, n + k − 1)
k=1
Ye, P. et al. "Spectral Aperture Code Design for Multi-Shot Compressive Spectral Imaging". Dig. Holography and
Three-Dimensional Imaging, OSA. Apr.2010.
Gonzalo R. Arce Compressive Spectral Imaging -31
32. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Mathematical Model of CASSI System
L
gmni = fk (m, n + k − 1)Pi (m, n + k − 1)
k=1
L
i
= fk (m, n + k − 1)Pr (m, n + k − 1)Pg (m, n + k − 1)
k=1
where i expresses ith shot
Each pattern Pi is given by,
i
Pi (m, n) = Pg (m, n)xPr (m, n)
i 1 mod(n, R) = mod(i, R)
Pg (m, n) =
0 otherwise
One different code aperture is used for each shot of CASSI system
Gonzalo R. Arce Compressive Spectral Imaging -32
33. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Code Apertures
Code patterns used
in multishot CASSI
system
Code patterns used in multishot CASSI system
Gonzalo R. Arce Compressive Spectral Imaging -33
34. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Cube Information and Subsets of Spectral Bands
Spectral axis, Spatial
L bands axis, N
Spectral data cube → L bands
pixels R subsets of M bands each one
Complete
Spectral (L = RM ) Each component
Data Cube of the subset is spaced by R
Spatial
bands of each other
axis, N
pixels
Subset 1
M bands
R R
Subset 1 Subset 2 Subset 3 ... Subset R
M=bands M=bands M=bands M bands
Gonzalo R. Arce Compressive Spectral Imaging -34
35. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Cube Information and Subsets of Spectral Bands
Spectral axis, Spatial
L bands axis, N Spectral data cube → L bands
pixels
R subsets of M bands each one
Complete (L = RM ) Each component
Spectral of the subset is spaced by R
Data Cube
Spatial bands of each other
axis, N
R R
pixels Subset 2
M bands
Subset 1 Subset
£ Subset
¢ ... Subset R
M=bands M=bands M=bands M=bands
Gonzalo R. Arce Compressive Spectral Imaging -35
36. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Multi-Shot CASSI System
First shot and Second shot and R shot and
measurement measurement measurement
Gonzalo R. Arce Compressive Spectral Imaging -36
37. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Single Shot
Multi-Shot
One shot of CASSI Information of all band exists in all shots
system. One high
compressing
measurement.
First shot Second shot Third shot
Reconstruction
Algorithm
Re-organization
algorithm
Reconstructed
spectral data
cube.
Bands 1,4,7 Bands 2,5,8 Bands 3,6,9
Gonzalo R. Arce Compressive Spectral Imaging -37
38. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Multi-Shot
Reorder Process R R R
′ L
gmnk = j=1 fj (m, n + j − 1)Pi (m, n + j − 1)
L i First shot Second shot Third shot
= j=1 fj (m, n + j − 1)Pr (m, n + j − 1)Pg (m, n + j − 1) Re-organization
algorithm
= mod(n+j−1,R)=mod(i,R) fk (m, n + k − 1)Pr (m, n + j − 1)
= (Hk Fk )mn
Bands 1,4,7 Bands 2,5,8 Bands 3,6,9
Gonzalo R. Arce Compressive Spectral Imaging -38
39. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Multi-Shot
Reorder Process R
R R
′ L
gmnk = j=1 fj (m, n + j − 1)Pi (m, n + j − 1)
L i First shot Second shot Third shot
= j=1 fj (m, n + j − 1)Pr (m, n + j − 1)Pg (m, n + j − 1) Re-organization
algorithm
= mod(n+j−1,R)=mod(i,R) fk (m, n + k − 1)Pr (m, n + j − 1)
= (Hk Fk )mn
Bands 1,4,7 Bands 2,5,8 Bands 3,6,9
Gonzalo R. Arce Compressive Spectral Imaging -39
40. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Multi-Shot
Recover any of the subsets
independently
Recover of complete spec-
tral data cube is not neces-
sary
Gonzalo R. Arce Compressive Spectral Imaging -40
41. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Multi-Shot
High SNR in each re-
construction
Enable to use paral-
lel processing
To use one proces-
sor for each indepen-
dent reconstruction
Gonzalo R. Arce Compressive Spectral Imaging -41
42. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Multi-Shot
Single Shot
One shot of CASSI
system. One high
compressing
measurement.
Reconstruction
Algorithm
Reconstructed
spectral data
cube.
Gonzalo R. Arce Compressive Spectral Imaging -42
43. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Multi-Shot Reconstruction
Reconstructed image of one spec-
tral channel in 256x256x24 data
cube from multiple shot measure-
ments.
(a) One shot result,PSNR
(a) One shot (b) 2 shots
P SN R = 17.6dB
(b) Two shots result,PSNR
P SN R = 25.7dB
(c) Eight shots result,PSNR
P SN R = 29.4
(d) Original image
(c) 8 shots (d) Original
Gonzalo R. Arce Compressive Spectral Imaging -43
44. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Multi-Shot Reconstruction
Reconstructed image for dif-
ferent spectral channels in the
256x256x24 data cube from
six shot measurements.
(a) Band 1
(b) Band 13
(c) Band 8
(d) Band 20
(a) and (b) are recon-
structed from the first
group of measurements
(c) and (d) are recon-
structed from the second
group of measurements
Gonzalo R. Arce Compressive Spectral Imaging -44
45. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Random Convolution Spectral Imaging
Gonzalo R. Arce Compressive Spectral Imaging -45
46. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Random Convolution Imaging
J. Romberg. "Compressive Sensing by Random Convolution." SIAM Journal on Imaging Science, July,2008.
Gonzalo R. Arce Compressive Spectral Imaging -46
47. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Random Convolution Imaging
Random Convolution
Circularly convolve signal x ∈ Rn with a pulse h ∈ Rn , then
subsample.
The pulse is random, global, and broadband in that its energy is
distributed uniformly across the discrete spectrum.
x ∗ h = Hx
where
H = n−1/2 F ∗ ΣF
Ft,ω = e−j2π(t−1)(ω−1)/n , 1 ≤ t, ω ≤ n
Σ as a diagonal matrix whose non-zero entries are the Fourier
transform of h.
Gonzalo R. Arce Compressive Spectral Imaging -47
48. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Random Convolution
σ1 0 · · ·
0 σ2 · · ·
Σ=
.
. ..
. .
σn
ω=1 : σ1 ∼ ±1 with equal probability,
2 ≤ ω < n/2 + 1 : σω = ejθω , where θω ∼ Uniform([0, 2π]),
ω = n/2 + 1 : σn/2+1 ∼ ±1 with equal probability,
n/2 + 2 ≤ ω ≤ n : ∗
σω = σn−ω+2 , the conjugate of σn−ω+2 .
Gonzalo R. Arce Compressive Spectral Imaging -48
49. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Random Convolution
H
The effect of H on a signal x can be broken down into a
discrete Fourier transform, followed by a randomization of
the phase (with constraints that keep the entries of H real),
followed by an inverse discrete Fourier transform.
Since F F ∗ = F ∗ F = nI and ΣΣ∗ = I,
H ∗ H = n−1 F ∗ Σ∗ F F ∗ ΣF = nI
So convolution with h as a transformation into a random
orthobasis.
Gonzalo R. Arce Compressive Spectral Imaging -49
51. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Main Result
H will not change the magnitude of the Fourier transform,
so signals which are concentrated in frequency will remain
concentrated and signals which are spread out will stay
spread out.
The randomness of Σ will make it highly probable that a
signal which is concentrated in time will not remain so after
H is applied.
Gonzalo R. Arce Compressive Spectral Imaging -51
52. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Main Result
(a) A signal x consisting of a single Daubechies-8 wavelet.
(b) Magnitude of the Fourier transform F x.
(c) Inverse Fourier transform after the phase has been
randomized. Although the magnitude of the Fourier transform is
the same as in (b), the signal is now evenly spread out in time.
J. Romberg. "Compressive Sensing by Random Convolution." SIAM Journal on Imaging Science, July,2008.
Gonzalo R. Arce Compressive Spectral Imaging -52
53. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Fourier Optics
Fourier optics imaging experiment.
(a) The 256 × 256 image x.
(b) The 256 × 256 image Hx.
(c) The 64 × 64 image P θHx.
Gonzalo R. Arce Compressive Spectral Imaging -53
54. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
(a) The 256 × 256 image we wish to acquire.
(b) High-resolution image pixellated by averaging over 4 × 4 blocks.
(c) The image restored from the pixellated version in (b), plus a set of
incoherent measurements. The incoherent measurements allow us to
effectively super-resolve the image in (b).
Gonzalo R. Arce Compressive Spectral Imaging -54
55. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Fourier Optics
a) b) c)
d) e) f)
Pixellated images: (a) 2 × 2. (b) 4 × 4. (c) 8 × 8. Restored from: (d) 2 × 2 pixellated
version. (e) 4 × 4 pixellated version. (f) 8 × 8 pixellated version.
Gonzalo R. Arce Compressive Spectral Imaging -55
56. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
Random Convolution Spectral Imaging
Gonzalo R. Arce Compressive Spectral Imaging -56
57. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
20
40
60
80
100
120
20 40 60 80 100 120
Gonzalo R. Arce Compressive Spectral Imaging -57
58. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)
Compressive Spectral Imaging Spectral Selectivity in (CASSI)
Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)
20
40
60
80
100
120
20 40 60 80 100 120
Gonzalo R. Arce Compressive Spectral Imaging -58
59. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
Low-rank Anomaly Recovery in (CASSI)
Spectral video analysis
Video surveillance: Anomaly detection
Stationary background corresponds to low-rank contribution
and the moving objects corresponds to sparse data.
Gonzalo R. Arce Compressive Spectral Imaging -59
60. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
Connection Between Low-Rank Matrix Recovery and
Compressed Sensing
Low-rank Rank miniz. Convex Relax.
Recovery min rank(X)
L min L
s.t. M=S+L s.t. M=S+L
Compressed Rank miniz. Convex Relax.
Sensing
B. Recht, M. Fazel and P. Parrilo, "Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm
Minimization," SIAM Review, Aug. 2010.
Gonzalo R. Arce Compressive Spectral Imaging -60
61. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
Low-Rank Anomaly Recovery in (CASSI)
Problem Description
(i)
Consider the video surveillance of Fk,n1,n2 ∈ RN1 ×N2 ×K ,
i = 1, ..., N frames.
The ith scene is assumed to be composed by a stationary
background L(i) and an event changing in time S(i) ,
(i) (i) (i)
Fk,n1,n2 = Lk,n1,n2 + Sk,n1 ,n2
CASSI encodes both 2D spatial information and spectral
information in a 2Dsingle measurement G(i) for
i = 1, ..., N .
GOAL: recover anomalies occurring in both time and spectra
from a sequence of spectrally compressed video frames
G(1) , G(2) , ..., G(N ) .
Gonzalo R. Arce Compressive Spectral Imaging -61
62. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
Low-Rank Anomaly Recovery in (CASSI)
Recovering anomalies:
Form G as the large data matrix G = [g(1) , g(2) , . . . , g(N ) ],
where g(i) is the column representation of G(i) .
G = L + S where L is the stationary background and S is
sparse capturing the anomalies in the foreground
Gonzalo R. Arce Compressive Spectral Imaging -62
63. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
Principal Component Pursuit
The matrix G is decomposed into a low-rank matrix L and a
sparse matrix S, such that
G =L+S (1)
Using Principal Component Pursuit.
Principal Component Pursuit
min L ∗ +λ S 1
n
L ∗ = i=1 σi (L), is the nuclear norm of L.
S 1 = ij Sij is the ℓ1 -norm of the matrix S
E. J. Candès, X. Li, Y. Ma, and J. Wright. "Robust Principal Component Analysis?," Submitted to Journal of the ACM.
2009.
Gonzalo R. Arce Compressive Spectral Imaging -63
64. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
Low-Rank Anomaly Recovery in (CASSI)
Spectral recovery of anomalies.
Coded measurements in S have been biased by the
background reconstruction
ˆ
Identify spatial location of the anomalies in S by:
ˆ
Filter |S| with a Weighted Median (WM) filter as
(i) ˆ (i)
Mn1 ,n2 = MEDIAN{Tv,w ⋄ |Sn1 +v,n2 +w | : (v, w) ∈ [−3, 3]}
where T is a WM filter of size (L × L) with centered weight
(L + 1)/2, and linearly decreasing weights
Spectrally coded measurements of anomalies denoted by
˜
G(i) are estimated as
G(i) = G(i) ⊙ U(M(i) − Th )
˜
Th is a thresholding parameter that extracts the pixels that
are most likely to be in the region of interest
Gonzalo R. Arce Compressive Spectral Imaging -64
65. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
Low-Rank Anomaly Recovery in (CASSI)
ˆ ˜
Recover S(i) from G(i) by
ˆ(i) = Ψ min( g(i) − HΨθ (i)
s ˜ 2
2 + τ θ (i) 1 ) (2)
θ
s ˜ ˜
where ˆ(i) and g(i) are the column representation of S(i)
˜
and G (i) , respectively.
Gonzalo R. Arce Compressive Spectral Imaging -65
66. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
(video) (video)
Gonzalo R. Arce Compressive Spectral Imaging -66
67. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
(video) (video)
Gonzalo R. Arce Compressive Spectral Imaging -67
68. Introduction to Compressive Sensing
Compressive Spectral Imaging
Low-rank Anomaly Recovery in (CASSI)
(video) (video)
Gonzalo R. Arce Compressive Spectral Imaging -68