Talk at SAMPTA 2013 conference, 2013/07/02.

1. Gabriel Peyré
www.numerical-tours.com
Exact Support Recovery
for Sparse Spikes
Deconvolution
Vincent Duval
VISI N

2. Overview
• Sparse Spikes Deconvolution
• Robust Support Recovery
• Vanishing Derivative Certificate
• Ideal Low Pass Filter

3. Radon measure m on T = R/Z.
Discrete measure:
ma,x =
PN
i=1 ai xi , a 2 RN
, x 2 TN
Deconvolution of Measures
ma,x

4. y = (m)
Convolution measurements:
' 2 C2
(T)
Radon measure m on T = R/Z.
Discrete measure:
ma,x =
PN
i=1 ai xi , a 2 RN
, x 2 TN
y(t) =
R
T
'(x t)dm(x)
Deconvolution of Measures
ma,x
'

5. y = (m)
Convolution measurements:
' 2 C2
(T)
Radon measure m on T = R/Z.
Discrete measure:
ma,x =
PN
i=1 ai xi , a 2 RN
, x 2 TN
Example: ideal low-pass
ˆ' = 1[ fc,fc] '(t) = sin((2fc+1)⇡t)
sin(⇡t)
y(t) =
R
T
'(x t)dm(x)
Deconvolution of Measures
ma,x
'
= 0.5/fc
y

6. y = (m)
Convolution measurements:
' 2 C2
(T)
Radon measure m on T = R/Z.
Discrete measure:
ma,x =
PN
i=1 ai xi , a 2 RN
, x 2 TN
Example: ideal low-pass
ˆ' = 1[ fc,fc] '(t) = sin((2fc+1)⇡t)
sin(⇡t)
y(t) =
R
T
'(x t)dm(x)
Minimum separation:
= mini6=j |xi xj|
! Signal-dependent recovery criteria.
Deconvolution of Measures
ma,x
'
= 0.5/fc
y
y
= 2/fc
y
= 1/fc

7. Not to be confounded with total variation of functions.
Sparse Deconvolution of Measures
Total variation of measures:
||m||TV = sup
R
dm 2 C(T), || ||1 6 1

8. Not to be confounded with total variation of functions.
Discrete measures:
Sparse Deconvolution of Measures
Total variation of measures:
||m||TV = sup
R
dm 2 C(T), || ||1 6 1
ma,x =
PN
i=1 ai xi
||ma,x||TV = ||a||1 =
PN
i=1 |ai|

9. Not to be confounded with total variation of functions.
Sparse recovery:
(P0(y))
(P (y))
Discrete measures:
Sparse Deconvolution of Measures
min
(m)=y
||m||TV
min
m
1
2
|| (m) y||2
+ ||m||TV
Total variation of measures:
||m||TV = sup
R
dm 2 C(T), || ||1 6 1
ma,x =
PN
i=1 ai xi
||ma,x||TV = ||a||1 =
PN
i=1 |ai|

10. Not to be confounded with total variation of functions.
Sparse recovery:
(P0(y))
(P (y))
! Algorithms:
[Bredies, Pikkarainen, 2010] (proximal-based)
[Cand`es, Fernandez-G. 2012] (root finding)
Discrete measures:
Sparse Deconvolution of Measures
min
(m)=y
||m||TV
min
m
1
2
|| (m) y||2
+ ||m||TV
Total variation of measures:
||m||TV = sup
R
dm 2 C(T), || ||1 6 1
ma,x =
PN
i=1 ai xi
||ma,x||TV = ||a||1 =
PN
i=1 |ai|

11. = 0.45/fc
= 0.55/fc= 0.3/fc
= 0.1/fc
Yes if > 1.85/fc.
Robustness and Support-stability
Is m0 solution of (P0(y)) for y = m0 ?
! [Cand`es, Fernandez-G. 2012]
ˆ' = 1[ fc,fc]

12. = 0.45/fc
= 0.55/fc= 0.3/fc
= 0.1/fc
How close is the solution m of (P (y + w)) to m0 ?
Yes if > 1.85/fc.
! [Cand`es, Fernandez-G. 2012]
Robustness and Support-stability
Is m0 solution of (P0(y)) for y = m0 ?
! [Cand`es, Fernandez-G. 2012]
ˆ' = 1[ fc,fc]

13. = 0.45/fc
= 0.55/fc= 0.3/fc
= 0.1/fc
Support localization.
How close is the solution m of (P (y + w)) to m0 ?
Yes if > 1.85/fc.
! [Cand`es, Fernandez-G. 2012]
! [Fernandez-G. 2012][de Castro 2012]
Robustness and Support-stability
Is m0 solution of (P0(y)) for y = m0 ?
! [Cand`es, Fernandez-G. 2012]
ˆ' = 1[ fc,fc]

14. Overview
• Sparse Spikes Deconvolution
• Robust Support Recovery
• Vanishing Derivative Certificate
• Ideal Low Pass Filter

15. Dual Certificates
S0(y) = ⌘ 2 L2
(T) ⌘ 2 Im( ⇤
) @||m?
||TV
Dual certificates of (P0(y)): for any solution m?
,
@||ma,x||TV = {⌘ 2 C(T) ||⌘||1 6 1, 8 i, ⌘(xi) = sign(ai)}

16. ⌘0 ⌘= 0.6/fc
⌘0 2 S0 ⌘ 2 S0
⌘0 ⌘= 0.7/fc
⌘ 2 Im( ⇤
) ⌘ /2 S0
Minimal-norm certificate:
p0 = argmin ||p|| s.t. ⇤
p 2 S0(y)
⌘0 = ⇤
p0 where
Dual Certificates
S0(y) = ⌘ 2 L2
(T) ⌘ 2 Im( ⇤
) @||m?
||TV
Dual certificates of (P0(y)): for any solution m?
,
@||ma,x||TV = {⌘ 2 C(T) ||⌘||1 6 1, 8 i, ⌘(xi) = sign(ai)}

17. ⌘0 ⌘= 0.6/fc
⌘0 2 S0 ⌘ 2 S0
⌘0 ⌘= 0.7/fc
⌘ 2 Im( ⇤
) ⌘ /2 S0
Minimal-norm certificate:
p0 = argmin ||p|| s.t. ⇤
p 2 S0(y)
⌘0 = ⇤
p0 where
Dual Certificates
Proposition: For any solution m of (P (y)),
p = y m !0+
! p0 in L2
(T)
S0(y) = ⌘ 2 L2
(T) ⌘ 2 Im( ⇤
) @||m?
||TV
Dual certificates of (P0(y)): for any solution m?
,
@||ma,x||TV = {⌘ 2 C(T) ||⌘||1 6 1, 8 i, ⌘(xi) = sign(ai)}

18. Non-degenerate Certificates
9p, ⇤
p 2 @||m0||TV
In finite dimension: SC(m0) () m0 solution of P0( m0).
Source condition SC(m0):

19. Non-degenerate Certificates
⌘0= 1/fc ⌘0 = 0.6/fc
9p, ⇤
p 2 @||m0||TV
In finite dimension: SC(m0) () m0 solution of P0( m0).
8 s 2 T {x0,1, . . . x0,N }, |⌘0(s)| < 1
8 i 2 {1, . . . N}, ⌘00
0 (x0,i) 6= 0
Non Degenerate Source Condition NDSC(m0):
Source condition SC(m0):
('(· x0,i), '0
(· x0,i))N
i=1 has full rank.

20. ↵, 0
= {( , w) 0 6 6 0, ||w||2 6 ↵ }
Noise Robustness
PN
i=1 ˜a ,i ˜x ,i
( , w) 2 7! (˜a , ˜x ) is C1where
Let m0 = ma0,x0 satisfying NDSC(m0).
is unique and reads
Theorem:
For ( , w) 2 ↵, 0 , the solution of (P (y + w))
0
||w||

21. ↵, 0
= {( , w) 0 6 6 0, ||w||2 6 ↵ }
In particular:
Noise Robustness
|˜x ,i x0,i| = O(||w||2)
|˜a ,i a0,i| = O(||w||2)
PN
i=1 ˜a ,i ˜x ,i
( , w) 2 7! (˜a , ˜x ) is C1where
Let m0 = ma0,x0 satisfying NDSC(m0).
is unique and reads
Theorem:
For ( , w) 2 ↵, 0 , the solution of (P (y + w))
0
||w||

22. ↵, 0
= {( , w) 0 6 6 0, ||w||2 6 ↵ }
In particular:
Noise Robustness
|˜x ,i x0,i| = O(||w||2)
|˜a ,i a0,i| = O(||w||2)
PN
i=1 ˜a ,i ˜x ,i
( , w) 2 7! (˜a , ˜x ) is C1where
Let m0 = ma0,x0 satisfying NDSC(m0).
is unique and reads
Theorem:
For ( , w) 2 ↵, 0 , the solution of (P (y + w))
˜xi,
Noiseless w = 0. ||w||/↵ 0
0 ˜xi,
0
||w||

23. Overview
• Sparse Spikes Deconvolution
• Robust Support Recovery
• Vanishing Derivative Certificate
• Ideal Low Pass Filter

24. Vanishing Derivative Pre-Certificate ¯⌘0 = ⇤
¯p0 where
¯p0 = argmin ||p|| s.t. 8 i
⇢
( ⇤
p)(x0,i) = sign(a0,i),
( ⇤
p)0
(x0,i) = 0.
Vanishing Derivative (Pre-)Certificate
⌘0= 1/fc ⌘0 = 0.6/fc
Intuition: 8 i, ⌘0
0(x0,i) = 0, where ma0,x0 solves (P0(y)).

25. Vanishing Derivative Pre-Certificate ¯⌘0 = ⇤
¯p0 where
¯p0 = argmin ||p|| s.t. 8 i
⇢
( ⇤
p)(x0,i) = sign(a0,i),
( ⇤
p)0
(x0,i) = 0.
Vanishing Derivative (Pre-)Certificate
⌘0= 1/fc ⌘0 = 0.6/fc
¯⌘0 = ⇤ +,⇤
x0
(sign(a0), 0)⇤
Proposition: if x0 has full rank,
x0 = ('(· x0,i), '0
(· x0,i))N
i=1
Intuition: 8 i, ⌘0
0(x0,i) = 0, where ma0,x0 solves (P0(y)).

26. Vanishing Derivative Pre-Certificate ¯⌘0 = ⇤
¯p0 where
¯p0 = argmin ||p|| s.t. 8 i
⇢
( ⇤
p)(x0,i) = sign(a0,i),
( ⇤
p)0
(x0,i) = 0.
Vanishing Derivative (Pre-)Certificate
⌘0= 1/fc ⌘0 = 0.6/fc
¯⌘0 = ⇤ +,⇤
x0
(sign(a0), 0)⇤
Proposition: if x0 has full rank,
x0 = ('(· x0,i), '0
(· x0,i))N
i=1
Theorem: if m0 satisfies NDSC(m0), ⌘0 = ¯⌘0.
Intuition: 8 i, ⌘0
0(x0,i) = 0, where ma0,x0 solves (P0(y)).

27. Overview
• Sparse Spikes Deconvolution
• Robust Support Recovery
• Vanishing Derivative Certificate
• Ideal Low Pass Filter

28. [Cand`es, Fernandez-G. 2012]
K(t) =
✓
sin((fc
2 +1)⇡t)
(fc
2 +1)sin(⇡t)
◆4
Fejer Kernel Pre-certificate:
ˆ⌘0(t) =
PN
i=1 (↵iK(t x0,i) + iK0
(t x0,i))
⇢
ˆ⌘0(x0,i) = sign(a0,i),
ˆ⌘0
0(x0,i) = 0.
8 i,
For m0 = ma0,x0 .
= 0.6/fc = 0.7/fc(ˆ⌘0, ¯⌘0)
Fejer Kernel Pre-certificates
ˆ' = 1[ fc,fc] '(t) = sin((2fc+1)⇡t)
sin(⇡t)
(ˆ⌘0, ¯⌘0)

29. [Cand`es, Fernandez-G. 2012]
K(t) =
✓
sin((fc
2 +1)⇡t)
(fc
2 +1)sin(⇡t)
◆4
Fejer Kernel Pre-certificate:
ˆ⌘0(t) =
PN
i=1 (↵iK(t x0,i) + iK0
(t x0,i))
⇢
ˆ⌘0(x0,i) = sign(a0,i),
ˆ⌘0
0(x0,i) = 0.
8 i,
For m0 = ma0,x0 .
= 0.6/fc = 0.7/fc(ˆ⌘0, ¯⌘0)
Fejer Kernel Pre-certificates
ˆ' = 1[ fc,fc] '(t) = sin((2fc+1)⇡t)
sin(⇡t)
Theorem: [C. F-G. 2012] If > 1.85/fc, ˆ⌘0 2 @||m0||TV.
(ˆ⌘0, ¯⌘0)

30. [Cand`es, Fernandez-G. 2012]
K(t) =
✓
sin((fc
2 +1)⇡t)
(fc
2 +1)sin(⇡t)
◆4
Fejer Kernel Pre-certificate:
ˆ⌘0(t) =
PN
i=1 (↵iK(t x0,i) + iK0
(t x0,i))
⇢
ˆ⌘0(x0,i) = sign(a0,i),
ˆ⌘0
0(x0,i) = 0.
8 i,
For m0 = ma0,x0 .
= 0.6/fc = 0.7/fc(ˆ⌘0, ¯⌘0)
Fejer Kernel Pre-certificates
ˆ' = 1[ fc,fc] '(t) = sin((2fc+1)⇡t)
sin(⇡t)
Theorem: [C. F-G. 2012] If > 1.85/fc, ˆ⌘0 2 @||m0||TV.
(ˆ⌘0, ¯⌘0)
Conjecture: ˆ⌘0 2 @||m0||TV ) ¯⌘0 2 @||m0||TV

31. Deconvolution of measures:
! L2
errors are not well-suited.
Weak-* convergence.
Optimal transport distance.
Exact support estimation.
...
Conclusion

32. Low-noise behavior: ! dictated by ⌘0.
Deconvolution of measures:
! L2
errors are not well-suited.
Weak-* convergence.
Optimal transport distance.
Exact support estimation.
...
Conclusion
= 1/fc ⌘0
⌘0= 0.6/fc

33. Extends to:
Arbitrary dimension.
Non-stationary “convolutions”.
Low-noise behavior: ! dictated by ⌘0.
Deconvolution of measures:
! L2
errors are not well-suited.
Weak-* convergence.
Optimal transport distance.
Exact support estimation.
...
Conclusion
= 1/fc ⌘0
⌘0= 0.6/fc

34. Extends to:
Arbitrary dimension.
Non-stationary “convolutions”.
Lasso on discrete grids:
Low-noise behavior: ! dictated by ⌘0.
Deconvolution of measures:
! L2
errors are not well-suited.
Weak-* convergence.
Optimal transport distance.
Exact support estimation.
...
similar ⌘0-analysis applies.
! Relate discrete and continuous recoveries.
Conclusion
= 1/fc ⌘0
⌘0= 0.6/fc