23. and 0 < ⌧ < 2/L, then f(k) k!+1
! f?
a solution of min
f
J(f).
If f is convex, C1
, rf is L-Lipschitz,Theorem:
f(k+1)
= f(k)
⌧krJ(f(k)
) f(0)
is given.
Gradient Descent
24. and 0 < ⌧ < 2/L, then f(k) k!+1
! f?
a solution of min
f
J(f).
If f is convex, C1
, rf is L-Lipschitz,Theorem:
f(k+1)
= f(k)
⌧krJ(f(k)
) f(0)
is given.
Optimal step size: ⌧k = argmin
⌧2R+
J(f(k)
⌧rJ(f(k)
))
Proposition: One has
hrJ(f(k+1)
), rJ(f(k)
)i = 0
Gradient Descent
43. Mask M, = diagi(1i2M )
Example: InpaintingFigure 3 shows iterations of the algorithm 1 to solve the inpainting problem
on a smooth image using a manifold prior with 2D linear patches, as defined in
16. This manifold together with the overlapping of the patches allow a smooth
interpolation of the missing pixels.
Measurements y Iter. #1 Iter. #3 Iter. #50
Fig. 3. Iterations of the inpainting algorithm on an uniformly regular image.
5 Manifold of Step Discontinuities
In order to introduce some non-linearity in the manifold M, one needs to go
log10(||f(k)
f( )
||/||f0||)
k k
E(f(k)
)
M
( f)[i] =
⇢
0 if i 2 M,
f[i] otherwise.