A Beginners Guide to Building a RAG App Using Open Source Milvus
Defocus Techniques for Camera Dynamic Range Expansion
1. Defocus Techniques for
Camera Dynamic Range
Expansion
Matthew Trentacoste, Cheryl Lau, Mushfiqur Rouf,
Rafal Mantiuk, Wolfgang Heidrich
University of British Columbia
2. Defocus DR expansion
• Sensorsexpanded, dynamic rangeexist
Can be
limited in
but tradeoffs
• Evaluate the scene incident onthe dynamic
range of
the opposite, reduce
the sensor
by optical blurring, restore in software
1/9 1/9 1/9 5/9 5/9 5/9
5 1/9 1/9 1/9
= 5/9 5/9 5/9
1/9 1/9 1/9 5/9 5/9 5/9
3. Approach
• Use 2 techniques to aid:
coded aperture + deconvolution
• Aperture filtermore information
PSF preserves
to improve deconvolution quality
[Rashkar 2006][Levin 2007][Veeraraghavan 2007]
• Deconvolution tousing natural image statistics
Recent advances
restore original image
[Bando 2007][Levin 2007]
4. Physical setup
• Rays from focused onto sensoraperture
plane and
scene pass through
• Cone of rays fromforming the shape of the
intersects sensor,
out-of-focus points
aperture
• A patternsensor aperture plane ispoints
onto the
in the
for out-of-focus
projected
5. Coded Aperture
• Originally from x-ray 1989]
[Fenimore 1978][Gottesman
astronomy
• Structured of pinhole, but better SNR with
resolution
arrays + decoding algorithm
• Employed in visible light photography
[Rashkar 2006][Levin 2007][Veeraraghavan 2007]
• Improve frequency properties of filter
6. Aperture filters
• What makes a good filter?
• Frequency response
• Position and spacing of zero frequencies
• Diffraction / transmission
7. Deconvolution
• Restore image distorted by PSF
[Wiener 1964][Richardson 1972][Lucy 1974]
f = f0 ⊗ k + η
• Ill-posed, infinite solutions
• No exact solution due to noise
• Division in FFT, issues with small
values in OTF of filter
8. Deconvolution
• Current state-of-the-art methods rely on natural
image statistics
• Real-worlddistribution of several properties:
Heavy-tail
images share
gradients
• Prior 2007][Levindeconvolution algorithms
[Bando
term in
2007]
• Favors interpretations fewthe image with all the
gradient intensity at a
of
pixels
9. Evaluation
• Goal : determine whether any combo of filterDR
deconvolution yields meaningful reduction in
/
with acceptable final image quality
• Measure DR reduction both in terms of image
local contrast and filter
• Measure image quality as images between
deconvolved and original
difference
10. Source material
Atrium Morning Atrium Night
Figure 3.3: Sample images used in evaluation.
Radius Atrium Morning Atrium Night
min max reduction min max reduction
Original 0.00 11.0 0.00 12.0
1 0.00 10.8 0.200 0.452 12.0 0.452
2 0.00 10.6 0.424 0.622 12.0 0.622
3 0.00 10.3 0.716 1.163 11.8 1.34
4 0.02 10.0 1.00 1.436 11.4 1.99
5 0.08 9.94 1.14 1.589 11.4 2.23
6 0.15 9.92 1.24 1.731 11.2 2.51
8 0.31 9.83 1.48 1.890 10.8 3.13
9 0.40 9.79 1.61 1.950 10.5 3.41
11 0.66 9.71 1.94 2.08 10.3 3.74
13 0.86 9.67 2.19 2.18 10.1 4.13
16 1.04 9.59 2.45 2.26 9.61 4.65
Figure 3.4: Amount of reduction in dynamic range as a function of radius of a standard aperture (disk)
filter in pixels. All units are in terms of powers of two, referred to as exposure value (EV) stops.
Atrium Morning Atrium Night
11. Source material
Atrium Morning Atrium Night
Figure 3.3: Sample images used in evaluation.
Radius Atrium Morning Atrium Night
min max reduction min max reduction
Original 0.00 11.0 0.00 12.0
1 0.00 10.8 0.200 0.452 12.0 0.452
2 0.00 10.6 0.424 0.622 12.0 0.622
3 0.00 10.3 0.716 1.163 11.8 1.34
4 0.02 10.0 1.00 1.436 11.4 1.99
5 0.08 9.94 1.14 1.589 11.4 2.23
6 0.15 9.92 1.24 1.731 11.2 2.51
8 0.31 9.83 1.48 1.890 10.8 3.13
9 0.40 9.79 1.61 1.950 10.5 3.41
11 0.66 9.71 1.94 2.08 10.3 3.74
13 0.86 9.67 2.19 2.18 10.1 4.13
16 1.04 9.59 2.45 2.26 9.61 4.65
2.45 EV
Figure 3.4: Amount of reduction in dynamic range as a function of radius of a standard aperture (disk)
filter in pixels. All units are in terms of powers of two, referred to as exposure value (EV) stops.
Atrium Morning Atrium Night
12. Source material
Atrium Morning Atrium Night
Figure 3.3: Sample images used in evaluation.
Radius Atrium Morning Atrium Night
min max reduction min max reduction
Original 0.00 11.0 0.00 12.0
1 0.00 10.8 0.200 0.452 12.0 0.452
2 0.00 10.6 0.424 0.622 12.0 0.622
3 0.00 10.3 0.716 1.163 11.8 1.34
4 0.02 10.0 1.00 1.436 11.4 1.99
5 0.08 9.94 1.14 1.589 11.4 2.23
6 0.15 9.92 1.24 1.731 11.2 2.51
8 0.31 9.83 1.48 1.890 10.8 3.13
9 0.40 9.79 1.61 1.950 10.5 3.41
11 0.66 9.71 1.94 2.08 10.3 3.74
13 0.86 9.67 2.19 2.18 10.1 4.13
16 1.04 9.59 2.45 2.26 9.61 4.65
2.45 EV 4.56 EV
Figure 3.4: Amount of reduction in dynamic range as a function of radius of a standard aperture (disk)
filter in pixels. All units are in terms of powers of two, referred to as exposure value (EV) stops.
Atrium Morning Atrium Night
13. Tests
• Filters evaluated: • Deconvolution evaluated:
• Normal aperture • Wiener filtering
• Gaussian • Richardson-Lucy
• Veeraraghavan • Bando
• Levin • Levin
• Zhou
14. Evaluation (cont)
• Success criteria:
• Reduction of computational cost of deconv
to justify the
at least 2 stops
• Quality of at least PSNR 35
22. Conclusions
• Levin deconv at very low noise levels with
coded filters
the best, obtaining results
• No combination of filter and deconvolution
consistently produced acceptable results
• Efficiency of the approach is scene dependent
Most efficient for small, isolated bright regions
Editor's Notes
Can be expanded by multiple exposures, new filter arrays, or better sensor tech
Blurring causes pixels to distribute energy over a local neighborhood
Reducing local contrast
Depending on image structure, can translate to reduction in global contrast
Images with small features: good
Images with large features: not good
Conv = FFT mult -> Deconv = FFT div -- properties of filter influence ability to deconvolve
Restore image convolved by a known function degraded by noise - Ill posed, numerous solutions
Real world images all share several properties - specifically the distribution of gradient intensity
Surfaces = large regions of flat intensity with sharp changes - mostly small changes but some very large
FFT of a conventional aperture is roughly a sinc function
Information loss
How well it preserves the information of the signal
Physical shape of pattern and whether it causes more diffraction
The more light it lets through the better
Heavy-tail = most values near zero, but a few with much high values
Narrower peak, and wider tail than a Gaussian
Results in sharper images with less noise and ringing
Blurring decreases local contrast
Image structure determines how much global contrast is reduced
Small features reduce more than large ones
CAN ONLY REDUCE CONTRAST OF FEATURES SMALLER THAN PSF DIAMETER
Done in simulation - evaluate best case
Change in dynamic range as each image is blurred by different filter radii
Size of bright and dark features affects how much dynamic range is reduced
Change in dynamic range as each image is blurred by different filter radii
Size of bright and dark features affects how much dynamic range is reduced
2 stops to justify computational cost -- Green area denotes acceptable by our criteria
Levin performs the best when there is no noise
Levin and Zhou perform best overall
Gaussian is worst - destroys too much information
Noise sensitivity of Weiner becomes apparent
Levin performs best in morning scene, RL wins out for night
Levin yields sharper results, but introduces more ringing - bright points ruin shadow detail
Levin and Zhou perform slightly better in the morning scene
All same in the night
Investigate deconvolution routines that are better able to handle the relative differences of HDR images
