1. Homework on multiview
computer vision
Victor Lempitsky (Skoltech) for the
“Computational Camera” class
“A taster of multiview computer vision”

2. Video example
“A taster of multiview computer vision”

3. Pipeline
1.
2.
3.
4.
5.
6.
Take photographs
Detect/describe features
Match features
Estimate homography
Generate a sequence of blended frames
Write a video
“A taster of multiview computer vision”

4. Take photographs
1. Take a sequence that will blend nicely
2. Remember that the algorithm will
find the dominant matching plane
3. Downsample to 1-2Mpix
“A taster of multiview computer vision”

5. Feature Detection/Description
Detecting features:
FeatureDetector::create (SIFT)
FeatureDetector::detect
Describing features:
DescriptorDetector::create (SIFT)
DescriptorExtractor::compute
“A taster of multiview computer vision”

6. Reminder
=> Feature
vector (128D)
“A taster of multiview computer vision”

7. Matching features
1. DescriptorMatcher::knnMatch(k=2)
2. Take only matches, where the distance
to the first neighbor is less then e.g.
0.7*distance to the second neighbor
3. (Do matching the opposite way and keep
only reciprocal matches)
4. Use “drawMatches” to debug
5. Use FlannBasedMatcher if too slow
“A taster of multiview computer vision”

8. Matching results
“A taster of multiview computer vision”

9. Estimating homography
• You need findHomography (calib3d
module)
• Use CV_RANSAC option
• Play with ‘reprojThreshold’
• Apply the estimated homography to
the candidate matches and store the
verified matches for later
• Use to Mat::inv() to find the reverse
mapping
“A taster of multiview computer vision”

10. Reminder how to apply homography
“A taster of multiview computer vision”

11. Image blending
1. Use warpPerspective() to map images
2. Use image arithmetics (‘+’,’*’) to blend
3. Define intermediate frames (e.g. by
moving the verified matches along the
trajectories and refitting homographies
without RANSAC)
4. Use VideoWriter (highgui) to do the video
writing
“A taster of multiview computer vision”