4. SLAM
● Simultaneous localization and mapping
● Given a series of sensor observations ot over
discrete time steps t, the SLAM problem is to
compute an estimate of the agent's location xt
and a map of the environment mt
14. ORB-SLAM2
● Advantage
○ Open source in C++
○ Strong MonoSLAM
○ Nice demo video
● Disadvantage
○ Cost time to understand details/algorithm
○ MonoSLAM is good only with 2D movement
https://github.com/raulmur/ORB_SLAM2
18. Implementation device
● Kinect
○ Mono camera + Depth sensor
● Mobile
○ Mono camera + gyro sensor + accelerometer
● Stereo camera
○ Able to estimate depth map
=> Mobile
19. Implementation technology
● Extendable with new technology / open source
● Save money
● Able to implement new algorithm
=> OpenCV (C++) + Unity + Android
21. Tutorial Unity + Android + OpenCV C++
● C++ with Unity
● OpenCV C++ with Unity Android
● Object Recognition with Unity Android
22. C++ with Unity
● Write code in C++
● Build as Shared Object (in Visual Studio 2015)
● Add builded file to Unity as library and call in
Unity
● Can Marshall simple object from C# to C++ and
from C++ to C#
23. C++ with Unity
● Is it able to pass by reference from C# to C++ ?
○ Yes, use pointer in C++ and ref/out keyword in C#
● Can I pass an array from C# to C++ ?
○ Yes, you can pass array of primitive type from C# to
C++ (I did not try if array struct type)
● Can I receive an array from C++ to C# ?
○ Yes, you can. There are 2 ways to archive
○ Void function with pass by reference
○ Pointer-returned function with marshall in C#
24. OpenCV with Unity Android
● Download opencv4android from OpenCV
homepage
● Create Shared Object project with OpenCV config
● Write your image processing code
● Build your source code in ARM -> .so file
● Add .so file to Unity as library
25. OpenCV with Unity Android
● How to pass image ?
○ Unity image: RGBA type (not RGB)
○ Can pass as unsigned char* array or Color32 array
○ Receive raw data and re-create image with OpenCV
Mat
● How to process image ?
○ Cannot use opencv.hpp
○ Use core.hpp, imgproc.hpp, highgui.hpp instead
○ Be careful about memory management !!!
28. OpenCV Object Recognition With Unity
● Use objdetect.hpp
● Using Cascade to detect object
○ How to load Cascade data ???
29. OpenCV Object Recognition With Unity
● Can I read files in Unity/apk from C++ ?
○ No, you can’t
● How can I load cascade file to use in OpenCV ?
○ Read cascade xml file by using unity to string
○ Pass string to C++
○ Use cv::FileStorage to load it in memory
○ Init cascade classifier by read the FileStorage created
above
32. OpenCV Object Recognition With Unity
● Why face detection cascade is too slow ?
○ Smartphone camera has high resolution (720p);
therefore, process on this image is really slow
● How can I speed up the detection process ?
○ Detect on low resolution image, then convert the result
and display on high resolution image
○ Use ORB/FAST feature to detect (really hard)