A tutorial given at MGIA, SIGGRAPH Asia 2013 on building mobile AR application using the Outdoor AR Library.

1. A Step-by-step guide to
Building a Mobile Outdoor AR
Application
Gun Lee
21 Nov. 2013

2. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Outdoor AR Browsers
Wikitude
Layar
Junaio

3. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
SW Dev. Tools & Frameworks
 High-level content description
 AR Browsers
 Hard to customize app logic and UI
 Low-level functional modules
 Programming libraries
 Lack of higher-level abstraction of AR scene
and content (BYO)

4. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Outdoor AR Framework
 High-level abstraction and low-level
programming support in a seamless way.
 Low-level functional modules
- Flexible and highly customizable
 Ready-to-use high level components
- Focus on building the interface, behavior and
content of the application rather than system
integration
 The high-level and low-level tools share the
common scene (or POI) data structure
- Both skilled and novice developers can work
together under the same framework.

5. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Apps built with OAR Framework
[CityViewAR 2011]
[CCDU 3D 2012]
[GeoBoids 3D 2012]
[AntarcticAR 2012]

6. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Outdoor AR Framework
 Covers full tier of ready-to-use
components, yet highly customizable
Mobile Device (Client)
Server
Mobile AR Application
Application Service
Mobile SW Library
Data
Communication
Server SW Library
(Optional)
Scene
data
Media
data
Mobile OS
Scene
data
Media
data
Server OS

7. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Server Software and Tools
Mobile Client
(App)
Server
Web Client
(Browser)
AR View
Web-based
Authoring Tool
HTML
Data
Manager
RESTful API
Javascript
Local
cache
Scene
Database

8. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Server Software and Tools
 Web-based Authoring Tool
 Please sign up for closed beta – Jan 2014

9. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Outdoor AR Library
 For mobile app (client)
 Android (v2.2 or above)
 Google Maps API
 Scenegraph based rendering engine
 OpenGL ES (1.1 and 2.0)
 Free download from our website:
http://www.hitlabnz.org/mobileAR

10. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Outdoor AR Library Structure
Service Component Layer
Views
Controllers
Models
Map View
AR View
List View
Map
Component
AR
Component
List
Component
Scene Data Manager
Tracking Data Manager
Functional Component Layer
Scene Data
Structures
Tracking
Sensors
3D Graphics
Rendering
3D
Sound
UI Views
Tools &
Utilities

11. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Model-View-Component
Controller
Application Logic &
Data Binding
Model
View
Scene & Tracking
Data Managers
Visualization &
UI Elements

12. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Mobile App Software Library
 Customization by extending components
 Customized AR scenes
- Customize scene class data structure
 Customized user interface
- Customize UI layout
- Customize UI behavior (e.g. onSceneSelected)
- Motion and touch gesture based interaction

13. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Step-by-step Guide





Setup the development environment
Create an Android app
Using the Outdoor AR Library in your app
Adding outdoor AR scenes with 3D models
Customizing UI
 Run your app on a Google Glass

14. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Setup the dev environment
 Prerequisite
 Android Development Environment
- http://developer.android.com/sdk/index.html
- Eclipse, Android SDK
 Google Play Services Library (for maps)
 Download the Outdoor AR Library
 http://www.hitlabnz.org/mobileAR
- Download, Tutorials, Forum

15. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Setup the dev environment
 Import the library and sample projects
 OutdoorARLibrary
 SampleOAComponent
 Dependency
 Google (not Android) API 4.1.2 (API Level 16)
 Google Play Services Library
- extras/google/google_play_services/libproject
 Android Support Library 4
- extras/android/compatibility/v4/
android-support-v4.jar

16. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Create an Android app
 Create a new Android Application project
 Blank Activity
- “An activity is a single, focused thing that the user
can do.” – Android SDK API Reference
 Anatomy of an Android App project




src
res & gen
assets
AndroidManifest.xml
 Add a “Hello MGIA” button

17. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Using the OA Library in your app
 Add the Outdoor AR Library to your
project’s build path
 Add permissions in the manifest
 Add a new AR view (activity)
 Create a custom class inheriting
OAARComponentBase
 Add activity to the manifest
- Full screen landscape style preferred
 Wire a button to start the AR view

18. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Adding scenes with 3D models
 Preparing 3D model
 OBJ/MTL with JPEG or PNG textures
 OpenGL Coordinate frame
- X-right, Y-up, Z-out from screen
- North = -Z
 Placing the 3D model in the real world
 Latitude, Longitude
- http://itouchmap.com/latlong.html
 Optionally, elevation from the ground plane
 SketchUp – 3D model on Google Earth

19. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Adding scenes with 3D models
 Adding 3D models to the project
 assets/OutdoorAR/3Dmodels(/name/model.obj)
 Adding scenes to the data manager
 Override onSetupScenes()
 Handling the scene selection event
 Override onSceneSelected()
 Using a mock location
 Override onCreate()
 Sensor manager’s enableMockLocation()

20. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Loading and managing scenes
 Add scenes by code – difficult to manage
 Load from assets or local file storage
 OADataManagerAssets
 OADataManagerLocal
 SQLite or XML file
 SQLite Database Browser
 Default path
- (assets/)OutdoorAR/scenes.db
- (assets/)OutdoorAR/scenes.xml

21. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Custom scene class
 OAScene
 Lat/Lon, 3D model, Name, Description,
Category
 Custom scene classes extend OAScene
 Override OADataManager’s loadScene()
to load in custom properties of your scene
 Cast OAScene to your custom class in
onSceneSelected() or onTouchedScene()
 Different scene classes can live together
- if(scene instanceof MyCustomScene)

22. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Customizing UI
 Add layout resource
 Override setupUILayout()
 LayoutInflater - read in the layout res file
 addContentView() - overlay UI layout on top
of AR view
 findViewById() - get reference to the UI
elements
 onSceneSelected() vs. onTouchedScene()

23. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Map and List views
 OAMapComponentBase, OAListComponentBase
 Similar class structure with OAARComponentBase
- Load scenes from a data manager
- setupXYZ() methods
- onSceneSelected() callback
 Map components needs API key
 Google Maps Android API v2
- OAGoogleMapsV2ComponentBase
 https://developers.google.com/maps/documentation/an
droid/start

24. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Run your app on Google Glass
 Startup with AR
 Set AR view as a launcher activity in the app
manifest file
 Aim to select
 Use Timer to trigger touch events at the
center of the screen
 Override onResume() and onPause() to
start/stop the timer

25. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Other tips for Google Glass
 Touch gestures
 onGenericMotionEvent()
 Swipe forward/backward/up
 Tap, double tap, tap and hold, etc.
 Head motion
 Add orientation listener to sensor manager
- getSensorManager().addSensorListener()
 Avoid conflicts with AR scene viewing motions
- Time-outs

26. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Other tips for Google Glass
 GPS sensor blocked
 Stream location from a phone through
Bluetooth
 Mock location
 Turning off camera background
 Application dependent
 Override setupOptions()
 enableCameraBackground()

27. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
CityViewAR for Glass
Demo

28. Thank you!

29. SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
More Information
 Website
 http://www.hitlabnz.org/mobileAR
 http://arforglass.org
 Gun Lee
 gun.lee@hitlabnz.org
 Mark Billinghurst
 mark.billinghurst@hitlabnz.org