Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Developing VR in Unity

A short overview on VR, developing it in Unity and various tips for interaction and performance.

This is presented in ProCodeCG's codeMeetUp() on 15 August 2016

Related Books

Free with a 30 day trial from Scribd

See all

Developing VR in Unity

  1. 1. DEVELOPING VR IN UNITY ADITYO PRATOMO - LABTEK INDIE
  2. 2. DEVELOPING VR IN UNITY ABOUT ME ▸ Chief Technology Officer at Labtek Indie ▸ Chief Academic Officer at Froyo Framework ▸ Certified Unity Developer #20167UCD724 ▸ Loves to tinker with tech
  3. 3. DEVELOPING VR IN UNITY VR IN LABTEK INDIE It’s Strange (2015) Oculus Rift & Leap Motion, developed in Unity https://irunsiregar.itch.io/its-strange https://www.youtube.com/watch?v=oTii7SSfyLM
  4. 4. DEVELOPING VR IN UNITY VR IN LABTEK INDIE Combat League (2016) Oculus Rift, developed in Unreal Engine https://www.youtube.com/watch?v=1sHrSOe8JM8
  5. 5. DEVELOPING VR IN UNITY VR IN LABTEK INDIE 99 Names (2015) Web VR, developed in JS with ThreeJS 99names.adityo.net
  6. 6. DEVELOPING VR IN UNITY VR IN LABTEK INDIE Mixed Reality Experiment Google Cardboard devloped in Unity with Vuforia SDK http://www.adityo.net/?p=1146
  7. 7. DEVELOPING VR IN UNITY WHAT I’LL TALK ▸ VR Tech Overview ▸ Desktop VR vs Mobile VR ▸ VR in Unity ▸ Interaction in VR ▸ Towards Better VR Experience ▸ Future Prediction
  8. 8. NOW IS THE TIME TO START MAKING SOMETHING IN VR, EVEN IF YOU DON'T PLAN ON LAUNCHING IT. IT IS A REALLY GOOD IDEA TO START EXPERIMENTING AND SEE WHAT YOU CAN LEARN. EVEN IF YOU'RE NOT GOING TO LAUNCH SOMETHING, IT IS GOOD TO HAVE EXPERIENCE IN VIRTUAL REALITY SO THAT IF IT MAKES SENSE FOR YOU TO GET INTO IT, YOU'RE ABLE TO Palmer Luckey (Co-founder of Oculus) DEVELOPING VR IN UNITY
  9. 9. DEVELOPING VR IN UNITY VR TECH OVERVIEW ▸ VR is a tech that creates new perception of reality in the mind of its users ▸ Achieved through the use of HMD (Head Mounted Display) that tricks its user into thinking he/she is in a different place ▸ Closely related to AR, but not similar nor has hierarchical relation
  10. 10. DEVELOPING VR IN UNITY VR TECH OVERVIEW
  11. 11. DEVELOPING VR IN UNITY VR TECH OVERVIEW 1. Lens ‣ Emulating depth via focal point adjustment 2. Display ‣ Stereo rendering ‣ FOV emulates real life view 3. Tracking Technology ‣ Tracking head movement ‣ Uses gyro and accelerometer in smartphone 4. 3D Audio ‣ Immersive 3D audio
  12. 12. DEVELOPING VR IN UNITY CURRENT VR PLATFORMS PC ANDROID CONSOLE OCULUS RIFT HTC VIVE GOOGLE VR (CARDBOARD & DAYDREAM) SAMSUNG GEAR VR PS VR OSVR
  13. 13. DEVELOPING VR IN UNITY DESKTOP VR VS MOBILE VR: FROM THE TECH SIDE ▸ Desktop VR requires more horse power, but has potential for many kind of interaction and contents ▸ Mobile VR experience is hampered by the device’s spec ▸ Desktop VR can do positional tracking ▸ Mobile VR can only do head positional tracking ▸ Batteries vs Cable ▸ Both do VR content rendering in roughly the same way
  14. 14. DEVELOPING VR IN UNITY DESKTOP VR VS MOBILE VR: FROM THE BUSINESS SIDE ▸ Mobile VR has lower entry barrier, mid-high end Android phone is sufficient ▸ Desktop VR requires high end gaming rig ▸ Mobile VR has way larger potential market than Desktop ▸ Desktop VR will interests hardcore gamer ▸ Mobile VR will interests budget-oriented gadget enthusiasts
  15. 15. DEVELOPING VR IN UNITY DESKTOP VR VS MOBILE VR ▸ Mobile VR will help promoting VR to the masses ▸ Desktop VR will help pushing the boundaries, tech wise ▸ Both is ready for mass development, NOW!
  16. 16. DEVELOPING VR IN UNITY WHY DEVELOP VR NOW? ▸ Relatively new tech ▸ Still no best practice sets in stone ▸ Experiments are highly appreciated ▸ Big or small developers, everybody are still finding its feet ▸ YOU can make the difference
  17. 17. DEVELOPING VR IN UNITY VR IN UNITY PC OCULUS RIFT HTC VIVE OSVR OCULUS RIFT DK2 / CV1 WIN 7+ UNITY 5.X
  18. 18. DEVELOPING VR IN UNITY VR IN UNITY PC OCULUS RIFT HTC VIVE OSVR HTC VIVE WIN 7+ UNITY 5.X
  19. 19. DEVELOPING VR IN UNITY VR IN UNITY PC OCULUS RIFT HTC VIVE OSVR OSVR HMD WIN 7+ UNITY 5.X
  20. 20. DEVELOPING VR IN UNITY VR IN UNITY ANDROID GOOGLE VR (CARDBOARD & DAYDREAM) SAMSUNG GEAR VR ANDROID VR HMD ANDROID 5.1 + UNITY 5.X GOOGLE CARDBOARD SDK NEXUS 6 (DAYDREAM)
  21. 21. DEVELOPING VR IN UNITY VR IN UNITY ANDROID GOOGLE VR (CARDBOARD & DAYDREAM) SAMSUNG GEAR VR ANDROID VR HMD ANDROID 5.1 + UNITY 5.X VUFORIA SDK
  22. 22. DEVELOPING VR IN UNITY VR IN UNITY ANDROID GOOGLE VR (CARDBOARD & DAYDREAM) SAMSUNG GEAR VR ANDROID 5.1 + UNITY 5.X SAMSUNG GALAXY S7 SAMSUNG GEAR VR
  23. 23. DEVELOPING VR IN UNITY VR IN UNITY: GENERAL ARCHITECTURE ▸ Default camera is replaced by vendor-specific VR camera (usually comes as prefab) ▸ VR Camera will render the game scene to be VR-ready ▸ Other game objects are used as usual GAME SCENE VR CAMERA OTHER GAME OBJECTS
  24. 24. DEVELOPING VR IN UNITY VR IN UNITY: TIPS FOR THE FIRST TIMER ▸ Most of the time, you’ll wrestle with setting up specific environment ▸ Oculus: Get the latest SDK and fulfil the requirements, use Rift DK2 at least, don’t use laptop ▸ Android: setup Android SDK properly, Google and Vuforia SDKs are generally easy to set up
  25. 25. DEVELOPING VR IN UNITY VR IN UNITY ▸ After replacing the Main Camera object with the vendor- specific, AR Camera prefab, you’re done. ▸ EASY! ▸ Demo…
  26. 26. WHAT ELSE TO DO? if getting VR to up and run is easy, then
  27. 27. LET’S TALK ABOUT INTERACTION
  28. 28. DEVELOPING VR IN UNITY INTERACTION IN VR ▸ With headset strapped on, it’s hard to look around ▸ Mouse + keyboard, won’t be the best way to interact
  29. 29. DEVELOPING VR IN UNITY INTERACTION IN VR LOOK AROUND GAMEPAD INTERACTION GAZE INTERACTION NATURAL INTERACTION (AVAILABLE NOW) OCULUS TOUCH VIVE CONTROLLER GOOGLE DAYDREAM LEAP MOTION EXPERIMENTAL INTERACTION (DIY) VOICE CONTROLLER CUSTOM ELECTRONIC (ARDUINO / RPI / ETC) ? easy to develop, minimal interaction harder to develop, rich interaction
  30. 30. DEVELOPING VR IN UNITY INTERACTION IN VR: LOOK AROUND ▸ The most basic thing you can do inside VR ▸ Replacing traditional FPS camera, controlled by mouse, with your head ▸ Results in a natural, immersive experience ▸ Used in many VR videos ▸ Minimum interaction, but can still produces excellent experience
  31. 31. DEVELOPING VR IN UNITY 360 PHOTOS / VIDEOS IN UNITY ▸ Generally speaking, here are the steps: ▸ Obtain 360 photo/video ▸ Make a skybox using sphere/cube ▸ Use the photo/video as the texture for the skybox
  32. 32. DEVELOPING VR IN UNITY 360 PHOTOS / VIDEOS IN UNITY
  33. 33. DEVELOPING VR IN UNITY 360 PHOTOS / VIDEOS IN UNITY ▸ Plugin: ▸ Easy Movie Texture (https://www.assetstore.unity3d.com/en/#!/ content/10032) ▸ Discussions: ▸ http://bernieroehl.com/360stereoinunity/ ▸ http://forum.unity3d.com/threads/projecting-a-360-degree-vr- video-on-a-sphere.376522/ ▸ http://talesfromtherift.com/use-a-360-degree-panorama-as-a- skybox/
  34. 34. DEVELOPING VR IN UNITY GAZE INTERACTION ▸ Using eye as an input ▸ Activating an object by looking at it for a certain period ▸ (Optional) use Gaze Reticle to help user find it’s view’s crosshair
  35. 35. DEVELOPING VR IN UNITY GAZE INTERACTION
  36. 36. DEVELOPING VR IN UNITY GAZE INTERACTION ▸ Google Cardboard SDK: ▸ Use GvrGaze script ▸ Drag the script from Assets/GoogleVR/Scripts/UI/ GvrGaze onto to the camera ▸ Implement IGvrGazePointer and IGvrGazeResponder to the interactive object ▸ Vuforia SDK uses Google’s Gaze
  37. 37. DEVELOPING VR IN UNITY NATURAL INTERACTION WITH LEAP MOTION ▸ Leap Motion offers the ability to track hand gestures ▸ It can be sticked in front of Oculus/Vive to enable user to see his/her hands ▸ Thus, enables a natural interaction using hands, while inside VR
  38. 38. DEVELOPING VR IN UNITY NATURAL INTERACTION WITH LEAP MOTION
  39. 39. DEVELOPING VR WITH UNITY NATURAL INTERACTION WITH LEAP MOTION ▸ Download and Import the Leap Motion Core Assets Unity Package ▸ Use the LeapOVRPlayerController Prefab from the Leap Motion + OVR directory as the main VR Camera ▸ The hand will be added to the screen and acts as a 3D objects, just like the others (with colliders too!)
  40. 40. DEVELOPING VR IN UNITY EXPERIMENTAL INTERACTION: VOICE CONTROL ▸ Use voice to give input to the VR world ▸ Closely tied to the deployment platform ▸ iOS, Android, Windows, Mac, has its own voice recognition system ▸ Unity 5.4 can natively do voice recognition in Windows (https://developer.microsoft.com/en-us/windows/ holographic/voice_input_in_unity)
  41. 41. DEVELOPING VR IN UNITY EXPERIMENTAL INTERACTION: VOICE CONTROL ▸ Unity-only solution: ▸ Ispikit: https://github.com/ispikit/ispikit-unity ▸ Word Detection: https://www.assetstore.unity3d.com/en/#!/ content/4518 ▸ Unity + other: ▸ Read Serial in Unity ▸ Do voice recognition in other using Python/NodeJS/etc and send serial from there
  42. 42. DEVELOPING VR IN UNITY EXPERIMENTAL INTERACTION: CUSTOM ELECTRONIC ▸ If you have something different, why not make it yourself ▸ Use Arduino / Raspberry Pi / etc. to read from certain sensor and send its data to Unity ▸ Vital to create haptic feedback and experimenting with different kinds of interaction ▸ Make one of a kind immersive experience
  43. 43. DEVELOPING VR IN UNITY EXPERIMENTAL INTERACTION: CUSTOM ELECTRONIC
  44. 44. DEVELOPING VR IN UNITY EXPERIMENTAL INTERACTION: CUSTOM ELECTRONIC ▸ Use Unity to receive Serial data ▸ Android & Microcontroller Bluetooth: https:// www.assetstore.unity3d.com/en/#!/content/16467 ▸ SimpleSerialPort: https://www.assetstore.unity3d.com/en/#!/ content/53594 ▸ Attach sensors to Arduino/Raspberry Pi/other ▸ Send data via Serial/Bluetooth ▸ Watch out for latency!
  45. 45. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Simulation sickness is VR’s biggest enemy ▸ Cause: brain is tricked to feel movement, while static ▸ When that trick is broken, nausea attacks, users opt out from the experience (and probably won’t come back)
  46. 46. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Technical causes: ▸ Latency (maximum should be 20ms) ▸ Many research has gone into how to solve this issue ▸ Many solutions, but none has quite hits the mark
  47. 47. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Hardware design solution: ▸ More comfortable lens ▸ More comfortable HMD when strapped on
  48. 48. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Content design: ▸ Adding nose and feet ▸ Shows a fixed reference point in a frame
  49. 49. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Game design - movement: ▸ In-game movement have more latency potential than standing still ▸ objects are continuously and dynamically rendered in real- time ▸ more processing power ▸ To move around, use teleport ▸ Teleport can be done with gaze interaction, no controller is required
  50. 50. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS Keep Talking and Nobody Explodes www.keeptalkinggame.com
  51. 51. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Game design - movement: ▸ To move around: ▸ use teleport ▸ use pre-determined route ▸ Teleport can be done with gaze interaction, no controller is required ▸ Pre-determined route will ease us in optimizing assets ▸ Don’t add unnecessary vertical movement
  52. 52. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS Teleport: https://developer.oculus.com/blog/introducing-the- oculus-sample-framework-for-unity-5/ INMIND VR: https://developer.oculus.com/blog/introducing-the- oculus-sample-framework-for-unity-5/
  53. 53. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Game design - camera view: ▸ It’s natural to have first person view, but it’s easier to result in simulation sickness ▸ To reduce that, use third person view, it will still have good experience
  54. 54. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS Chronos: www.chronosvr.com Colosse: colossevr.tumblr.com
  55. 55. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Tech tips: ▸ Use low poly objects, with high-res textures to compensate ▸ Toon-shading is okay, photorealistic can probably causes latency
  56. 56. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS Job Simulator: www.jobsimulatorgame.com Land’s End: www.landsendgame.com
  57. 57. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS ▸ Tech tips: ▸ Use as many static object as possible (uses least RAM) ▸ Use LOD, culling and batching to prevent rendering too many object at the same time ▸ Use as many baked lighting as possible ▸ https://unity3d.com/learn/tutorials/topics/virtual-reality/ optimisation-vr-unity
  58. 58. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
  59. 59. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE ▸ Experience design: ▸ Design experience from the ground up, don’t just adapt already available 3D game to VR ▸ VR is now best experienced in short but meaningful time (5-20 minutes) ▸ Design focal point, because the user can look into any direction
  60. 60. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE ▸ UI design: ▸ Make sure the user can see the UI at any time ▸ Go for “no-UI” style, embed the required information in the objects, just like in normal life ▸ Enable the player to look around, even when in pause mode
  61. 61. DEVELOPING VR IN UNITY TOWARDS BETTER VR EXPERIENCE
  62. 62. DEVELOPING VR IN UNITY FUTURE PREDICTION ▸ Lower cost of desktop VR ▸ WebVR ▸ VR beyond entertainment
  63. 63. AT THE SAME TIME, I'D SAY DEVELOPERS SHOULD BE REALISTIC. DON'T BELIEVE THE MOST OPTIMISTIC THINGS THE ANALYSTS SAY. ESPECIALLY WHEN THOSE ANALYSTS DON'T ACTUALLY KNOW ANYTHING ABOUT VR. IT'S GOING TO TAKE TIME FOR VIRTUAL REALITY TO BECOME TRULY MASS MARKET AND SUCCESSFUL, SO PEOPLE SHOULD SCALE THEIR EFFORTS APPROPRIATELY. Palmer Luckey (Co-founder of Oculus) DEVELOPING VR IN UNITY
  64. 64. THANK YOUdidit@labtekindie.com @kotakmakan

×